We Keep Coding

Reading and writing to the same file from different programs

I have two programs written in python and converted to one-file exe using auto-py-to-exe.
the first program writes to a file, which is read by the second program. The problem is when the second program wants to read the file the same time as it is being written, the code stops with a permission error.
The solutions that seemed to work are:
Using time management which is not useful in my case, since the reading and writing times are not constant.
I could check if the file is accessible, which might be a solution, however, I suppose it would raise an error if while reading the file, the writer tries to change the file.
I could use the size of the file to check if writing to the file has been finished, and then execute the reader, however, this does not seem to be both logical and pythonic!
I found some solutions using os.pipe(), but to be honest, I couldn't understand what the process does. If this is a solution, I would be glad to have it explained in simple English.
That's it. Any suggestions?
P.S: OS is windows and I am using Python 3.9
Solved:
Thanks to the replies and suggestions, I didn't know that the try except commands accept ErrorType. Thus, I solved the problem by using 'except' and 'PermissionError'. the code runs in a loop and it is checked again in a few seconds.
However, the drawback is this: the reading time should be less than the time the writer comes back to rewrite the file! In my case, as suggested by friends, I combined the two programs so they are run sequentially.

Opening a file using a specific program doesn't produce output

First of all, I am new to coding, and I have looked for similar questions and I know some commands which can be used in Python and in CMD to open a file using an executable. The problem I'm having is that when I try to use those commands, the program runs without any error, but it doesn't give the output file that should give. On the other hand, when I just double click the file, which is set to open with *.exe, it works and produces the output.
I tried the CMD command:
start "path of .exe" "filepath"
also just:
"path of .exe" "filepath"
Then I tried the os and subprocess modules in python:
subprocess.Popen(...)
os.system(...)
subprocess.run(...)
and many other solution I found on the internet. The point is that all of these solution don't give errors and should work, but they don't produce the wanted file for me. I used the same commands on another file with a different .exe and they work.
This is the step that is not working in my attempt to automate a whole process. If someone is willing to take a look at the files, you can download them from the following link:
https://gofile.io/?c=5TJtS9
The files are as follows:
1. running the rdam.grd file with hist_dam2d.exe produces the hist.plotps file
2. then running the hist.plotps with plotps.exe produces the wanted diagram
It should be an easy task that doesn't work for me.
For more information... This is part a random finite element software which is freely distributed by the authors. You can see the whole documentation and download all parts of the program from this link:
http://random.engmath.dal.ca/rfem/
The parts that are causing the problem are used just for extracting and showing results.
It is an old software so maybe there is some problem with that.

I don't know what exactly you did with your Python code since you didn't provide the exact code snippets. I also don't have the reputation to comment. So I'll just provide code examples for the 3 methods you listed (CMD, Powershell, Python). All three methods worked on my machine.
1) CMD
start "" "plotps.exe" hist.plotps
The double quotes after the start keyword are there to specify an optional title. What went wrong in your CMD example is that windows thought "plotps.exe" was the title. You don't need to specify a title, but you need to write the quotes. More info on this keyword can be found here: https://ss64.com/nt/start.html
Also note that start is asynchronous
A synchronous way of doing it would be:
plotps.exe hist.plotps
2) Powershell
Start-Process -FilePath "plotps.exe" -ArgumentList "hist.plotps"
I would strongly recommend using powershell over CMD if you have access to both.
This method is synchronous.
More info on Start-Process: https://learn.microsoft.com/en-us/powershell/module/Microsoft.PowerShell.Management/Start-Process?view=powershell-5.0
3) Python
I'm not a python expert but this worked for me:
import subprocess
subprocess.call(['plotps.exe', 'hist.plotps'])

Taking Input from an external file in Python

If you ever came across a situation where you have test your program against a very large pile of input, you have wondered that if there's any way to shortcut o it.
There are certain methods often come very handy when you have to test your program time and space complexity while processing a large input
You cant always input some large input manually , so there's the method by which you can provide input to your program using external txt file
Below is my answer to it :)

You just write simple program and then run it from command line in any platform like Windows/ Linux.
python program.py < input.txt > output.txt
<,> are redirection operators which simply redirects the stdin and stdout to input.txt and output.txt. This is the easiest way.
Alternatively, you can do the following
import sys sys.stdin=open('input.txt','r') sys.stdout=open('output.txt','w')
Alternatively you can do the following
input=open('input.txt','r')
ouput=open('ouput.txt','w')
n=input.read()
output.write(n)
I prefer method 1 as it is simple and no need of file handling and this helps a lot in Codejam, FaceBook HackerCup. Hope it helps

How the OS handles python and subprocesses of a python script...?

My question is somewhat unique. I am currently working on a project for my computer forensics class. This project is aimed at hiding disk data from investigators. The method by which this is supposed to be achieved is by writing the bytes of a "clean" file over the "bad" file. Once overwritten, the "bad" file is deleted.
This concept sounds simple enough, but what my partner and I have observed is interesting. If we open a file in a python script, we can easily overwrite the memory associated with that file on disk (verified using dd). We can also easily delete a file using from inside the script. However, a write then delete results in no write actually taking place, only the file's removal.
This makes sense from an OS optimization standpoint. From that point, we thought it might work if we split the writing and deleting into two separate scripts, and controlled both by a third. However, it seems that even if we run the scripts as a subprocess of another script, the same thing happens. We've tried to use bash scripts for the deletion process instead of pure python, and still, nothing sticks.
This project was supposed to be a whole mess of little anti-forensics tools like this, but this particular one has captured our whole attention because of this issue. Does anyone have an idea as to why this is happening and what we can do to move forward?
We know this can be achieved in C, etc, but we want to solve this using python because of the interesting constraints it's presented.
---EDIT---
This is a snippet from our controller, it calls "ghost.py" with the associated params.
ghost.py prints the edited file names/paths to stdout.
Relevant code follows:
proc = subprocess.Popen(['python', 'ghost.py', '-c', 'good.txt', '-d','/mnt/evil.txt'], stdout=subprocess.PIPE,)
files = proc.communicate()
for i in files:
if i != None and i != "\n":
os.system("./del.sh " + i)

Using a subprocess doesn't change any interesting aspect of your design, so don't use them. You probably need os.fsync(). Try this pattern:
myfile.write('all of my good data')
myfile.flush()
os.fsync(myfile.fileno())
myfile.close()
os.remove(myfile)
Reference: https://docs.python.org/2/library/os.html#os.fsync

executable files and django

Is it possible to use executable files with Django. For instance I have a Django form that takes input a file. I want to pass the file as an argument to a script/executable/program and output the results back with django. What would be a good starting point for this idea? Thanks a lot

subprocess

The question you should you be asking is it possible in python. Essentially all you would need to do it take in the inputs, into a python script which you can call from your django view or what not. Once there run the execute subprocess command with your executable, and take in the results. :)
Goodluck.

do you want some things like this?
>>> import commands
>>> commands.getstatusoutput('pwd')
(0, '/home/efazati')

Yes, you can do this. There are a few different ways to do it in Python. If you want to read the output and use it then you probably want popen, found at http://docs.python.org/library/os.html#os.popen
Warning, this can be very dangerous. If the data added to the command line is malicious (even accidentally) then you could cause bad stuff to happen on the server. You're effectively giving people who submit forms the ability to run commands on the server with the same permissions the web server has.
import os
ls_fd = os.popen('ls -l /tmp')
output = ls_fd.read()
ls_fd.close()
Doing it this way your ls_fd is a file like object. you .read() it like a file. You can only do it once, then you're at the end of the file. Likewise, you should .close() it when you're done.
Strongly consider against adding user input to the command line. You can create a list of possible options and then give the user a choice which one they want. Then instead of taking the user's input and adding it to the end of the command line you can use one of your pre-configured command lines and execute that.