We Keep Coding

In Python 3 on Windows, how can I set NTFS compression on a file? Nothing I've googled has gotten me even close to an answer

(Background: On an NTFS partition, files and/or folders can be set to "compressed", like it's a file attribute. They'll show up in blue in Windows Explorer, and will take up less disk space than they normally would. They can be accessed by any program normally, compression/decompression is handled transparently by the OS - this is not a .zip file. In Windows, setting a file to compressed can be done from a command line with the "Compact" command.)
Let's say I've created a file called "testfile.txt", put some data in it, and closed it. Now, I want to set it to be NTFS compressed. Yes, I could shell out and run Compact, but is there a way to do it directly in Python code instead?

In the end, I ended up cheating a bit and simply shelling out to the command line Compact utility. Here is the function I ended up writing. Errors are ignored, and it returns the output text from the Compact command, if any.
def ntfscompress(filename):
import subprocess
_compactcommand = 'Compact.exe /C /I /A "{}"'.format(filename)
try:
_result = subprocess.run(_compactcommand, timeout=86400,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,text=True)
return(_result.stdout)
except:
return('')

Python Subprocess how to pass subshell argument

I am doing a script using SoX to merge multiple audio file together.
This command works in the terminal
sox &(ls *.mp3) out.mp3
but if I try using it inside a python script by calling subprocess.run() it doesn't
subprocess.run(['sox', '$(ls *.mp3)', 'out.mp3'])
> sox FAIL formats: can't open input file `$(ls *.mp3)': No such file or
> directory
I image that is because of the subshell operation, but I don't know how to pass it correctly.
I also tried, as some other post suggested, passing the argument shell=True but then it says
> sox FAIL sox: Not enough input filenames specified
I am in the same working directory and I also tried supplying the full path but doesn't work either.
I could just write a bash script and call it, but I would like to know how to deal in this scenario with Python.

you want to use shell=True to force subprocess to run your command through the shell interpreter and parse the wildcards/sub-commands. However this (depending on the platform) imposes that the argument is passed as string, not as list of parameters. A lot of constraints for a lazy & unsafe way of doing it.
Wait. You can do without shell=True using glob.glob:
subprocess.run(['sox'] + glob.glob('*.mp3') + ['out.mp3'])
Would be better to check if there actually are mp3 files in the current folder so:
input_files = glob.glob('*.mp3')
if input_files:
subprocess.run(['sox'] + input_files + ['out.mp3'])
else:
raise Exception("No mp3 files")
if you get the "No mp3 files" message, then check the current directory. It's always good to use a parameter for the input directory, and avoid relying on the current directory (glob.glob(os.path.join(input_directory,'*.mp3')))

Python - Open file in notepad that is contained in a variable

I couldn't find this anywhere, so sorry if I missed it. It seems like it should be simple but somehow isn't. I have a simple program that opens a log (log1.lg let's say) and strips any lines that don't contain keywords. It then tosses them into a 2nd file that is renamed to Log1.lg.clean.
The way I've implemented this is by using os.rename so the code looks like this:
#define source and key words
source_log = 'Log1.lg'
bad_words = ['word', 'bad']
#clean up the log
with open(source_log) as orig_log, open('cleanlog.lg', 'w') as cleanlog:
for line in orig_log:
if not any9bad_word in line for bad_word in bad_words):
cleanlog.write(line)
#rename file and open in Notepad
rename = orig_log + '.clean'
new_log = os.rename("cleanlog.lg", rename)
prog = "notepad.exe"
subprocess.Popen(prog, new_log)
Error I'm getting is this:
File "C:\Users\me\Downloads\PythonStuff\stripMmax.py", line 23, in cleanLog
subprocess.Popen(prog, new_log)
File "C:\Python27\lib\subprocess.py", line 339, in __init__
raise TypeError("bufsize must be an integer")
TypeError: bufsize must be an integer
I'm using Python 2.7 if that's relevant. I don't get why this isn't working or why it's requiring a bufsize. I've seen other examples where this works this way so I'm thinking maybe this command doesn't work in 2.7 the way I'm typing it?
The documentation shows how to use this properly using the actual file name in quotes, but as you can see, mine here is contained in a variable which seems to cause issues. Thanks in advance!

See the Popen constructor here: subprocess.Popen. The second argument to Popen is bufsize. That explains your error. Also note that os.rename does not return anything so new_log will be None. Use your rename variable instead. Your call should look like this:
subprocess.Popen([prog, rename])
You likely also want to wait on the created Popen object:
proc = subprocess.Popen([prog, rename])
proc.wait()
Or something like that.

Why doesn't my bash script read lines from a file when called from a python script?

I am trying to write a small program in bash and part of it needs to be able to get some values from a txt file where the different files are separated by a line, and then either add each line to a variable or add each line to one array.
So far I have tried this:
FILE=$"transfer_config.csv"
while read line
do
MYARRAY[$index]="$line"
index=$(($index+1))
done < $FILE
echo ${MYARRAY[0]}
This just produces a blank line though, and not what was on the first line of the config file.
I am not returned with any errors which is why I am not too sure why this is happening.
The bash script is called though a python script using os.system("$HOME/bin/mcserver_config/server_transfer/down/createRemoteFolder"), but if I simply call it after the python program has made the file which the bash script reads, it works.
I am almost 100% sure it is not an issue with the directories, because pwd at the top of the bash script shows it in the correct directory, and the python program is also creating the data file in the correct place.
Any help is much appreciated.
EDIT:
I also tried the subprocess.call("path_to_script", shell=True) to see if it would make a difference, I know it is unlikely but it didn't.

I suspect that when calling the bash script from python, having just created the file, you are not really finished with that file: you should either explicitly close the file or use a with construct.
Otherwise, the written data is still in any buffer (from the file object, or in the OS, or wherever). Only closing (or at least flushing) the file makes sure the data is indeed in the file.
BTW, instead of os.system, you should use the subprocess module...

subprocessing library in Python

import subprocess
sample_file_directory = "..." # directory where file is
SCRIPT_DIR = "..." # directory where script is
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
min_side_len, err = p.communicate()
print len(err)
So i have this script that analyzes .ply files (3d file format) and gives me data about the file. I am trying to find out in my directory which files are corrupted and which aren't. So I am trying to use the subprocess library to run the script to find an arbitrary feature of the .ply files (in this case the minimum edge length) If err has anything in it, it means it couldn't retrieve the arbitrary feature and the file is corrupted. Here, I am only running it on one file. However, I keep getting an error.
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Can anyone help me figure out why I am getting this error? Both my data directory and script directory are valid. Also, sorry if I left anything out that's important. First time posting.

Is your SCRIPT_DIR variable the absolute path or a relative path valid in the location you're running the script? If it's the latter, it may not work, as the subprocess may start in a different environment that has a different working directory.

The problem lies here:
p = subprocess.Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
Notice this part:
Popen([SCRIPT_DIR,sample_file_directory,'min','edgelen']
Now, see, you already know that subprocess takes a list. But the problem is that you are passing the directory as part of the list. The list is formatted like this:
[command, arg1, arg2...]
Now, when you run that command, you are doing this:
[SCRIPT_DIR -> command
sample_file_directory -> arg1
'min' -> arg2
'edgelen' -> arg3]
See the problem? You are passing the script's directory as the command to run and the script's name as an argument. Import the os module and do this:
p = subprocess.Popen([os.path.join(SCRIPT_DIR,sample_file_directory),'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
This does this:
[SCRIPT_DIR and sample_file_directory joined together -> command
'min' -> arg1
'edgelen' -> arg2]
os.path.join automatically adds the path separator.

subprocess.Popen() won't combine path components for you. From what I see, you're only providing a directory. You'll need to add the target executable's name to the path being given to Popen. Try the following:
p = subprocess.Popen([os.path.join(SCRIPT_DIR,SCRIPT_NAME),sample_file_directory,'min','edgelen'],stdout=subprocess.PIPE,stderr=subprocess.PIPE)

"One other thing it could potentially be is that you're trying to run a 32-bit binary on a 64-bit system without the 32-bit libraries installed. I actually ran into this issue earlier today. A fix is documented here. I don't know anything about your system, so this may be completely irrelevant advice, but I figured it wouldn't hurt to mention it." Dan Albert. He came up with the answer to the solution! Thanks so much Dan!