We Keep Coding

Unexpected double quotes while appending file items to subprocess.run

I am trying to read from a file which has contents like this:
#\5\5\5
...
#\5\5\10
This file content is then fed into subprocess module of python like this:
for lines in file.readlines():
print(lines)
cmd = ls
p = subprocess.run([cmd, lines])
The output turns into something like this:
CompletedProcess(args=['ls', "'#5\\5\\5'\n"], returncode=1)
I don't understand why the contents of the file is appended with a double quote and another backward slash is getting appended.

The real problem here isn't Python or the subprocess module. The problem the use of subprocess to invoke shell commands, and then trying to parse the results. In this case, it looks like the command is ls, and the plan appears to be to read some filesystem paths from a text file (each path on a separate line), and list the files at that location on the filesystem.
Using subprocess to invoke ls is really, really, really not the way to accomplish that in Python. This is basically an attempt to use Python like a shell script (this use of ls would still be problematic, but that's a different discussion).
If a shell script is the right tool for the job, then write a shell script. If you want to use Python, then use one of the API's that it provides for interacting with the OS and the filesystem. There is no need to bring in external programs to achieve this.
import os
with open("list_of_paths.txt", "r") as fin:
for line in fin.readlines():
w = os.listdir(line.strip())
print(w)
Note the use of .strip(), this is a string method that will remove invisible characters like spaces and newlines from the ends of the input.
The listdir method provided by the os module will return a list of the files in a directory. Other options are os.scandir, os.walk, and the pathlib module.
But please do not use subprocess. 95% of the time, when someone thinks "should I use Python's subprocess module for this?" the ansewr is "NO".

It is because \ with a relevant character or digit becomes something else other than the string. For example, \n is not just \ and n but it means next line. If you really want a \n, then you would add another backslash to it (\\n). Likewise \5 means something else. here is what I found when i ran \5:
and hence the \\ being added, if I am not wrong

Python How to open filepath with spaces when passing through formatter

How can I run a command that requires a filepath that contains spaces when using the start command with os.system
For Example:
# path_d[key] = C:\Users\John\Documents\Some File With Space.exe
path = path_d[key]
os.system("start {0}".format(path))
When I try running it I end up getting an error saying:
Windows cannot find 'C:\Users\John\Documents\Some.'. Make sure you typed the name correctly, and then try again.

i do the following
path = path_d[key]
os.system(r'start "{0}"'.format(path))
so, surround the path with double quotes. that way it will take care of spaces in path.
if there is no default application to open, it might open command prompt. So, if its a text file do the following
os.system(r'notepad "{0}"'.format(path))

You need to properly escape your special characters in path, which might be easily done as such:
path = r"C:\Users\John\Documents\Some File With Space.exe"
To execute it under Windows:
import os
os.system(r"C:\Users\John\Documents\Some File With Space.exe")
EDIT
per request of OP:
path_dict = {"path1": r"C:\Users\John\Documents\Some File With Space.exe"}
os.system('{}'.format(path_dict["path1"]))

os.listdir can't see my directory

I am working on a python script that installs an 802.1x certificate on a Windows 8.1 machine. This script works fine on Windows 8 and Windows XP (haven't tried it on other machines).
I have isolated the issue. It has to do with clearing out the folder
"C:\Windows\system32\config\systemprofile\AppData\LocalLow\Microsoft\CryptURLCache\Content"
The problem is that I am using the module os and the command listdir on this folder to delete each file in it. However, listdir errors, saying the folder does not exist, when it does indeed exist.
The issue seems to be that os.listdir cannot see the LocalLow folder. If I make a two line script:
import os
os.listdir("C:\Windows\System32\config\systemprofile\AppData")
It shows the following result:
['Local', 'Roaming']
As you can see, LocalLow is missing.
I thought it might be a permissions issue, but I am having serious trouble figuring out what a next step might be. I am running the process as an administrator from the command line, and it simply doesn't see the folder.
Thanks in advance!
Edit: changing the string to r"C:\Windows\System32\config\systemprofile\AppData", "C:\Windows\System32\config\systemprofile\AppData", or C:/Windows/System32/config/systemprofile/AppData" all produce identical results
Edit: Another unusual wrinkle in this issue: If I manually create a new directory in that location I am unable to see it through os.listdir either. In addition, I cannot browse to the LocalLow or my New Folder through the "Save As.." command in Notepad++
I'm starting to think this is a bug in Windows 8.1 preview.

I encountered this issue recently.
I found it's caused by Windows file system redirector
and you can check out following python snippet
import ctypes
class disable_file_system_redirection:
_disable = ctypes.windll.kernel32.Wow64DisableWow64FsRedirection
_revert = ctypes.windll.kernel32.Wow64RevertWow64FsRedirection
def __enter__(self):
self.old_value = ctypes.c_long()
self.success = self._disable(ctypes.byref(self.old_value))
def __exit__(self, type, value, traceback):
if self.success:
self._revert(self.old_value)
#Example usage
import os
path = 'C:\\Windows\\System32\\config\\systemprofile\\AppData'
print os.listdir(path)
with disable_file_system_redirection():
print os.listdir(path)
print os.listdir(path)
ref : http://code.activestate.com/recipes/578035-disable-file-system-redirector/

You must have escape sequences in your path. You should use a raw string for file/directory paths:
# By putting the 'r' at the start, I make this string a raw string
# Raw strings do not process escape sequences
r"C:\path\to\file"
or put the slashes the other way:
"C:/path/to/file"
or escape the slashes:
# You probably won't want this method because it makes your paths huge
# I just listed it because it *does* work
"C:\\path\\to\\file"

I'm curious as to how you are able to list the contents with those two lines. You are using escape sequences \W, \S, \c, \s, \A in your code. Try escaping the back slash like this:
import os
os.listdir('C:\\Windows\\System32\\config\\systemprofile\\AppData')

Cannot read in files

I have a small problem with reading in my file. My code:
import csv as csv
import numpy
with open("train_data.csv","rb") as training:
csv_file_object = csv.reader(training)
header = csv_file_object.next()
data = []
for row in csv_file_object:
data.append(row)
data = numpy.array(data)
I get the error no such file "train_data.csv", so I know the problem lies with the location. But whenever I specify the pad like this: open("C:\Desktop...etc) it doesn't work either. What am I doing wrong?

If you give the full file path, your script should work. Since it is not, it must be that you have escape characters in your path. To fix this, use a raw-string to specify the file path:
# Put an 'r' at the start of the string to make it a raw-string.
with open(r"C:\path\to\file\train_data.csv","rb") as training:
Raw strings do not process escape characters.
Also, just a technical fact, not giving the full file path causes Python to look for the file in the directory that the script is launched from. If it is not there, an error is thrown.

When you use open() and Windows you need to deal with the backslashes properly.
Option 1.) Use the raw string, this will be the string prefixed with an r.
open(r'C:\Users\Me\Desktop\train_data.csv')
Option 2.) Escape the backslashes
open('C:\\Users\\Me\\Desktop\\train_data.csv')
Option 3.) Use forward slashes
open('C:/Users/Me/Desktop/train_data.csv')
As for finding the file you are using, if you just do open('train_data.csv') it is looking in the directory you are running the python script from. So, if you are running it from C:\Users\Me\Desktop\, your train_data.csv needs to be on the desktop as well.

Open a text file using notepad as a help file in python?

I would like to give users of my simple program the opportunity to open a help file to instruct them on how to fully utilize my program. Ideally i would like to have a little blue help link on my GUI that could be clicked at any time resulting in a .txt file being opened in a native text editor, notepad for example.
Is there a simple way of doing this?

import webbrowser
webbrowser.open("file.txt")
Despite it's name it will open in Notepad, gedit and so on. Never tried it but it's said it works.
An alternative is to use
osCommandString = "notepad.exe file.txt"
os.system(osCommandString)
or as subprocess:
import subprocess as sp
programName = "notepad.exe"
fileName = "file.txt"
sp.Popen([programName, fileName])
but both these latter cases you will need to find the native text editor for the given operating system first.

os.startfile('file.txt')
From the python docs:
this acts like double clicking the file in Windows Explorer, or giving the file name as an argument to the start command from the interactive command shell: the file is opened with whatever application (if any) its extension is associated.
This way if your user changed their default text editor to, for example, notepad++ it would use their preference instead of notepad.

If you'd like to open the help file with the application currently associated with text files, which might not be notepad.exe, you can do it this way on Windows:
import subprocess
subprocess.call(['cmd.exe', '/c', 'file.txt'])

You can do this in one line:
import subprocess
subprocess.call(['notepad.exe', 'file.txt'])
You can rename notepad.exe to the editor of your choice.

Here's somewhat of a cross-platform one (edit if you have any other methods):
import shutil, subprocess, os
file_name = "whatever.txt"
if hasattr(os, "startfile"):
os.startfile(file_name)
elif shutil.which("xdg-open"):
subprocess.call(["xdg-open", file_name])
elif "EDITOR" in os.environ:
subprocess.call([os.environ["EDITOR"], file_name])

If you have any preferred editor, you can just first try opening in that editor or else open in a default editor.
ret_val = os.system("gedit %s" % file_path)
if ret_val != 0:
webbrowswer.open(file_path)
In the above code, I am first trying to open my file in gedit editor which is my preferred editor, if the system does not have gedit installed, it just opens the file in the system's default editor.

If anyone is getting an instance of internet explorer when they use import webbrowser, try declaring the full path of the specified file.
import webbrowser
import os
webbrowser.open(os.getcwd() + "/path/to/file/in/project")
#Gets the directory of the current file, then appends the path to the file
Example:
import webbrowser
import os
webbrowser.open(os.getcwd() + "/assets/ReadMe.txt")
#Will open something like "C:/Users/user/Documents/project/assets/ReadMe.txt"