We Keep Coding

Pandas Read CSV for file address with \t in it

This may be a redundant question because I know that I can rename the file and solve the issue, but I'm still pretty new at this and it would be really useful information for the future. Thanks in advance to respondents!
So, I have a CSV file which is a table exported from SQL with the filename "t_SQLtable" located in a sub-folder of my working directory.
In order to open the file in Pandas I use the following command:
SQLfile= pd.read_csv('SUBFOLDER\t_SQLtable.csv', sep=',')
This is the error I receive:
FileNotFoundError: [Errno 2] File SUBFOLDER _SQLtable.csv does not exist: 'SUBFOLDER\t_SQLtable.csv'
My understanding is that Pandas is reading the <\t> as a tab and thus is not able to find the file, because that's not the file name it is looking for. But I don't know how to format the text in order to tell Pandas how to recognize the <t> as part of the filename. Would anyone know how to resolve this?
Thank you!

Folders are navigated using / which won't escape any character
SQLfile= pd.read_csv('SUBFOLDER/t_SQLtable.csv', sep=',')
in future if you want to keep \t without it being considered as tab
use raw string
print('SUBFOLDER\t_SQLtable.csv')
print(r'SUBFOLDER\t_SQLtable.csv')
SUBFOLDER _SQLtable.csv
SUBFOLDER\t_SQLtable.csv

Try with this.
SQLfile= pd.read_csv('SUBFOLDER\\t_SQLtable.csv', sep=',')
SQLfile= pd.read_csv('SUBFOLDER/t_SQLtable.csv', sep=',')
If doesn't work , then try this:
import os
file_path = os.path.join(os.getcwd(), "SUBFOLDER", "t_SQLtable.csv")
SQLfile= pd.read_csv(file_path, sep=',')

Simply do what you did before, except add an r right before the string:
SQLfile = pd.read_csv(r'SUBFOLDER\t_SQLtable.csv', sep=',')
Adding r to the start of a string will make python treat it as a raw string, as in, all escape codes won't be evaluated.

How to get everything before a character and x amount after?

It has been trial and error and can't seem to get what I want.
I am accessing an API to get some info. Unfortunately it's the only API to get that info and to do it, it downloads a binary content of a file and names it:
folder\filename.whatever
i.e. test\purpleMonkeyTest.docx
There is a bunch more info that comes in from the call but there is this line:
Saved the binary content to: /home/user/python/test\purpleMonkeyTest.docx
Some of the files have " or other special characters so I can't just get the file name and delete it as part of the script, since I won't know what to escape.
So my goal here is to strip the line and get:
/home/user/python/test\purpleMonkeyTest.docx
then get only:
/home/user/python/test\pu
then:
os.remove "/home/user/python/test\pu"*
I'm thinking that a wildcard should work for all, unless there is a better way to do it. All files saved have the character \ in them so I've got to the point where I'm getting everything prior to the \ but I want one or two characters after that as well.
Here's what I've tried:
def fileName(itemID):
import fnmatch
details = itemDetails(itemID, True) # get item id and file details
filepath = matchPattern((details), 'Saved the binary content to: *')
filepath = (filepath).split('\\')[0]
print(filepath)
#os.remove(re.escape(filepath))
return (matchPattern((details), 'Binary component: *'))
def matchPattern(details, pattern):
import fnmatch
return (fnmatch.filter((details), pattern)[0].split(": " ,1)[1])
Output:
/home/user/python/test
purpleMonkeyTest.docx
I do want the file name for later: that's actually the main goal. The API downloads the damn file automatically though.
EDIT:
Answer below works for getting the chars I want. Os remove is not removing the file though.
OSError: [Errno 2] No such file or directory: '/home/user/python/test\\Re*'
Managed to get it to work using glob, I guess os.remove doesn't support Wilds.
files = glob.glob((filepath)+"*")
for file in files:
os.remove(file)
Thanks for the help!!

As far as I understand your question you would like to retrieve 2 parts - everything between first / and \ with 2 chars afterwards and then everything after \:
str = "Saved the binary content to: /home/user/python/test\purpleMonkeyTest.docx"
print (str[str.index("/"):str.rindex("\\") + 3])
print (str[str.rindex("\\") + 1:])
Output
/home/user/python/test\pu
purpleMonkeyTest.docx

os.path.basename to outfile

For every input file processed (see code below) I am trying to use "os.path.basename" to write to a new output file - I know I am missing something obvious...?
import os
import glob
import gzip
dbpath = '/home/university/Desktop/test'
for infile in glob.glob( os.path.join(dbpath, 'G[D|E]/????/*.gz') ):
print("current file is: " + infile)
**
outfile=os.path.basename('/home/university/Desktop/test/G[D|E]
/????/??????.xaa.fastq.gz').rsplit('.xaa.fastq.gz')[0]
file=open(outfile, 'w+')
**
gzsuppl = Chem.ForwardSDMolSupplier(gzip.open(infile))
for m in gzsuppl:
if m is None: continue
...etc
file.close()
print(count)
It is not clear to me how to capture the variable [0] (i.e. everything upstream of .xaa.fastq.gz) and use as the basename for the new output file?
Unfortunately it simply writes the new output file as "??????" rather than the actual sequence of 6 letters.
Thanks for any help given.

This seems like it will get everything upstream of the .xaa.fastq.gz in the paths returned from glob() in your sample code:
import os
filepath = '/home/university/Desktop/test/GD /AAML/DEAAML.xaa.fastq.gz'
filepath = os.path.normpath(filepath) # Changes path separators for Windows.
# This section was adapted from answer https://stackoverflow.com/a/3167684/355230
folders = []
while 1:
filepath, folder = os.path.split(filepath)
if folder:
folders.append(folder)
else:
if filepath:
folders.append(filepath)
break
folders.reverse()
if len(folders) > 1:
# The last element of folders should contain the original filename.
filename_prefix = os.path.basename(folders[-1]).split('.')[0]
outfile = os.path.join(*(folders[:-1] + [filename_prefix + '.rest_of_filename']))
print(outfile) # -> \home\university\Desktop\test\GD \AAML\DEAAML.rest_of_filename
Of course what ends-up in outfile isn't the final path plus filename since I don't know what the remainder of the filename will be and just put a placeholder in (the '.rest_of_filename').

I'm not familiar with the kind of input data you're working with, but here's what I can tell you:
The "something obvious" you're missing is that outfile has no connection to infile. Your outfile line uses the ?????? rather than the actual filename because that's what you ask for. It's glob.glob that turns it into a list of matches.
Here's how I'd write that aspect of the outfile line:
outfile = infile.rsplit('.xaa.fastq.gz', 1)[0]
(The , 1 ensures that it'll never split more than once, no matter how crazy a filename gets. It's just a good habit to get into when using split or rsplit like this.)
You're setting yourself up for a bug, because the glob pattern can match *.gz files which don't end in .xaa.fastq.gz, which would mean that a random .gz file which happens to wind up in the folder listing would cause outfile to have the same path as infile and you'd end up writing to the input file.
There are three solutions to this problem which apply to your use case:
Use *.xaa.fastq.gz instead of *.gzin your glob. I don't recommend this because it's easy for a typo to sneak in and make them different again, which would silently reintroduce the bug.
Write your output to a different folder than you took your input from.
outfile = os.path.join(outpath, os.path.relpath(infile, dbpath))
outparent = os.path.dirname(outfile)
if not os.path.exists(outparent):
os.makedirs(outparent)
Add an assert outfile != infile line so the program will die with a meaningful error message in the "this should never actually happen" case, rather than silently doing incorrect things.
The indentation of what you posted could be wrong, but it looks like you're opening a bunch of files, then only closing the last one. My advice is to use this instead, so it's impossible to get that wrong:
with open(outfile, 'w+') as file:
# put things which use `file` here
The name file is already present in the standard library and the variable names you chose are unhelpful. I'd rename infile to inpath, outfile to outpath, and file to outfile. That way, you can tell whether each one is a path (ie. a string) or a Python file object just from the variable name and there's no risk of accessing file before you (re)define it and getting a very confusing error message.

How to append new data onto a new line

My code looks like this:
def storescores():
hs = open("hst.txt","a")
hs.write(name)
hs.close()
so if I run it and enter "Ryan"
then run it again and enter "Bob"
the file hst.txt looks like
RyanBob
instead of
Ryan
Bob
How do I fix this?

If you want a newline, you have to write one explicitly. The usual way is like this:
hs.write(name + "\n")
This uses a backslash escape, \n, which Python converts to a newline character in string literals. It just concatenates your string, name, and that newline character into a bigger string, which gets written to the file.
It's also possible to use a multi-line string literal instead, which looks like this:
"""
"""
Or, you may want to use string formatting instead of concatenation:
hs.write("{}\n".format(name))
All of this is explained in the Input and Output chapter in the tutorial.

In Python >= 3.6 you can use new string literal feature:
with open('hst.txt', 'a') as fd:
fd.write(f'\n{name}')
Please notice using 'with statment' will automatically close the file when 'fd' runs out of scope

All answers seem to work fine. If you need to do this many times, be aware that writing
hs.write(name + "\n")
constructs a new string in memory and appends that to the file.
More efficient would be
hs.write(name)
hs.write("\n")
which does not create a new string, just appends to the file.

The answer is not to add a newline after writing your string. That may solve a different problem. What you are asking is how to add a newline before you start appending your string. If you want to add a newline, but only if one does not already exist, you need to find out first, by reading the file.
For example,
with open('hst.txt') as fobj:
text = fobj.read()
name = 'Bob'
with open('hst.txt', 'a') as fobj:
if not text.endswith('\n'):
fobj.write('\n')
fobj.write(name)
You might want to add the newline after name, or you may not, but in any case, it isn't the answer to your question.

I had the same issue. And I was able to solve it by using a formatter.
file_name = "abc.txt"
new_string = "I am a new string."
opened_file = open(file_name, 'a')
opened_file.write("%r\n" %new_string)
opened_file.close()
I hope this helps.

There is also one fact that you have to consider.
You should first check if your file is empty before adding anything to it. Because if your file is empty then I don't think you would like to add a blank new line in the beginning of the file. This code
first checks if the file is empty
If the file is empty then it will simply add your input text to the file else it will add a new line and then it will add your text to the file. You should use a try catch for os.path.getsize() to catch any exceptions.
Code:
import os
def storescores():
hs = open("hst.txt","a")
if(os.path.getsize("hst.txt") > 0):
hs.write("\n"+name)
else:
hs.write(name)
hs.close()

I presume that all you are wanting is simple string concatenation:
def storescores():
hs = open("hst.txt","a")
hs.write(name + " ")
hs.close()
Alternatively, change the " " to "\n" for a newline.

import subprocess
subprocess.check_output('echo "' + YOURTEXT + '" >> hello.txt',shell=True)

f=open("Python_Programs/files_forhndling.txt","a+")
inpt=str(input("Enter anything:\n>>"))
f.write(inpt)
f.write("\n")
print("Data inserted Successfully")
f.close()
welcome to file handling
new line
file handling in python
123456
78875454
✔ Output 💡 CLICK BELOW & SEE ✔

You need to change parameter "a" => "a+".
Follow this code bellows:
def storescores():
hs = open("hst.txt","a+")

Python add character to string

I am trying to open a user's text file and replace a string in Python. I have the replacement working, but to open a file I understand that I need to add another backslash after each one in the file path. I am not sure how to do that. I looked at other stack overflow questions, but they were mostly about adding to the beginning or end of the string. Please help! Here's the code so far:
yourfile = input()
with open ("C:\\Users\\Rajrishi\\Documents\\MyJava\\text.txt") as myfile:
data = myfile.readlines()
strdata = "".join(data)
strdata = strdata.replace("a string","a replacement")
print(strdata)

You may find it easier to pass a raw string by prefixing with r
like so:
with open (r"C:\Users\Rajrishi\Documents\MyJava\text.txt") as myfile:
This will mean that you don't need to escape slashes

You can actually use forward slashes:
with open("C:/Users/Rajrishi/Documents/MyJava/text.txt") as myfile:
...

If your code and your file in the same folder, you can do this :
with open (r"text.txt") as myfile:
...
Just write name of the file.