Exporting multiple csv files with dynamic naming - python

I created about 200 csv files in Python and now need to download them all.
I created the files from a single file using:
g = df.groupby("col")
for n,g in df.groupby('col'):
g.to_csv(n+'stars'+'.csv')
When I try to use this same statement to export to my machine I get a syntax error and I'm not sure what I'm doing wrong:
g = df.groupby("col")
for n,g in df.groupby('col'):
g.to_csv('C:\Users\egagne\Downloads\'n+'stars'+'.csv'')
Error:
File "<ipython-input-27-43a5bfe55259>", line 3
g.to_csv('C:\Users\egagne\Downloads\'n+'stars'+'.csv'')
^
SyntaxError: invalid syntax
I'm in Jupyter lab, so I can download each file individually but I really don't want to have to do that.

You're possibly mixing up integers and strings, and the use of backslash in literals is dangerous anyway. Consider using the following
import os
inside the loop
f_name = os.path.join('C:', 'users', ' egagne', 'Downloads', str(n), 'stars.csv')
g.to_csv(f_name)
with os.path.join taking care of the backslashes for you.

g.to_csv('C:\Users\egagne\Downloads\'n+'stars'+'.csv'')
needs to be
g.to_csv('C:\\Users\\egagne\\Downloads\\'+n+'stars.csv').
There were two things wrong -- the backslash is an escape character so if you put a ' after it, it will be treated as part of your string instead of a closing quote as you intended it. Using \\ instead of a single \ escapes the escape character so that you can include a backslash in your string.
Also, you did not pair your quotes correctly. n is a variable name but from the syntax highlighting in your question it is clear that it is part of the string. Similarly you can see that stars and .csv are not highlighted as part of a string, and the closing '' should be a red flag that something has gone wrong.
Edit: I addressed what is causing the problem but Ami Tavory's answer is the right one -- though you know this is going to run on windows it is a better practice to use os.path.join() with directory names instead of writing out a path in a string. str(n) is also the right way to go if you are at all unsure about the type of n.

Related

Python FileNotFound error caused by Python inserting unwanted slashes in string

I am trying to save a figure from Matplotlib to a folder location on a drive and i am getting some unwanted behavior from the filepath.
This is what i have set up to run with a real string type to handle the "\" escape character.
save_path = r"\\nemesis\Network Planning\Team Members\Taylor\2020_04_23 - COVID Impact
Adjustment\Test Stores\State and Region Growth - " +str(Store_ID)+ ".jpg"
print(save_path)
plt.savefig(save_path)
The print statement displays the correct file path string
However when i run the savefig python appears to add an extra slash next to every existing slash in the string and gives the FileNotFound error. Full error transcript below.
FileNotFoundError: [Errno 2] No such file or directory: '\\\\nemesis\\Network Planning\\Team Members\\Taylor\\2020_04_23 - COVID Impact Adjustment\\Test Stores\\State and Region Growth - 17062.jpg'
I am at a loss for the reasons as to why this is occurring and have tried a bunch of different string methods and none have seemed to work.
Any help is much appreciated
To answer your question, I'll need to explain some background on raw strings. Raw strings are just an easier way to include backslashes in a normal string without you needing to escape them. For example, defining a string that would be printed as "a\b\c" using normal string syntax, you would need to write my_string = "a\\b\\c", but with raw strings, you only need to write r"a\b\c", but the resulting string is equal in both cases:
s = r"a\b\c"
s2 = "a\\b\\c"
s == s2 # Evaluates to True
When you print the string, print() excludes the extra backslashes required to recreate the string using normal syntax:
print(s) # -> a\b\c
To view a representation of the string suitable for recreating it, use repr(s):
print(repr(s)) # -> "a\\b\\c"
Now for your question. The raw string you make may look like what you want when you use print(), as it excludes the extra slashes, but isn't what you want. For one thing, I don't think you meant to have two backslashes at the beginning of the path.
save_path = r"\\nemesis\Network Planning\..."
print(save_path) # Prints the correct path, save the extra leading backslash
print(repr(save_path)) # Reveals the normal string representation, which requires 4 backslashes to create (where there should be only two).
Fixing this problem is simple: represent your file path differently. Either use normal strings and escape all the backslashes manually: "\\nemesis\\Network Planning\\Team Members\\Taylor\\2020_04_23 - COVID Impact Adjustment\\Test Stores\\State and Region Growth - " +str(Store_ID)+ ".jpg" or just use os.path.join("\\nemesis", "Network Planning", "Team Members", "Taylor", "2020_04_23 - COVID Impact Adjustment", "Test Stores", "State and Region Growth - "+ str(Store_ID)+ ".jpg") to automatically join the directories with all the proper backslashes (I can't test that second one because I'm on Linux)
Hope this helped!

Why is r' still duplicating the forward slashes in my code?

So I have tried to read the solutions to Python duplicating a forwardslash from my code so it can find the file and most of the questions seem to indicate adding r' solves the problem.
In most of my code this works. But for this file path it is still duplicating all of the forwardslashes. Does anyone know why this would be the case?
I also tried using pathlib.Path to string together my path and it has produced the same result
For privacy I have removed the true file path but it is still replicating the issue. This is in my Jupyter Notebook.
"Raw strings" are the exact same type as regular strings, just a different way of entering them as input. Because their in-memory representation is identical, their "rawness" doesn't persist past the parser and change the way they behave later.
Thus, they still print the same way when repr()ed as any other string: You'll note that the representation didn't include the r'...' sigils, but was only '...'. As the way to represent r'\' as a non-raw-string is '\\', so the interpreter was correct to do so.
There was an absent file path that needed to be included

How to open a file in its default program with python

I want to open a file in python 3.5 in its default application, specifically 'screen.txt' in Notepad.
I have searched the internet, and found os.startfile(path) on most of the answers. I tried that with the file's path os.startfile(C:\[directories n stuff]\screen.txt) but it returned an error saying 'unexpected character after line continuation character'. I tried it without the file's path, just the file's name but it still didn't work.
What does this error mean? I have never seen it before.
Please provide a solution for opening a .txt file that works.
EDIT: I am on Windows 7 on a restricted (school) computer.
It's hard to be certain from your question as it stands, but I bet your problem is backslashes.
[EDITED to add:] Or actually maybe it's something simpler. Did you put quotes around your pathname at all? If not, that will certainly not work -- but once you do, you will find that then you need the rest of what I've written below.
In a Windows filesystem, the backslash \ is the standard way to separate directories.
In a Python string literal, the backslash \ is used for putting things into the string that would otherwise be difficult to enter. For instance, if you are writing a single-quoted string and you want a single quote in it, you can do this: 'don\'t'. Or if you want a newline character, you can do this: 'First line.\nSecond line.'
So if you take a Windows pathname and plug it into Python like this:
os.startfile('C:\foo\bar\baz')
then the string actually passed to os.startfile will not contain those backslashes; it will contain a form-feed character (from the \f) and two backspace characters (from the \bs), which is not what you want at all.
You can deal with this in three ways.
You can use forward slashes instead of backslashes. Although Windows prefers backslashes in its user interface, forward slashes work too, and they don't have special meaning in Python string literals.
You can "escape" the backslashes: two backslashes in a row mean an actual backslash. os.startfile('C:\\foo\\bar\\baz')
You can use a "raw string literal". Put an r before the opening single or double quotes. This will make backslashes not get interpreted specially. os.startfile(r'C:\foo\bar\baz')
The last is maybe the nicest, except for one annoying quirk: backslash-quote is still special in a raw string literal so that you can still say 'don\'t', which means you can't end a raw string literal with a backslash.
The recommended way to open a file with the default program is os.startfile. You can do something a bit more manual using os.system or subprocess though:
os.system(r'start ' + path_to_file')
or
subprocess.Popen('{start} {path}'.format(
start='start', path=path_to_file), shell=True)
Of course, this won't work cross-platform, but it might be enough for your use case.
For example I created file "test file.txt" on my drive D: so file path is 'D:/test file.txt'
Now I can open it with associated program with that script:
import os
os.startfile('d:/test file.txt')

Python escaping backslash

I'm trying to escape the backslash, but trying to understand the right way of doing it
foo = r'C:\Users\test.doc'
The above works fine
However, when I want to escape the path stored in a variable
For example :
parser.add_argument('-s', '--source', type = str, help = 'Source file path')
Now, how do escape the value in - args.source
So there are a few escape sequences to be aware of in python and they are mainly these.
So when the string for that file location is parsed by the add_argument method it may not be interpreted as a raw string like you've declared and there will be no way to escape the backslashes outside of the declaration.
What you can do instead is to keep it as a regular string (removing the 'r' prefix from the string) and using the escape character for a backslash in any places there may be conflict with another escape character (in this case \t). This may work as the method may evaluate the string correctly.
Try declaring your string like this.
foo = "C:\Users\\test.doc"
Hopefully this helps fix your issue!
EDIT:
In response to handling the dynamic file location you could maybe do something like the following!
def clean(s):
s = s.replace("\t", "\\t")
s = s.replace("\n", "\\n")
return s
Until you've covered all of your bases with what locations you may need to work with! This solution might be more appropriate for your needs. It's kind of funny I didn't think of doing it this way before. Hopefully this helps!

Formatting File Paths

I'm new to Python so I may be going about this completely wrong, but I'm having problems getting and changing to the directory of a file. My script takes in multiple file names that can be in any directory. In my script I need python to change to the directory of the file and then perform some actions. However, I'm having problems changing directories.
Here is what I've tried so far:
path=os.path.split(<file path>)
os.chdir(path[0])
<Do things to file specified by path[1]>
The way I've been getting the file path is by dragging from explorer to the command line. This enters the path name as something like "C:\foo\bar\file_name.txt" . When I run the first line in the interpreter I get out ('C:\\foo\bar','file_name.txt'). The problem is that for some reason the last backslash isn't automatically escaped so when I run the os.chdir(path[0]) line I get errors.
My question is why is the last backslash not being automatically escaped like the others? How can I manually escape the last backslash? Is there a better way to get the file's directory and change to it?
The last backslash isn't being automatically escaped because Python only escapes backslashes in regular strings when the following character does not form an escape sequence with the backslash. In fact, in your example, you would NOT get 'C:\\foo\bar' from 'C:\foo\bar', you'd get 'C:\x0coo\x08ar'.
What you want to do is either replace the backslashes with forwardslashes, or to make it simpler for drag-and-drop operations, just prepend the path with r so that it's a raw string and doesn't recognize the escape sequences.
>>> os.path.split(r"C:\foo\bar\file_name.txt")
('C:\\foo\\bar','file_name.txt')
You're using the right modules and methods. Just when you're putting that windows path in there, make the string a raw string, so your command should look like:
path=os.path.split(r'C:\foo\bar\file_name.txt')
Note the r in front of the first quote, that makes Python not treat the backslashes in the string as escape sequences.

Categories