We Keep Coding

Why is the raw file path not working in python?

I have looked and tried different methods posted online but I cant get this to work.
This is my file path:
file_path = '\\something.com\1400_somethingelse\1400_somethingyes\1400_Design\1500_sketch\ShotCam'
This file path is going to change multiple times so I need this to become a variable.
I have tried using the following in order to convert it into a raw file as another variable:
the r before to get the raw file path and that works but what happens if this file path is going to be change? Naturally I would need a variable
I have used the repr(file_path) and that works in some cases but when I try to use the os.listdir it doesnt!
I have tried the r'%s' %file_path as well but nothing
Im sure its something very simple so please let me know if someone has an answer for this!
Thank you!
**UPDATE
This is the code Im having problems with:
variable = '\\something.com\1400_somethingelse\1400_somethingyes\1400_Design\1500_sketch\ShotCam'
#Result:\something.com`0_somethingelse`0_somethingyes`0_Designh0_sketch\ShotCam
raw_path= repr(variable)
#Result:'\\something.com\x0somethingelse\x0somethingyes\x0Designh\x00_sketch\\ShotCam'
list_dir = os.listdir(raw_path)
print list_dir
#Result: # Error: WindowsError: file <maya console> line 8: 3 #
To make it a raw path I know I have to do:
variable = r'\\something.com\1400_somethingelse\1400_somethingyes\1400_Design\1500_sketch\ShotCam'
But I want this to become a variable so I can change it multiple times.

You can verify that your string is not a valid path by just printing it.
>>> '\\something.com\1400_somethingelse\1400_somethingyes\1400_Design\1500_sketch\ShotCam'
'\\something.com`0_somethingelse`0_somethingyes`0_Designh0_sketch\\ShotCam'
>>> r'\\something.com\1400_somethingelse\1400_somethingyes\1400_Design\1500_sketch\ShotCam'
'\\\\something.com\\1400_somethingelse\\1400_somethingyes\\1400_Design\\1500_sketch\\ShotCam'
use r to escape those \.

how to "source" file into python script

I have a text file /etc/default/foo which contains one line:
FOO="/path/to/foo"
In my python script, I need to reference the variable FOO.
What is the simplest way to "source" the file /etc/default/foo into my python script, same as I would do in bash?
. /etc/default/foo

Same answer as #jil however, that answer is specific to some historical version of Python.
In modern Python (3.x):
exec(open('filename').read())
replaces execfile('filename') from 2.x

You could use execfile:
execfile("/etc/default/foo")
But please be aware that this will evaluate the contents of the file as is into your program source. It is potential security hazard unless you can fully trust the source.
It also means that the file needs to be valid python syntax (your given example file is).

Keep in mind that if you have a "text" file with this content that has a .py as the file extension, you can always do:
import mytextfile
print(mytestfile.FOO)
Of course, this assumes that the text file is syntactically correct as far as Python is concerned. On a project I worked on we did something similar to this. Turned some text files into Python files. Wacky but maybe worth consideration.

Just to give a different approach, note that if your original file is setup as
export FOO=/path/to/foo
You can do source /etc/default/foo; python myprogram.py (or . /etc/default/foo; python myprogram.py) and within myprogram.py all the values that were exported in the sourced' file are visible in os.environ, e.g
import os
os.environ["FOO"]

If you know for certain that it only contains VAR="QUOTED STRING" style variables, like this:
FOO="some value"
Then you can just do this:
>>> with open('foo.sysconfig') as fd:
... exec(fd.read())
Which gets you:
>>> FOO
'some value'
(This is effectively the same thing as the execfile() solution
suggested in the other answer.)
This method has substantial security implications; if instead of FOO="some value" your file contained:
os.system("rm -rf /")
Then you would be In Trouble.
Alternatively, you can do this:
>>> with open('foo.sysconfig') as fd:
... settings = {var: shlex.split(value) for var, value in [line.split('=', 1) for line in fd]}
Which gets you a dictionary settings that has:
>>> settings
{'FOO': ['some value']}
That settings = {...} line is using a dictionary comprehension. You could accomplish the same thing in a few more lines with a for loop and so forth.
And of course if the file contains shell-style variable expansion like ${somevar:-value_if_not_set} then this isn't going to work (unless you write your very own shell style variable parser).

There are a couple ways to do this sort of thing.
You can indeed import the file as a module, as long as the data it contains corresponds to python's syntax. But either the file in question is a .py in the same directory as your script, either you're to use imp (or importlib, depending on your version) like here.
Another solution (that has my preference) can be to use a data format that any python library can parse (JSON comes to my mind as an example).
/etc/default/foo :
{"FOO":"path/to/foo"}
And in your python code :
import json
with open('/etc/default/foo') as file:
data = json.load(file)
FOO = data["FOO"]
## ...
file.close()
This way, you don't risk to execute some uncertain code...
You have the choice, depending on what you prefer. If your data file is auto-generated by some script, it might be easier to keep a simple syntax like FOO="path/to/foo" and use imp.
Hope that it helps !

The Solution
Here is my approach: parse the bash file myself and process only variable assignment lines such as:
FOO="/path/to/foo"
Here is the code:
import shlex
def parse_shell_var(line):
"""
Parse such lines as:
FOO="My variable foo"
:return: a tuple of var name and var value, such as
('FOO', 'My variable foo')
"""
return shlex.split(line, posix=True)[0].split('=', 1)
if __name__ == '__main__':
with open('shell_vars.sh') as f:
shell_vars = dict(parse_shell_var(line) for line in f if '=' in line)
print(shell_vars)
How It Works
Take a look at this snippet:
shell_vars = dict(parse_shell_var(line) for line in f if '=' in line)
This line iterates through the lines in the shell script, only process those lines that has the equal sign (not a fool-proof way to detect variable assignment, but the simplest). Next, run those lines into the function parse_shell_var which uses shlex.split to correctly handle the quotes (or the lack thereof). Finally, the pieces are assembled into a dictionary. The output of this script is:
{'MOO': '/dont/have/a/cow', 'FOO': 'my variable foo', 'BAR': 'My variable bar'}
Here is the contents of shell_vars.sh:
FOO='my variable foo'
BAR="My variable bar"
MOO=/dont/have/a/cow
echo $FOO
Discussion
This approach has a couple of advantages:
It does not execute the shell (either in bash or in Python), which avoids any side-effect
Consequently, it is safe to use, even if the origin of the shell script is unknown
It correctly handles values with or without quotes
This approach is not perfect, it has a few limitations:
The method of detecting variable assignment (by looking for the presence of the equal sign) is primitive and not accurate. There are ways to better detect these lines but that is the topic for another day
It does not correctly parse values which are built upon other variables or commands. That means, it will fail for lines such as:
FOO=$BAR
FOO=$(pwd)

Based off the answer with exec(.read()), value = eval(.read()), it will only return the value. E.g.
1 + 1: 2
"Hello Word": "Hello World"
float(2) + 1: 3.0

Python: Adding Variable paths into os.system("open"+filename)

I'm trying to make a script where python goes through a directory, finds all files that ends with *ref.SDAT, and then opens them all.
So far, I have the sorting of files process. The wanted files are put into reflist:
import os
import glob
import subprocess
os.chdir("/Users/BabyJ/Desktop/MRSDATA")
reflist = glob.glob('*raw_ref.SDAT')
print "These are the ref files:"
for i in reflist:
os.system('open+%s') %i
I don't know how to structure the syntax so that os.system will open all of the components of the list though.

The % operator wants the 'open+%s' string as its left-hand side. You are offering it the return value of os.system(). Also, I think you wanted a space, not a + in the string.
Try this:
os.system('open %s'%i)

I assuming judging from your use of open that you are on a Mac or Unix system. If that is the case use either of the following to get you up and running.
for i in reflist:
os.system('open ' + '%s' % i)
or:
for i in reflist:
subprocess.call('open ' + '%s' % i, shell = True)
Using subprocess is the better solution as os.system is, though not technically gone from the language, deprecated in Python per the official documentation.
Hope that helps.
EDIT:
If you're using windows sub in start for open.

Python: Echoing to a File (like Bash)

I have a simple bash command here for a script that I am re-writing in Python, and I've done a lot of searching and haven't found a simple answer. I am trying to echo the output of Print to a file, making sure there are no line breaks and that I can pass a variable into it. Here is just a little snippet (there are a lot of lines like this):
echo " ServerName www.${hostName}" >> $prjFile
Now I know it would end up looking something like:
print ("ServerName www.", hostName) >> prjFile
Right? But that doesn't work. Mind you, this is in Python 2.6 (as the machine this script will run on is using that version and there are other dependencies reliant on sticking with that version).

The syntax is;
print >>myfile, "ServerName www.", hostName,
where myfile is a file object opened in mode "a" (for "append").
The trailing comma prevents line breaks.
You might also want to use sys.stdout.softspace = False to prevent the spaces that Python adds between comma-separate arguments to print, and/or to print things as a single string:
print >>myfile, "ServerName www.%s" % hostName,

You can try a simple:
myFile = open('/tmp/result.file', 'w') # or 'a' to add text instead of truncate
myFile.write('whatever')
myFile.close()
In your case:
myFile = open(prjFile, 'a') # 'a' because you want to add to the existing file
myFile.write('ServerName www.{hostname}'.format(hostname=hostname))
myFile.close()

Newbie question about file formatting in Python

I'm writing a simple program in Python 2.7 using pycURL library to submit file contents to pastebin.
Here's the code of the program:
#!/usr/bin/env python2
import pycurl, os
def send(file):
print "Sending file to pastebin...."
curl = pycurl.Curl()
curl.setopt(pycurl.URL, "http://pastebin.com/api_public.php")
curl.setopt(pycurl.POST, True)
curl.setopt(pycurl.POSTFIELDS, "paste_code=%s" % file)
curl.setopt(pycurl.NOPROGRESS, True)
curl.perform()
def main():
content = raw_input("Provide the FULL path to the file: ")
open = file(content, 'r')
send(open.readlines())
return 0
main()
The output pastebin looks like standard Python list: ['string\n', 'line of text\n', ...] etc.
Is there any way I could format it so it looks better and it's actually human-readable? Also, I would be very happy if someone could tell me how to use multiple data inputs in POSTFIELDS. Pastebin API uses paste_code as its main data input, but it can use optional things like paste_name that sets the name of the upload or paste_private that sets it private.

First, use .read() as virhilo said.
The other step is to use urllib.urlencode() to get a string:
curl.setopt(pycurl.POSTFIELDS, urllib.urlencode({"paste_code": file}))
This will also allow you to post more fields:
curl.setopt(pycurl.POSTFIELDS, urllib.urlencode({"paste_code": file, "paste_name": name}))

import pycurl, os
def send(file_contents, name):
print "Sending file to pastebin...."
curl = pycurl.Curl()
curl.setopt(pycurl.URL, "http://pastebin.com/api_public.php")
curl.setopt(pycurl.POST, True)
curl.setopt(pycurl.POSTFIELDS, "paste_code=%s&paste_name=%s" \
% (file_contents, name))
curl.setopt(pycurl.NOPROGRESS, True)
curl.perform()
if __name__ == "__main__":
content = raw_input("Provide the FULL path to the file: ")
with open(content, 'r') as f:
send(f.read(), "yournamehere")
print
When reading files, use the with statement (this makes sure your file gets closed properly if something goes wrong).
There's no need to be having a main function and then calling it. Use the if __name__ == "__main__" construct to have your script run automagically when called (unless when importing this as a module).
For posting multiple values, you can manually build the url: just seperate different key, value pairs with an ampersand (&). Like this: key1=value1&key2=value2. Or you can build one with urllib.urlencode (as others suggested).
EDIT: using urllib.urlencode on strings which are to be posted makes sure content is encoded properly when your source string contains some funny / reserved / unusual characters.

use .read() instead of .readlines()

The POSTFIELDS should be sended the same way as you send Query String arguments. So, in the first place, it's necessary to encode the string that you're sending to paste_code, and then, using & you could add more POST arguments.
Example:
paste_code=hello%20world&paste_name=test
Good luck!