I am working with FITS files and to open a FITS file in the usual way I have to go through two commands:
hdu = fits.open('image1.fits')
data = hdu[0].data
Now I do realize that this is necessary in order to get the primary data of the FITS file from the header. But I was wondering if there was any way to convert it to a single command that would just get me the primary data of a FITS file just by mentioning the name of the file. (Or any other way to converting this two step operation into a single step operation.)
Apparently the answer was very straightforward and I found it myself after thinking about it a bit. All you have to do it write it like this:
data = fits.open('image1.fits')[0].data
There's nothing wrong with the other answers, but a more specific answer to this exists and it's getdata()
See also Convenience Functions.
Although you can put the command in one line as such:
data = fits.open('image1.fits')[0].data
Best practice is to use a with statement:
with open("image1.fits", "r") as f:
data = f[0].data
See What is the python “with” statement designed for?
Related
This is my very first question on stackoverflow, so I must beg your patience.
I believe there is something wrong with the format of a csv file I need to load into Python. I'm using a Jupyter Notebook. The link to the file is here.
It is from the World Inequality Database data portal.
I'm pretty sure the delimiter is a semi-colon ( sep=";" ) because the bottom half of the data renders neatly when I specify this argument. However the first half of the text in the file seems to make no sense. I have no idea how to tell the pd.read_csv() function how to read it. I suspect the first half of the data simply has terrible formatting. I've also tried header=None and sep="|" to no avail.
Any ideas or suggestions would be very helpful. Thank you very much!
This is common with speadsheets. You have have some commentary, tables may be inserted all over the place. It looks great to the content creator, but the CSV is a mess. You need to preprocess the CSV to create clean content for your analysis. In this case, its easy. The content starts at canned header and you can split the file there. If that header changes, you'll get an error and now its just one more sleepless night figuring out what they've done.
import itertools
canned_header_line = "Variable Code;country;year;perc;agdpro999i;"\
"npopul999i;mgdpro999i;inyixx999i;xlceux999i;xlcusx999i;xlcyux999i"
def scrub_WID_file(in_csv_filename, out_csv_filename):
with open(in_csv_filename) as in_file,\
open(out_csv_filename, 'w') as out_file:
out_file.writelines(itertools.dropwhile(
lambda line: line.strip() != canned_header_line,
in_fp))
if not os.stat.st_size:
raise ValueError("No recognized header in " + in_csv_filename)
What I'm trying to do is to ask a user for a name of a file to make and then save some stuff in this file.
My portion of the program looks like this:
if saving == 1:
ask=raw_input("Type the name file: ")
fileout=open(ask.csv,"w")
fileout.write(output)
I want the format to be .csv, I tried different options but can't seem to work.
The issue here is you need to pass open() a string. ask is a variable that contains a string, but we also want to append the other string ".csv" to it to make it a filename. In python + is the concatenation operator for strings, so ask+".csv" means the contents of ask, followed by .csv. What you currently have is looking for the csv attribute of the ask variable, which will throw an error.
with open(ask+".csv", "w") as file:
file.write(output)
You might also want to do a check first if the user has already typed the extension:
ask = ask if ask.endswith(".csv") else ask+".csv"
with open(ask, "w") as file:
file.write(output)
Note my use of the with statement when opening files. It's good practice as it's more readable and ensures the file is closed properly, even on exceptions.
I am also using the python ternary operator here to do a simple variable assignment based on a condition (setting ask to itself if it already ends in ".csv", otherwise concatenating it).
Also, this is presuming your output is already suitable for a CSV file, the extension alone won't make it CSV. When dealing with CSV data in general, you probably want to check out the csv module.
You need to use ask+'.csv' to concatenate the required extension on to the end of the user input.
However, simply naming the file with a .csv extension is not enough to make it a comma-separated file. You need to format the output. Use csvwriter to do that. The python documentation has some simple examples on how to do this.
I advise you not to attempt to generate the formatted comma-separated output yourself. That's a surprisingly hard task and utterly pointless in the presence of the built-in functionality.
Your variable ask is gonna be of type string after the raw_input.
So, if you want to append the extension .csv to it, you should do:
fileout = open(ask + ".csv", "w")
That should work.
I want to copy the contents of a JSON file in another JSON file, with Python
Any ideas ?
Thank you :)
Given the lack of research effort, I normally wouldn't answer, but given the poor suggestions in comments, I'll bite and give a better option.
Now, this largely depends on what you mean, do you wish to overwrite the contents of one file with another, or insert? The latter can be done like so:
with open("from.json", "r") as from, open("to.json", "r") as to:
to_insert = json.load(from)
destination = json.load(to)
destination.append(to_insert) #The exact nature of this line varies. See below.
with open("to.json", "w") as to:
json.dump(to, destination)
This uses python's json module, which allows us to do this very easily.
We open the two files for reading, then open the destination file again in writing mode to truncate it and write to it.
The marked line depends on the JSON data structure, here I am appending it to the root list element (which could not exist), but you may want to place it at a particular dict key, or somesuch.
In the case of replacing the contents, it becomes easier:
with open("from.json", "r") as from, open("to.json", "w") as to:
to.write(from.read())
Here we literally just read the data out of one file and write it into the other file.
Of course, you may wish to check the data is JSON, in which case, you can use the JSON methods as in the first solution, which will throw exceptions on invalid data.
Another, arguably better, solution to this could also be shutil's copy methods, which would avoid actually reading or writing the file contents manually.
Using the with statement gives us the benefit of automatically closing our files - even if exceptions occur. It's best to always use them where we can.
Note that in versions of Python before 2.7, multiple context managers are not handled by the with statement, instead you will need to nest them:
with open("from.json", "r") as from:
with open("to.json", "r+") as to:
...
I'm used to C++, and I build my data handling classes/functions to handle stream objects instead of files. I'd like to know how I might modify the following code, so that it can handle a stream of binary data in memory, rather than a file handle.
def get_count(self):
curr = self.file.tell()
self.file.seek(0, 0)
count, = struct.unpack('I', self.file.read(c_uint32_size))
self.file.seek(curr, 0)
return count
In this case, the code assumes self.file is a file, opened like so:
file = open('somefile.data, 'r+b')
How might I use the same code, yet instead do something like this:
file = get_binary_data()
Where get_binary_data() returns a string of binary data. Although the code doesn't show it, I also need to write to the stream (I didn't think it was worth posting the code for that).
Also, if possible, I'd like the new code to handle files as well.
You can use an instance of StringIO.StringIO (or cStringIO.StringIO, faster) to give a file-like interface to in-memory data.
Take a look at Python's StringIO module, docs here, which could be pretty much what you're after.
Have a look at 'StringIO' (Read and write strings as files)
Use StringIO.
I like the timing of the answer. (except mine)
We can see response time in milliseconds ?
of-course StringIO
How to get the inputs from excel and use those inputs in python.
Take a look at xlrd
This is the best reference I found for learning how to use it: http://www.dev-explorer.com/articles/excel-spreadsheets-and-python
Not sure if this is exactly what you're talking about, but:
If you have a very simple excel file (i.e. basically just one table filled with string-values, nothing fancy), and all you want to do is basic processing, then I'd suggest just converting it to a csv (comma-seperated value file). This can be done by "saving as..." in excel and selecting csv.
This is just a file with the same data as the excel, except represented by lines seperated with commas:
cell A:1, cell A:2, cell A:3
cell B:1, cell B:2, cell b:3
This is then very easy to parse using standard python functions (i.e., readlines to get each line of the file, then it's just a list that you can split on ",").
This if of course only helpful in some situations, like when you get a log from a program and want to quickly run a python script which handles it.
Note: As was pointed out in the comments, splitting the string on "," is actually not very good, since you run into all sorts of problems. Better to use the csv module (which another answer here teaches how to use).
import win32com
Excel=win32com.client.Dispatch("Excel.Application")
Excel.Workbooks.Open(file path)
Cells=Excel.ActiveWorkBook.ActiveSheet.Cells
Cells(row,column).Value=Input
Output=Cells(row,column).Value
If you can save as a csv file with headers:
Attrib1, Attrib2, Attrib3
value1.1, value1.2, value1.3
value2,1,...
Then I would highly recommend looking at built-in the csv module
With that you can do things like:
csvFile = csv.DictReader(open("csvFile.csv", "r"))
for row in csvFile:
print row['Attrib1'], row['Attrib2']