Can you write an h5 file from an h5py object? - python

The documentation on h5py is very good https://docs.h5py.org/en/stable/quick.html though I cannot see an example or way of achieving what I need to do..
I want to read an h5 file in, adjust it in some way and write the new file to a separate location without changing the original. Is this possible? The following option doesn't suit my use case as the files are in a workflow:
f = h5py.File('myfile.hdf5','r+')
Ideally I would do something like:
f = h5py.File('myfile.hdf5','r')
... do something to f
f.write('newfile.hdf5')
Any help appreciated!

Related

Can I read a FITS file in a single command?

I am working with FITS files and to open a FITS file in the usual way I have to go through two commands:
hdu = fits.open('image1.fits')
data = hdu[0].data
Now I do realize that this is necessary in order to get the primary data of the FITS file from the header. But I was wondering if there was any way to convert it to a single command that would just get me the primary data of a FITS file just by mentioning the name of the file. (Or any other way to converting this two step operation into a single step operation.)
Apparently the answer was very straightforward and I found it myself after thinking about it a bit. All you have to do it write it like this:
data = fits.open('image1.fits')[0].data
There's nothing wrong with the other answers, but a more specific answer to this exists and it's getdata()
See also Convenience Functions.
Although you can put the command in one line as such:
data = fits.open('image1.fits')[0].data
Best practice is to use a with statement:
with open("image1.fits", "r") as f:
data = f[0].data
See What is the python “with” statement designed for?

How to read Json files in a directory separately with a for loop and performing a calculation

Update: Sorry it seems my question wasn't asked properly. So I am analyzing a transportation network consisting of more than 5000 links. All the data included in a big CSV file. I have several JSON files which each consist of subset of this network. I am trying to loop through all the JSON files INDIVIDUALLY (i.e. not trying to concatenate or something), read the JSON file, extract the information from the CVS file, perform calculation, and save the information along with the name of file in new dataframe. Something like this:
enter image description here
This is the code I wrote, but not sure if it's efficient enough.
name=[]
percent_of_truck=[]
path_to_json = \\directory
import glob
z= glob.glob(os.path.join(path_to_json, '*.json'))
for i in z:
with open(i, 'r') as myfile:
l=json.load(myfile)
name.append(i)
d_2019= final.loc[final['LINK_ID'].isin(l)] #retreive data from main CSV file
avg_m=(d_2019['AADTT16']/d_2019['AADT16']*d_2019['Length']).sum()/d_2019['Length'].sum() #calculation
percent_of_truck.append(avg_m)
f=pd.DataFrame()
f['Name']=name
f['% of truck']=percent_of_truck
I'm assuming here you just want a dictionary of all the JSON. If so, use the JSON library ( import JSON). If so, this code may be of use:
import json
def importSomeJSONFile(f):
return json.load(open(f))
# make sure the file exists in the same directory
example = importSomeJSONFile("example.json")
print(example)
#access a value within this , replacing key with what you want like "name"
print(JSON_imported[key])
Since you haven't added any Schema or any other specific requirements.
You can follow this approach to solve your problem, in any language you prefer
Get Directory of the JsonFiles, which needs to be read
Get List of all files present in directory
For each file-name returned in Step2.
Read File
Parse Json from String
Perform required calculation

Duplicating PDF file by variable

I'm working on a project and I've ran into a brick wall.
We have a text file which has a numeric value and pdf file name. If the value is 6 we need to print 6 copies of the PDF. I was thinking of creating X copies of the PDF per line then combining them after. I'm not sure this is the most efficient way to do it but was wondering if anyone else has another idea.
DATA
1,PDF1
2,PDF5
7,PDF2
923,PDF33
You should be using the python CSV module to read in your data into two variables, numCopies and filePath https://docs.python.org/2/library/csv.html
You can then just use
for i in range(1, numCopies):
shutil.copyfile(filePath, newFilePath)
or something along those lines.
If you want to physically work with the PDF files I'd recommend the pyPdf module.

Python - Open subset of a binary file

In python if I want to open something in binary I simply do:
f = open("myfile.bin", "rb")
Then f is an handler, and I can use it to read, detect end of file, seek etc.
Now, what if I wanted to do this on a subset of myfile.bin? Something like:
f = open("myfile.bin", "rb", beg=1000, end=1100)
So that I end up with an handler to some sort of virtual file, of length 100 bytes, that actually maps to a subset of the original file?
I know that if I was to work on the file's content it would be easy: I could just seek around and be careful. But there are several libraries that ask for file handles as inputs, and I would like them to work only on subset of files. Again, I know that I could make a copy of it and make the library read that, but I am working on quite huge files and I would prefer not to have to duplicate files.
Thank you very much!

How to copy a JSON file in another JSON file, with Python

I want to copy the contents of a JSON file in another JSON file, with Python
Any ideas ?
Thank you :)
Given the lack of research effort, I normally wouldn't answer, but given the poor suggestions in comments, I'll bite and give a better option.
Now, this largely depends on what you mean, do you wish to overwrite the contents of one file with another, or insert? The latter can be done like so:
with open("from.json", "r") as from, open("to.json", "r") as to:
to_insert = json.load(from)
destination = json.load(to)
destination.append(to_insert) #The exact nature of this line varies. See below.
with open("to.json", "w") as to:
json.dump(to, destination)
This uses python's json module, which allows us to do this very easily.
We open the two files for reading, then open the destination file again in writing mode to truncate it and write to it.
The marked line depends on the JSON data structure, here I am appending it to the root list element (which could not exist), but you may want to place it at a particular dict key, or somesuch.
In the case of replacing the contents, it becomes easier:
with open("from.json", "r") as from, open("to.json", "w") as to:
to.write(from.read())
Here we literally just read the data out of one file and write it into the other file.
Of course, you may wish to check the data is JSON, in which case, you can use the JSON methods as in the first solution, which will throw exceptions on invalid data.
Another, arguably better, solution to this could also be shutil's copy methods, which would avoid actually reading or writing the file contents manually.
Using the with statement gives us the benefit of automatically closing our files - even if exceptions occur. It's best to always use them where we can.
Note that in versions of Python before 2.7, multiple context managers are not handled by the with statement, instead you will need to nest them:
with open("from.json", "r") as from:
with open("to.json", "r+") as to:
...

Categories