When I use satpy to read Himawari-8 standard datas, the terminal always says " 'filenames' was provided but is empty. "
The Himawari-8 has 16 observation bands, and it take one full disk picture every 10 minutes so for one folder of observation result which contains 16 folders(one folder is one band), and each band is separated into 10 parts so in one band folder it has 10 files.
The format of Himawari-8 standard data is .dat, and the data named as
HS_aaa_yyyymmdd_hhnn_Bbb_cccc_Rjj_Skkll.DAT.
H08: Himawari-8
yyyy: Observation start time(year)
mm: Observation start time(month)
dd: Observation start time(day)
hh: Observation start time(hour)
nn: Observation start time(min.)
bb: Band number (01 – 16)
cccc: Observation area and number, FLDK: Full Disk
jj: Spatial resolution
kk: segment number (01 – 10)
ll: total number of segments (01 – 10)
e.g. HS_H08_20210518_1100_B01_FLDK_R10_S0110.DAT
These are the all message from vscode:
(my_satpy_env) E:\HK\Python>python Process_SatHima_Imagery.py
Traceback (most recent call last):
File "E:\HK\Python\Process_SatHima_Imagery.py", line 18, in <module>
scn = Scene(filenames=files, reader='ahi_hsd',filter_parameters={'start_time': datetime(2021,5,25,2,00), 'end_time': datetime(2021,5,25,2,10)})
File "C:\Users\RSFBioL\anaconda3\envs\my_satpy_env\lib\site-packages\satpy\scene.py", line 108, in __init__
self._readers = self._create_reader_instances(filenames=filenames,
File "C:\Users\RSFBioL\anaconda3\envs\my_satpy_env\lib\site-packages\satpy\scene.py", line 157, in _create_reader_instances
return load_readers(filenames=filenames,
File "C:\Users\RSFBioL\anaconda3\envs\my_satpy_env\lib\site-packages\satpy\readers\__init__.py", line 546, in load_readers
raise ValueError("'filenames' was provided but is empty.")
ValueError: 'filenames' was provided but is empty.
I use the code from Github and he uploaded the code " https://github.com/gSasikala/ftp-himawari8-hsd/blob/main/examples/Processing_Satellite_Imagery.ipynb "
I use the editor, vs code, to write and the python version is 3.9.7 and I have installed anaconda and satpy to my computer.
Sorry that I'm a newbie at Python and Himawari-8 standard data, if there anything I missed just remind me.
Thanks for any reply or recommand to me.
The error message means that you are passing an empty list to the Scene object. So in this line of code (you can see it in the error traceback):
scn = Scene(filenames=files, reader='ahi_hsd',filter_parameters={'start_time': datetime(2021,5,25,2,00), 'end_time': datetime(2021,5,25,2,10)})
your files variable is an empty list ([]). Satpy can't load anything because it wasn't given any files. The notebook you linked to creates the files variable with this line of code:
files = glob.glob(r'D:\ftp_h8_hsd_25Jul\*.dat')
Which means get a list of all of the files ending with .dat in the directory D:\ftp_h8_hsd_25Jul\. This is a directory (folder) on the author of the notebook's Windows machine. If you don't have these data files on your local machine then you will need to download them from somewhere. If you have the files, then you need to change the D:\ftp_h8_hsd_25Jul\*.dat part of the code to match your download location. For example, if you are on linux and downloaded the files using a browser they may be in /home/<username>/Downloads/*.dat.
It isn't clear to me what you are trying to do or what your starting point is so I'm not sure I can help much further. Stackoverflow isn't the most widely used support forum for Satpy questions. If you'd like to chat more I'd suggest joining the Pytroll slack (see http://pytroll.github.io/#getting-in-touch), we'd be happy to help.
Related
Working on a data transfer program, to move data from an oracle database to another
application that I cant see or change. I have to create several text files described below and drop them off on sftp site.
I am converting from a 20+ year old SQR report. (yes SQR) :(
I have to create text files that have a format as such an_alpa_code:2343,34533,4442,333335,.....can be thousands or numbers separated by comma.
The file may have only 1 line, but the file might be 48k in size.
There is no choice on the file format, it is required this way.
Tried using Oracle UTL_FILE, but that cannot deal with a line over 32k in length, so looking for an alterative. Python is a language my company has approved for use, so I am hoping it could do this
I too once [was forced] to use SQR many years ago, and so you have my sympathy.
python can definitely do this. If you set the end argument of the print command to an empty string, then you can ensure that no new line is output:
print("Hello world",end='')
perl could also be a good candidate language.
print("Hello world");
Both python and perl have Oracle client libraries.
This gave me one long line
file_obj = open("writing.txt", "w")
for i in range(0,10000):
file_obj.write("mystuff"+str(i)+",")
# file_obj.write('\n')
file_obj.close()
When I try to write something, such as variables, the code is renamed to the file name on the computer.
For example, if I write:
a = 20
f = 15
print(a+f)
then the code file will automatically be renamed to the first line, i.e. "a = 20"
Then, when I try to run the code, the program outputs nothing but "Python" and some incomprehensible words.
What could it be related to?
enter image description here
enter image description here
I installed the latest version of Visual Stuio Code with Python, they are new, so there should be no problems. But this time it went wrong.
After reinstalling the program, the problem remains.
First of all, if there is no special requirement, please do not use Code Runner to run the script, using the official extension Python is a better choice.
In addition, the dot on your file label means that you have not saved the file, you can add the following setting to enable automatic saving in the settings.
"files.autoSave": "afterDelay",
You may have created the file using the following method. File --> New File... --> Python File. At this time, the file has not been named, also not saved. You can see that there is no such file in the resource manager list at this time.
So the file label shows the first line of codes. This is a feature of vscode, you can refer to this link. And because the file has not been saved, there will be problems executing the script.
You can rename the script file directly (F2), or vscode will remind you to name the file when saving. Another way to create a file is to right click and choose New File..., enter filename and end with .py extension.
I am trying to replicate another researcher's findings by using the Python file that he added as a supplement to his paper. It is the first time I am diving into Python, so the error might be extremely simple to fix, yet after two days I haven't still. For context, in the Readme file there's the following instruction:
"To run the script, make sure Python2 is installed. Put all files into one folder designated as “cf_dir”.
In the script I get an error at the following lines:
if __name__ == '__main__':
cf_dir, cf_file, cf_phys_file = sys.argv[1:4]
os.chdir(cf_dir)
cf = pd.read_csv(cf_file)
cf_phys = pd.read_csv(cf_phys_file)
ValueError: need more than 0 values to unpack
The "cf_file" and "cf_phys_file" are two major components of all files that are in the one folder named "cf_dir". The "cf_phys_file" relates only to two survey question's (Q22 and Q23), and the "cf_file" includes all other questions 1-21. Now it seems that the code is meant to retrieve those two files from the directory? Only for the "cf_phys_file" the columns 1:4 are needed. The current working directory is already set at the right location.
The path where I located "cf_dir" is as follows:
C:\Users\Marc-Marijn Ossel\Documents\RSM\Thesis\Data\Suitable for ML\Data en Artikelen\Per task Suitability for Machine Learning score readme\cf_dir
Alternative option in readme file,
In the readme file there's this option, but also here I cannot understand how to direct the path to the right location:
"Run the following command in an open terminal (substituting for file names
below): python cfProcessor_AEAPnP.py cf_dir cf_file cf_phys_file task_file jobTaskRatingFile
jobDataFile OESfile
This should generate the data and plots as necessary."
When I run that in "Command Prompt", I get the following error, and I am not sure how to set the working directory correctly.
- python: can't open file 'cfProcessor_AEAPnP.py': [Errno 2] No such file or directory
Thanks for the reading, and I hope there's someone who could help me!
Best regards & stay safe out there during Corona!!
Marc
cf_dir, cf_file, cf_phys_file = sys.argv[1:4]
means, the python file expects few arguments when called.
In order to run
python cfProcessor_AEAPnP.py cf_dir cf_file cf_phys_file task_file jobTaskRatingFile jobDataFile OESfile
the command prompt should be in that folder.
So, open command prompt and type
cd path_to_the_folder_where_ur_python_file_is_located
Now, you would have reached the path of the python file.
Also, make sure you give full path in double quotes for the arguments.
I am using Python3.6 and I need to run my code in command line. The code works when I run it in PyCharm but when I use command line I get this error:
File "path", line 43, in <module>
rb = ds.GetRasterBand(1)
AttributeError: 'NoneType' object has no attribute 'GetRasterBand'
It seems that I have a problem with these lines:
ds = gdal.Open('tif_file.tif', gdal.GA_ReadOnly)
rb = ds.GetRasterBand(1)
img_array = rb.ReadAsArray()
Does anyone know what I might have done wrong?
EDIT
Some magic just happened. I tried to run my code this morning and everything seems fine. I guess what my computer needed was a restart or something. Thanks to you all for help.
from the gdal documentation:
from osgeo import gdal
dataset = gdal.Open(filename, gdal.GA_ReadOnly)
if not dataset:
...
Note that if GDALOpen() returns NULL it means the open failed, and
that an error messages will already have been emitted via CPLError().
If you want to control how errors are reported to the user review the
CPLError() documentation. Generally speaking all of GDAL uses
CPLError() for error reporting. Also, note that pszFilename need not
actually be the name of a physical file (though it usually is). It's
interpretation is driver dependent, and it might be an URL, a filename
with additional parameters added at the end controlling the open or
almost anything. Please try not to limit GDAL file selection dialogs
to only selecting physical files.
looks like the file you are trying to open is not a valid gdal file or some other magic is going on in the file selection. you could try to direct the program to a known good file online to test it.
I have written a program in Python 3.6 that makes use of IBM Watson's Speech to Text library. When the program searches a folder and reads through each .wav file individually, it's supposed to check the file's frequency and flag my IBM Watson integration differently. Then, it takes the response and maps it to a list. Through stub testing, the main, problematic code in question is here:
speech_to_text.set_detailed_response(True)
# Narrowband
if rate < 16000:
x = json.loads(
json.dumps(speech_to_text.recognize(audio_file, content_type='audio/basic', timestamps=True, max_alternatives=0).get_result(),
indent=2), object_hook=lambda d: namedtuple('X', d.keys())(*d.values())
)
# Broadband
else:
x = json.loads(
json.dumps(speech_to_text.recognize(audio_file, content_type='audio/wav', timestamps=True, max_alternatives=0).get_result(),
indent=2), object_hook=lambda d: namedtuple('X', d.keys())(*d.values())
)
This program is completely functional when I supply it with a file over 16 kbps. However, anything less than that, and I get this error:
File "echo_cli.py", line 64, in <module>
json.dumps(speech_to_text.recognize(audio_file, content_type='audio/basic', timestamps=True, max_alternatives=0).get_result(),
File "C:\Python37\lib\site-packages\watson_developer_cloud\speech_to_text_v1.py", line 373, in recognize
accept_json=True)
File "C:\Python37\lib\site-packages\watson_developer_cloud\watson_service.py", line 479, in request
info=error_info, httpResponse=response)
watson_developer_cloud.watson_service.WatsonApiException: Error: This 8000hz audio input requires a narrow band model. See https://<STT_API_ENDPOINT>/v1/models for a list of available models., Code: 400 , Information: {'code_description': 'Bad Request'} , X-dp-watson-tran-id: stream01-167902601 , X-global-transaction-id: f257b1145ba417780a01fd89
As a note, the files I'm using are over a network drive. However, I get the same error when I copy them to my local drive, so I'm thinking that this is an unrelated issue. I'm including this text just in case it rings any bells I'm unaware of.
According to this documentation, I should be able to accept a narrowband file with audio/basic, and according to print commands I've used, when I load a narrowband .wav, my program is executing the correct code. What am I doing wrong?
Thanks!
You should only pass the audio/basic MIME type if that's the type of the file you're uploading (also known as a "Sun .au" file, it's one of the oldest audio file types out there). If you're uploading a WAV file, specify the MIME type as audio/wav, no matter what the sample rate.