We Keep Coding

python gdal not processing grib file properly on CENTOS linux

I am trying to use python gdal to process a grib2 file into a geotiff file based on the following example:
https://geoexamples.com/d3-raster-tools-docs/code_samples/vardah.html
based on this I have the following code:
import gdal
import osr
ds = gdal.Open(r"/home/test/gfs.t12z.sfluxgrbf000.grib2",1)
gph = ds.GetRasterBand(84).ReadAsArray()
press = ds.GetRasterBand(54).ReadAsArray() / 100
temp = ds.GetRasterBand(52).ReadAsArray()
u = ds.GetRasterBand(50).ReadAsArray()
v = ds.GetRasterBand(51).ReadAsArray()
corr_press = press * (1 - (0.0065*gph/(0.0065*gph + temp + 273.15)))**-5.257
driver = gdal.GetDriverByName('GTiff')
outRaster = driver.Create("/home/test/vardah2.tiff", ds.RasterXSize, ds.RasterYSize, 4, gdal.GDT_Float32)
outRaster.SetGeoTransform(ds.GetGeoTransform())
outband = outRaster.GetRasterBand(1)
outband.WriteArray(corr_press)
outband.SetMetadata({'name': 'press'})
outRasterSRS = osr.SpatialReference()
outRasterSRS.ImportFromEPSG(4326)
outRaster.SetProjection(outRasterSRS.ExportToWkt())
outband.FlushCache()
I am attempting to do this on centos 7 and when I run the program I get the following error:
ERROR 6: The GRIB driver does not support update access to existing datasets.
Traceback (most recent call last):
File "ficky.py", line 4, in <module>
gph = ds.GetRasterBand(84).ReadAsArray()
AttributeError: 'NoneType' object has no attribute 'GetRasterBand'
How do I resolve this error to get a successful run of this script on a centos interface?

Change
ds = gdal.Open(r"/home/test/gfs.t12z.sfluxgrbf000.grib2",1)
to
ds = gdal.Open("/home/test/gfs.t12z.sfluxgrbf000.grib2")
or even better
ds = gdal.Open("/home/test/gfs.t12z.sfluxgrbf000.grib2", GA_ReadOnly)

Stuck with pytube

Oh my python guys, please help me.
One of my python projects suddenly stopped working for no reason.
I'm using pytube module and when i try to run the code i get this error:
Traceback (most recent call last):
File "C:\Users\giova\AppData\Local\Programs\Python\Python39\lib\site-packages\pytube\contrib\search.py", line 94, in fetch_and_parse
sections = raw_results['contents']['twoColumnSearchResultsRenderer'][
KeyError: 'twoColumnSearchResultsRenderer' fetch_and_parse
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\giova\OneDrive\Desktop\Coding\Python\youtubeapp.py", line 38, in <module>
videoSearch()
File "C:\Users\giova\OneDrive\Desktop\Coding\Python\youtubeapp.py", line 21, in videoSearch
availableResults = len(vid.results)
File "C:\Users\giova\AppData\Local\Programs\Python\Python39\lib\site-packages\pytube\contrib\search.py", line 62, in results
videos, continuation = self.fetch_and_parse() results
File "C:\Users\giova\AppData\Local\Programs\Python\Python39\lib\site-packages\pytube\contrib\search.py", line 97, in fetch_and_parse fetch_and_parse
sections = raw_results['onResponseReceivedCommands'][0][
KeyError: 'onResponseReceivedCommands'
This is not even the only error i get, sometimes i got "http error 410: gone" error or some like this. I haven't changed the code for about two weeks (it was working two weeks ago) and it stopped working. I don't know what is happening to my code.
This is the full code:
from pytube import Search, YouTube
print("================================\n What do you want to do?: ")
availableChoose = [
'1 Search videos',
'...',
'================================'
]
for choose in availableChoose:
print(choose)
userChoose = input()
userChoose = userChoose.lower()
def videoSearch():
userSearch = input("Enter the title of the video you want to search: ")
vid = Search(userSearch)
availableResults = len(vid.results)
strAvailableResults = str(availableResults)
print("The available results are " + strAvailableResults)
vidResultsList = vid.results
vidResultsList = str(vidResultsList)
vidResultsList = vidResultsList.replace("<pytube.__main__.YouTube object: videoId=", "")
vidResultsList = vidResultsList.replace(">", "")
vidResultsList = vidResultsList.replace("[", "")
vidResultsList = vidResultsList.replace("]", "")
vidResultsList = vidResultsList.replace(" ", "")
vidResultsList = vidResultsList.split(',')
for vidResultsObject in vidResultsList:
vidLink = ("https://www.youtube.com/watch?v=" + vidResultsObject)
vidTempObject = YouTube(vidLink)
print(vidTempObject.title + " - " + vidLink)
if(userChoose == "search" or userChoose == "search video" or userChoose == "search videos" or userChoose == "1"):
videoSearch()

pytube's 11.0.0 API docs list the Search.fetch_and_parse() method.
The corresponding source-code shows that it internally handles a KeyError for onResponseReceivedCommands:
# Initial result is handled by try block, continuations by except block
try:
sections = raw_results['contents']['twoColumnSearchResultsRenderer'][
'primaryContents']['sectionListRenderer']['contents']
except KeyError:
sections = raw_results['onResponseReceivedCommands'][0][
'appendContinuationItemsAction']['continuationItems']
This method is an inner one, that is used by your vid.results call.
Might be, that the Youtube API has changed there response and your version of pytube is not fitting anymore.
Bug already filed
See pytube issue #1082 and issue #1106.
Meanwhile use another branch
tfdahlin's fork has a bugfixed version. It was already proposed as Pull-Request to the project:
PR #1090, opened on 2021-08-13 (currently waiting for approval).

I am also struggling to get around the search results provided by pytube Search module. Since search results are looking like objects, I was thinking (lazily) that I cannot convert the object list in to strings.
After modifying your function as below, the results are printed as youtube links.
'''
def videoSearch(search_list): #provide the search.results list as input
for result in search_list: #Begin video id extraction
result = str(result)
temp = result.replace("<pytube.__main__.YouTube object: videoId=", "")
temp = temp.split(',')[0] #get the correct video id
vidLink = ("https://www.youtube.com/watch?v=" + temp)
print(vidLink)
'''
Try it out.

CiscoConfigParser & Functions

I am trying to make the code cleaner and better manageable and i wanted to start with reading a cisco file. However when i try to put it in a function, it is not able to give me the outputs. The same works perfectly out of a function
Working model
parse = CiscoConfParse("C:\\python\\mydata\\TestConfigFile.txt")
TCPSrv = parse.find_objects("service\stcp\sdestination\seq")
UDPSrv = parse.find_objects("service\sudp\sdestination\seq")
ObjectNetwork = parse.find_objects("^object\snetwork\s")
ObjectGroupSrv = parse.find_objects("^object-group\sservice")
ObjectGroupNetwork = parse.find_objects("^object-group\snetwork\s")
This creates a list for all the above like the one below
TCPSrv = [<IOSCfgLine # 83 ' service tcp destination eq https' (parent is # 82)>,<IOSCfgLine # 97 ' service tcp destination eq www '(parent is # 102)>]
But when i put this into a function, it does not work. This is the first time i am trying out to use functions, and I know that i am doing something wrong.
This is my code for Functions
def cisco(filename):
parse = CiscoConfParse(filename)
TCPSrv = parse.find_objects("service\stcp\sdestination\seq")
UDPSrv = parse.find_objects("service\sudp\sdestination\seq")
ObjectNetwork = parse.find_objects("^object\snetwork\s")
ObjectGroupSrv = parse.find_objects("^object-group\sservice")
ObjectGroupNetwork = parse.find_objects("^object-group\snetwork\s")
return TCPSrv, UDPSrv, ObjectNetwork, ObjectGroupSrv, ObjectGroupNetwork
file = C:\\python\\mydata\\TestConfigFile.txt
cisco(file)
This does not give any output.
>>> TCPSrc
Traceback (most recent call last):
File "<input>", line 1, in <module>
NameError: name 'TCPSrc' is not defined
I have tried putting it like this below also
cisco("C:\\python\\mydata\\TestConfigFile.txt")
Can someone kindly assist what I am doing wrong.

This does not give any output
>>> TCPSrc
Traceback (most recent call last):
File "", line 1, in
NameError: name 'TCPSrc' is not defined
You have not assigned the return values to anything. When you call cisco(), you need to assign the return values to something... please use:
from ciscoconfparse import CiscoConfParse
def cisco(filename):
parse = CiscoConfParse(filename)
TCPSrv = parse.find_objects("service\stcp\sdestination\seq")
UDPSrv = parse.find_objects("service\sudp\sdestination\seq")
ObjectNetwork = parse.find_objects("^object\snetwork\s")
ObjectGroupSrv = parse.find_objects("^object-group\sservice")
ObjectGroupNetwork = parse.find_objects("^object-group\snetwork\s")
return TCPSrv, UDPSrv, ObjectNetwork, ObjectGroupSrv, ObjectGroupNetwork
values = cisco("C:\\python\\mydata\\TestConfigFile.txt")
TCPsrv = values[0]
UDPsrv = values[1]
# ... etc unpack the remaining values as illustrated above

Traceback (most recent call last): File "newdata.py", line 51, in <module> temp_res.update(i['statistics']) KeyError: 'statistics'

search_response = youtube.search().list(q=options.q, type='video',
part='id,snippet', maxResults=options.max_results).execute()
videos = {}
# Add each result to the appropriate list, and then display the lists of
# matching videos.
# Filter out channels, and playlists.
for search_result in search_response.get('items', []):
if search_result['id']['kind'] == 'youtube#video':
# videos.append("%s" % (search_result["id"]["videoId"]))
videos[search_result['id']['videoId']] = search_result['snippet'
]['title']
# print "Videos:\n", "\n".join(videos), "\n"
s = ','.join(videos.keys())
videos_list_response = youtube.videos().list(id=s,
part='id,statistics,snippet').execute()
res = []
for i in videos_list_response['items']:
tempres = dict(v_id=i['id'], v_title=videos[i['id']])
tempres.update(i['snippet'])
tempres.update(i['statistics'])
res.append(tempres)
Please enter the search term for videos: football
Your search term is: football
Traceback (most recent call last):
File "data_pull.py", line 61, in
tempres.update(i['statistics'])
KeyError: 'statistics'
Need help to solve this error. got this code from https://www.analyticsvidhya.com/blog/2014/09/mining-youtube-python-social-media-analysis/#comment-126365 and not able to solve this error.
this file ran before error free, but after 4-5 days this error has popped up without changing the code.
Its weird but help would be appreciated!
Thanks

Because with some videos, you cannot get its statistics.
Try with Search term = Lac Troi Son Tung MTP and you will find out this video does not have statistics attribute.

Weird TypeError in jython/python?

I'm bukkit jython/python plugin programmer. Last few days, I'm struggling with this problem. I have to add an potion effect to an user, it's not problem if I enter effect name, duration and amplifier manually in code, but when I want to get them from config, I get this error:
13:38:20 [SEVERE] Could not pass event PlayerInteractEvent to ItemEffect v1.0
Traceback (most recent call last):
File "<iostream>", line 126, in onPlayerInteractEvent
TypeError: addPotionEffect(): 1st arg can't be coerced to org.bukkit.potion.Poti
onEffect
Here's that part of code:
effectname = section.getString("%s.effect"%currentKey)
duration = section.getInt("%s.duration"%currentKey)
durationinticks = duration * 20
geteffectname = "PotionEffectType.%s"%effectname
getpotioneffect = "PotionEffect(%s, %i, 1)"%(geteffectname, durationinticks)
geteffectname = "PotionEffectType.%s"%effectname
if iteminhand == currentKey:
event.getPlayer().addPotionEffect(getpotioneffect)
When I print getpotioneffect out, I get:
13:38:20 [INFO] PotionEffect(PotionEffectType.SPEED, 600, 1)
which is okay, and should work. I tested it without getting informations from config, and it works perfectly... To sum up, code above is not working, but below one works:
getpotioneffect = PotionEffect(PotionEffectType.SPEED, 600, 1)
if iteminhand == currentKey:
event.getPlayer().addPotionEffect(getpotioneffect)
Link to javadocs of this event!
http://jd.bukkit.org/rb/apidocs/org/bukkit/entity/LivingEntity.html#addPotionEffect(org.bukkit.potion.PotionEffect)
Thanks!

In your first snippet, getpotioneffect is a string. You can check it adding print type(getpotioneffect) somewhere.
What you want is to replace this :
geteffectname = "PotionEffectType.%s"%effectname
getpotioneffect = "PotionEffect(%s, %i, 1)"%(geteffectname, durationinticks)
with this:
effect_type = getattr(PotionEffectType, effectname)
potion_effect = PotionEffect(effect_type, durationinticks, 1)