I am trying to webscrape the latitude and longitude from the Maps in the URL
The HTML Script looks below,
<div class="map-container"><script>$(document).ready(function() {
var $injector = angular.element(document.body).injector();
var $compile = $injector.get('$compile');
var $rootScope = $injector.get('$rootScope');
var sfMap = $compile('<div data-sf-map data-center="center" data-zoom="15" data-markers="markers" data-scrollwheel="false" data-zoom-level-change="14"></div>');
var scope = $rootScope.$new();
scope.center = {
latitude: 40.7515022729943,
longitude: -74.0071970200125
};
scope.markers = [{
id: 4673,
latitude: 40.7515022729943,
longitude: -74.0071970200125
}];
var $el = sfMap(scope);
$('.listing-map .map-wrapper .map-container').append($el);
});
I tried to use the option of Copy as XPath but it is not working for
this case. How to get the latitude and longitude?
You can try below code to get latitude and longitude values:
import re
script = driver.find_element_by_xpath('//div[#class="map-container"]/script').get_attribute('textContent')
latitude = re.search('(?<=latitude: )-*\d+.\d+', script).group(0)
longitude = re.search('(?<=longitude: )-*\d+.\d+', script).group(0)
Related
I want to export a chart created in gee to a png format, but I don't know if that's possible. This is my code, I wanna export an NDVI over time chart.
var startDate = '2022-01-01'
var endDate = '2022-12-04'
var roi= ee.Geometry.Point([-50,-30])
var l8 = ee.ImageCollection('LANDSAT/LC08/C02/T1_L2')
.filterBounds(roi)
.filterDate(startDate, endDate)
// Map a function over the Landsat 8 TOA collection to add an NDVI band.
var withNDVI = l8.map(function(image) {
var ndvi = image.normalizedDifference(['SR_B5', 'SR_B4']).rename('NDVI');
return image.addBands(ndvi);
});
// Create a chart.
var chart = ui.Chart.image.series({
imageCollection: withNDVI.select('NDVI'),
region: roi,
reducer: ee.Reducer.first(),
scale: 30
}).setOptions({title: 'NDVI over time'});
I wanted to get the latitude and longitude of an adress with geopy. My code is
geolocator = Nominatim(user_agent="EXAMPLE")
location = geolocator.geocode("Wismarsche Straße 393-397 19049 Schwerin")
lat = str(location.latitude)
lon = str(location.longitude)
latLon = str(lat) + ";" + str(lon) + ";"
and it outputs NoneType.
Does anyone know why?
The documentation states that the geocode method returns None when no results are found. In your case, removing 19049 from your query returns a result:
location = geolocator.geocode("Wismarsche Straße 393-397 Schwerin")
lat = str(location.latitude)
lon = str(location.longitude)
latLon = str(lat) + ";" + str(lon) + ";"
print(f'Latitude: {lat}, longitude: {lon}, latLon: {latLon}')
Ouput:
Latitude: 53.6519479, longitude: 11.4073671, latLon: 53.6519479;11.4073671;
I want to get nearby cities from passed latitude and longitude. I have used the geonames and geobytes APIs but want to use Google API for finding nearby cities.
This is my code:
def getNearbyCities(self):
# lat, lon = self.getLatLon()
# res_url = urlopen('http://gd.geobytes.com/GetNearbyCities?callback=?&radius=100&limit=100&Latitude=' + str(lat) + '&Longitude=' + str(lon))
res_url = urlopen('http://getnearbycities.geobytes.com/GetNearbyCities?callback=?&radius=100&locationcode=' + str(self.city))
resp = str(res_url.read())
print(resp)
validate_res = resp.split("b'?(")[-1].split(");'")[0]
validated_res = ast.literal_eval(validate_res)
cities_nd_distence = []
for data in validated_res:
data_tuple = (data[1], data[7])
if data[1] not in cities_nd_distence:
cities_nd_distence.append(data_tuple)
import pprint
pprint.pprint(cities_nd_distence)
return cities_nd_distence
If you only want to get cities based on latitude and longitude, you can have a look at https://github.com/coderholic/django-cities
from cities.models import City
from django.contrib.gis.geos import Point
from django.contrib.gis.db.models.functions import Distance
p = Point(-118, 34, srid=4326)
City.objects.annotate(distance=Distance('location', p)).order_by("distance").first()
<City: Hacienda Heights>
I am trying to use a slider to update my Bokeh Plot. I am finding it difficult to achieve it using pandas dataframe(did not find any examples so far).
The other way is to use the "columndatasource" (found some examples over forums) but still not able to achieve the functionality.
So I have two columns, X axis is date and the Y axis is Volume. I want to change my Y values based on slider input. I am able to see the plot but the slider functionality is not working
Any help will be very much appreciable.
source = ColumnDataSource(data=dict(x=df2['Date'],y=df2['Vol']))
S1 = figure(plot_width=400,plot_height=400,tools=TOOLS1,title="Volume Per Day",x_axis_type="datetime")
S1.line('x','y',source=source)
callback_test = CustomJS(args=dict(source=source), code="""
var data = source.get('data');
var s_val = cb_obj.value
x = data['x']
y = data['y']
console.log(cb_obj)
for (i = 0; i < s_val; i++) {
y[i] = y[i]
}
source.trigger('change');
""")
slider = Slider(start=0, end= max_Vol, value=1, step=100,title="Vol Per Day",callback=callback_test)
You are trying to update the range of data that is plotted using a slider.
When you do:
y = data['y']
for (i = 0; i < s_val; i++) {
y[i] = y[i]
}
the python equivalent would be, if y is some array with length>s_val:
for i in range(s_val):
y[i] = y[i]
This just replaces the elements from 0 to s_val-1 by themselves and doesn't change the rest of the list.
You can do two things:
update the displayed axis range directly
use an empty source that you will fill from your existing source based on the slider value
.
source = ColumnDataSource(data=dict(x=df2['Date'],y=df2['Vol']))
fill_source = ColumnDataSource(data=dict(x=[],y=[]))
S1 = figure(plot_width=400,plot_height=400,tools=TOOLS1,title="Volume Per Day",x_axis_type="datetime")
S1.line('x','y',source=fill_source)
callback_test = CustomJS(args=dict(source=source,fill_source=fill_source), code="""
var data = source.data;
var fill_data = fill_source.data;
var s_val = cb_obj.value;
fill_data['x']=[];
fill_data['y']=[];
for (i = 0; i < s_val; i++) {
fill_data['y'][i].push(data['y'][i]);
fill_data['x'][i].push(data['x'][i]);
}
fill_source.trigger('change');
""")
Here is the changes I have made to make it work with Bokeh last version
Some syntax error in the JavaScript part have been corrected, the method to trigger change is now change.emit, and the callback for a stand alone document is set after the Slider definition thanks to js_on_change
I have added all the import commands to give a complete example
I have also changed the visualization to show only data below the number of flight set by the slider (for more comprehension when moving the Slider towards lower values)
Below is the resulting code:
from bokeh.layouts import column, widgetbox
from bokeh.models import ColumnDataSource, CustomJS
from bokeh.models.widgets import Slider
from bokeh.plotting import Figure
import pandas as pd
from datetime import datetime, date, timedelta
from bokeh.plotting import show
from random import randint
today = date.today()
random_data = [[today + timedelta(days = i), randint(0, 10000)] for i in range(10)]
df2 = pd.DataFrame(random_data, columns = ['Date', 'Vol'])
source = ColumnDataSource(data = dict(x = df2['Date'], y = df2['Vol']))
fill_source = ColumnDataSource(data = dict(x = df2['Date'], y = df2['Vol'])) # set the graph to show all data at loading
TOOLS1 = []
S1 = Figure(plot_width = 400, plot_height = 400, tools = TOOLS1, title = "Volume Per Day", x_axis_type = "datetime")
S1.line('x', 'y', source = fill_source)
callback_test = CustomJS(args = dict(source = source, fill_source = fill_source), code = """
var data = source.data;
var fill_data = fill_source.data;
var s_val = cb_obj.value;
fill_data['x']=[];
fill_data['y']=[];
for (var i = 0; i <= data.x.length; i++) { // added "var" declaration of variable "i"
if (data['y'][i] <= s_val) { // more meaningful visualization: assuming you want to focuss on dates with less number of flights
fill_data['y'].push(data['y'][i]); // [i] index on left side of assignment removed
}
else {
fill_data['y'].push(0);
}
fill_data['x'].push(data['x'][i]);
}
fill_source.change.emit() ; // "trigger" method replaced by "change.emit"
""")
max_Vol = df2['Vol'].max()
slider = Slider(start = 0, end = max_Vol, value = max_Vol, step = 100, title = "Vol Per Day") # Remove attribute "callback = callback_test"
slider.js_on_change('value', callback_test) # new way of defining event listener
controls = widgetbox(slider)
layout = column(controls, S1)
show(layout)
Would be nice if I could embbed the resulting (HTML) visualization directly in this answer, let me now if it's possible ;)
I want to get latitude and longitude from a webpage using beautifulsoup but they are in a script:
//<![CDATA[
theForm.oldSubmit = theForm.submit;
theForm.submit = WebForm_SaveScrollPositionSubmit;
theForm.oldOnSubmit = theForm.onsubmit;
theForm.onsubmit = WebForm_SaveScrollPositionOnSubmit;
var GMapsProperties={};function getGMapElementById(mapId,GMapElementId){var _mapId=typeof(mapId)=='string'? mapId : mapId.getDiv().id;var overlayArray=GMapsProperties[_mapId]['overlayArray'];for(var i=0;i < overlayArray.length;i++){if(overlayArray[i][0]==GMapElementId){return overlayArray[i][1];}}return null;}function removeGMapElementById(mapId,GMapElementId){var _mapId=typeof(mapId)=='string'? mapId : mapId.getDiv().id;var overlayArray=GMapsProperties[_mapId]['overlayArray'];for(var i=0;i < overlayArray.length;i++){if(overlayArray[i][0]==GMapElementId){overlayArray.splice(i,1);return;}}}function closeWindows(mapId){for(var i=0;i<GMapsProperties[mapId]['windowArray'].length;i++){GMapsProperties[mapId]['windowArray'][i][1].close();}}var _sg=_sg ||{};_sg.cs=(function(){var p={};p.createMarker=function(opt,id){var m=new google.maps.Marker(opt);if(id && m.getMap())GMapsProperties[m.getMap().getDiv().id]['overlayArray'].push([id,m]);return m;};p.createPolyline=function(opt,id){var m=new google.maps.Polyline(opt);if(id && m.getMap())GMapsProperties[m.getMap().getDiv().id]['overlayArray'].push([id,m]);return m;};p.createPolygon=function(opt,id){var m=new google.maps.Polygon(opt);if(id && m.getMap())GMapsProperties[m.getMap().getDiv().id]['overlayArray'].push([id,m]);return m;};return p;})();function addEvent(el,ev,fn){if(el.addEventListener)el.addEventListener(ev,fn,false);else if(el.attachEvent)el.attachEvent('on'+ev,fn);else el['on'+ev]=fn;}GMapsProperties['subgurim_GoogleMapControl'] = {}; var GMapsProperties_subgurim_GoogleMapControl = GMapsProperties['subgurim_GoogleMapControl']; GMapsProperties_subgurim_GoogleMapControl['enableStore'] = false; GMapsProperties_subgurim_GoogleMapControl['overlayArray'] = new Array(); GMapsProperties_subgurim_GoogleMapControl['windowArray'] = new Array();var subgurim_GoogleMapControl;function load_subgurim_GoogleMapControl(){var mapDOM = document.getElementById('subgurim_GoogleMapControl'); if (!mapDOM) return;subgurim_GoogleMapControl = new google.maps.Map(mapDOM);function subgurim_GoogleMapControlupdateValues(eventId,value){var item=document.getElementById('subgurim_GoogleMapControl_Event'+eventId);item.value=value;}google.maps.event.addListener(subgurim_GoogleMapControl, 'addoverlay', function(overlay) { if(overlay) { GMapsProperties['subgurim_GoogleMapControl']['overlayArray'].push(overlay); } });google.maps.event.addListener(subgurim_GoogleMapControl, 'clearoverlays', function() { GMapsProperties['subgurim_GoogleMapControl']['overlayArray'] = new Array(); });google.maps.event.addListener(subgurim_GoogleMapControl, 'removeoverlay', function(overlay) { removeGMapElementById('subgurim_GoogleMapControl',overlay.id) });google.maps.event.addListener(subgurim_GoogleMapControl, 'maptypeid_changed', function() { var tipo = subgurim_GoogleMapControl.getMapTypeId(); subgurim_GoogleMapControlupdateValues('0', tipo);});google.maps.event.addListener(subgurim_GoogleMapControl, 'dragend', function() { var lat = subgurim_GoogleMapControl.getCenter().lat(); var lng = subgurim_GoogleMapControl.getCenter().lng(); subgurim_GoogleMapControlupdateValues('2', lat+','+lng); });google.maps.event.addListener(subgurim_GoogleMapControl, 'zoom_changed', function() { subgurim_GoogleMapControlupdateValues('1', subgurim_GoogleMapControl.getZoom()); });subgurim_GoogleMapControl.setOptions({center:new google.maps.LatLng(35.6783546483511,51.4196634292603),disableDefaultUI:true,keyboardShortcuts:false,mapTypeControl:false,mapTypeId:google.maps.MapTypeId.ROADMAP,scrollwheel:false,zoom:14});var marker_subgurim_920435_=_sg.cs.createMarker({position:new google.maps.LatLng(35.6783546483511,51.4196634292603),clickable:true,draggable:false,map:subgurim_GoogleMapControl,raiseOnDrag:true,visible:true,icon:'/images/markers/Site/Tourism/vase.png'}, 'marker_subgurim_920435_');}addEvent(window,'load',load_subgurim_GoogleMapControl);//]]>
and I want information in this part:
{position:new google.maps.LatLng(35.6783546483511,51.4196634292603)
is it possible to access that information by using beautifulsoup or any other web-scraper?
Use Regular expression for this purpose.
import re
#Suppose script is stored in variable script_file
m = re.search('LatLng(\(.+?\))', script_file)
latlng = m.group(1)
latlng = eval(latlng)
print(latlng) #(35.6783546483511, 51.4196634292603)
import re
s = 'position:new google.maps.LatLng(35.6783546483511,51.4196634292603)'
lat, lng = map(float, re.search(r'\(([^,]+),([^)]+)', s).groups())
If you want to get Latitude and Longitude separately, use regex expression in this way:
import re
s = 'position:new google.maps.LatLng(35.6783546483511,51.4196634292603)'
Lat, Lng = map(float, re.search(r'LatLng\(([\d.]+),([\d.]+)\)',s).groups())