I have exported my google-maps Point Of Interests (saved places / locations) via the takeout tool. How can i convert this to GPX, so that i can import it into OSMAnd?
I tried using gpsbabel:
gpsbabel -i geojson -f my-saved-locations.json -o gpx -F my-saved-locations_converted.gpx
But this did not retain the title/name of each point of interest - and instead just used names like WPT001, WPT002, etc.
in the end I solved this by creating a small python script to convert between the formats.
This could be easily adapted for specific needs:
#!/usr/bin/env python3
import argparse
import json
import xml.etree.ElementTree as ET
from xml.dom import minidom
def ingestJson(geoJsonFilepath):
poiList = []
with open(geoJsonFilepath) as fileObj:
data = json.load(fileObj)
for f in data["features"]:
poiList.append({'title': f["properties"]["Title"],
'lon': f["geometry"]["coordinates"][0],
'lat': f["geometry"]["coordinates"][1],
'link': f["properties"].get("Google Maps URL", ''),
'address': f["properties"]["Location"].get("Address", '')})
return poiList
def dumpGpx(gpxFilePath, poiList):
gpx = ET.Element("gpx", version="1.1", creator="", xmlns="http://www.topografix.com/GPX/1/1")
for poi in poiList:
wpt = ET.SubElement(gpx, "wpt", lat=str(poi["lat"]), lon=str(poi["lon"]))
ET.SubElement(wpt, "name").text = poi["title"]
ET.SubElement(wpt, "desc").text = poi["address"]
ET.SubElement(wpt, "link").text = poi["link"]
xmlstr = minidom.parseString(ET.tostring(gpx)).toprettyxml(encoding="utf-8", indent=" ")
with open(gpxFilePath, "wb") as f:
f.write(xmlstr)
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--inputGeoJsonFilepath', required=True)
parser.add_argument('--outputGpxFilepath', required=True)
args = parser.parse_args()
poiList = ingestJson(args.inputGeoJsonFilepath)
dumpGpx(args.outputGpxFilepath, poiList=poiList)
if __name__ == "__main__":
main()
...
it can be called like so:
./convert-googlemaps-geojson-to-gpx.py \
--inputGeoJsonFilepath my-saved-locations.json \
--outputGpxFilepath my-saved-locations_converted.gpx
There is also a NPM script called "togpx":
https://github.com/tyrasd/togpx
I didn't try it, but it claims to keep as much information as possible.
Related
I am using confluent-kafka and I need to serialize my keys as strings and produce some messages. I have a working code for the case where I retrieve the schema from the schema registry and use it to produce a message. The problem is that it fails when I am trying to read the schema from a local file instead.
The code below is the working one for the schema registry:
import argparse
from confluent_kafka.schema_registry.avro import AvroSerializer
from confluent_kafka.serialization import StringSerializer
from confluent_kafka.schema_registry import SchemaRegistryClient
from confluent_kafka import SerializingProducer
import avro.schema
SCHEMA_HOST = '192.168.40.10'
TOPIC = 'my_topic'
SCHEMA = 'path/to/schema.avsc'
# Just parse argumments
parser = argparse.ArgumentParser(description="Avro Kafka Generator")
parser.add_argument('--schema_registry_host', default=SCHEMA_HOST, help="schema registry host")
parser.add_argument('--schema', type=str, default=SCHEMA, help="schema to produce under")
parser.add_argument('--topic', type=str, default=TOPIC, help="topic to publish to")
parser.add_argument('--frequency', type=float, default=1.0, help="number of message per second")
args = parser.parse_args()
# Actual code
schema_registry_conf = {'url': "http://{}:8081".format(SCHEMA_HOST)}
schema_registry_client = SchemaRegistryClient(schema_registry_conf)
schema = schema_registry_client.get_latest_version(subject_name=TOPIC + "-value")
# schema = schema_registry_client.get_schema(schema.schema_id)
schema = schema_registry_client.get_schema(schema.schema_id)
schema_str = schema.schema_str
pro_conf = {"auto.register.schemas": False}
avro_serializer = AvroSerializer(schema_registry_client=schema_registry_client, schema_str=schema_str, conf=pro_conf)
conf = {'bootstrap.servers': "{}:9095".format(args.schema_registry_host),
'schema.registry.url': "http://{}:8081".format(args.schema_registry_host)}
# avro_producer = AvroProducer(conf, default_value_schema=value_schema)
producer_conf = {'bootstrap.servers': "{}:9095".format(SCHEMA_HOST),
'key.serializer': StringSerializer('utf_8'),
'value.serializer': avro_serializer}
avro_producer = SerializingProducer(producer_conf)
But when I try to use a variation for the local file it fails:
# Read schema from local file
value_schema = avro.schema.Parse(open(args.schema, "r").read())
schema_str = open(args.schema, "r").read().replace(' ', '').replace('\n', '')
pro_conf = {"auto.register.schemas": True}
avro_serializer = AvroSerializer(schema_registry_client=schema_registry_client, schema_str=schema_str, conf=pro_conf)
this part is common to both versions:
producer_conf = {'bootstrap.servers': "{}:9095".format(SCHEMA_HOST),
'key.serializer': StringSerializer('utf_8'),
'value.serializer': avro_serializer}
avro_producer = SerializingProducer(producer_conf)
avro_producer.produce(topic=args.topic, value=message)
The error I am getting is the following;
KafkaError{code=_VALUE_SERIALIZATION,val=-161,str="'RecordSchema'
object has no attribute 'lookup_schema'"}
Obviously, it's not the best approach and I guess if it worked the code seems ugly and error prone. But it doesn't even work so, I need some help on how I could read a local schema and use the AvroSerializer afterwards.
I have a csv file where i have datas like
reviews pos_neg
men hecne bawa duwmedim bu kitabdan. yeqin wie sunni ayrimi etmisiz men wieyem dede babadan bildiyim dualar bu kitabda bawqadi Negative
Cavidanquluzade_official instagram seyfem xeber pırogramı Negative
səhər və axşam zikrlərində 108 ci zikrin başlığının latınca yazılışı verilmiyib. Positive
Bir müslümana aid herşey var Positive
now i need to change all "positive" to "1" and all negatives to "0". Could you please help me?
If I would write that kind of utility, I'd do something like this:
import argparse
import csv
import sys
parser = argparse.ArgumentParser()
parser.add_argument('file', type=argparse.FileType())
args = parser.parse_args()
out = csv.DictWriter(sys.stdout, ('review', 'pos_neg'))
out.writeheader()
for row in csv.DictReader(args.file):
row['pos_neg'] = ['Negative', 'Positive'].index(row['pos_neg'])
out.writerow(row)
Can be used like this on the command line:
$ python converter.py the_file.csv > the_new_file.csv
You may need to fiddle with the options for the CSV reader, as it appears you have a TSV or fix-width file perhaps…?
I have a script that takes some data in an excel file and sorts it and adds some colors to some key things....
I'm using an external .ini file because it needs to change sometimes based on the users needs for the day
the ini file basically looks like this
[section]
#Color 1
color01 = ('00FCC84E')
cat1 = ('Item1','Item2')
#Color 2
color02 = ('00F4426E')
cat2 = ('Thingy Size 5/16')
My Script portion that uses Config Parser does this
import configparser
from configparser import ConfigParser
from ast import literal_eval
config = ConfigParser()
config.read("MyFile.ini")
config.sections()
def variables(section):
dict1 = {}
options = config.options(section)
for option in options:
try:
dict1[option] = config.get(section, option)
if dict1[option] == -1:
DebugPrint("skip: %s" % option)
except:
print("exception on %s!" % option)
dict1[option] = None
return dict1
color01V = literal_eval(config['ConfigFile']['color01'])
color02V = literal_eval(config['ConfigFile']['color02'])
cat01V = literal_eval(config['ConfigFile']['cat1'])
cat02V = literal_eval(config['ConfigFile']['cat2'])
print(cat01V)
print(cat02V)
this returns
('Item1','Item2')
Thingy Size 5/16
Why does the 2nd string return without the () and how do I fix it.
I actually NEED the () to appear when I use the variable later
Have you tried putting value of "cat1" in quotations?
[section]
color01 = ('00FCC84E')
cat1 = "('Item1','Item2')"
...
cat2 = "('Thingy Size 5/16')"
ConfigParser also reads comments. Why? Shouldn't this be a default thing to "ignore" inline comments?
I reproduce my problem with the following script:
import configparser
config = configparser.ConfigParser()
config.read("C:\\_SVN\\BMO\\Source\\Server\\PythonExecutor\\Resources\\visionapplication.ini")
for section in config.sections():
for item in config.items(section):
print("{}={}".format(section, item))
The ini file looks as follows:
[LPI]
reference_size_mm_width = 30 ;mm
reference_size_mm_height = 40 ;mm
print_pixel_pitch_mm = 0.03525 ; mm
eye_cascade = "TBD\haarcascade_eye.xml" #
The output:
C:\_Temp>python read.py
LPI=('reference_size_mm_width', '30 ;mm')
LPI=('reference_size_mm_height', '40 ;mm')
LPI=('print_pixel_pitch_mm', '0.03525 ; mm')
LPI=('eye_cascade', '"TBD\\haarcascade_eye.xml" #')
I don't want to read 30 ;mm but I want to read just the number '30'.
What am I doing wrong?
PS: Python3.7
hi use inline_comment_prefixes while creating configparser object check example below
config = configparser.ConfigParser(inline_comment_prefixes = (";",))
Here is detailed documentation.
I have just started with lxml basics and I am stuck with namespaces: I need to generate an xml like this:
<CityModel
xmlns:bldg="http://www.opengis.net/citygml/building/2.0"
<cityObjectMember>
<bldg:Building>
<bldg:function>1000</bldg:function>
</bldg:Building>
</cityObjectMember>
</CityModel>
By using the following code:
from lxml import etree
cityModel = etree.Element("cityModel")
cityObject = etree.SubElement(cityModel, "cityObjectMember")
bldg = etree.SubElement(cityObject, "{http://schemas.opengis.net/citygml/building/2.0/building.xsd}bldg")
function = etree.SubElement(bldg, "{bldg:}function")
function.text = "1000"
print etree.tostring(cityModel, pretty_print=True)
I get this:
<cityModel>
<cityObjectMember>
<ns0:bldg xmlns:ns0="http://schemas.opengis.net/citygml/building/2.0/building.xsd">
<ns1:function xmlns:ns1="bldg:">1000</ns1:function>
</ns0:bldg>
</cityObjectMember>
</cityModel>
which is quite different from what I want, and my software doesn't parse it.
How to get the correct xml?
from lxml import etree
ns_bldg = "http://www.opengis.net/citygml/building/2.0"
nsmap = {
'bldg': ns_bldg,
}
cityModel = etree.Element("cityModel", nsmap=nsmap)
cityObject = etree.SubElement(cityModel, "cityObjectMember")
bldg = etree.SubElement(cityObject, "{%s}Building" % ns_bldg)
function = etree.SubElement(bldg, "{%s}function" % ns_bldg)
function.text = "1000"
print etree.tostring(cityModel, pretty_print=True)
prints
<cityModel xmlns:bldg="http://www.opengis.net/citygml/building/2.0">
<cityObjectMember>
<bldg:Building>
<bldg:function>1000</bldg:function>
</bldg:Building>
</cityObjectMember>
</cityModel>
See lxml.etree Tutorial - Namespaces.