We Keep Coding

How to send image in C# to a Flask Server that is decoded by OpenCV?

Here is part of my Flask API in Python:
image_data = flask.request.get_data() # image_data's data type
string image_vector = numpy.frombuffer(image_data, dtype=numpy.uint8)
image = cv2.imdecode(image_vector, cv2.IMREAD_COLOR)
How would I send a image that I encoded like below, in C#:
ResultString = "Loading...";
var surface = SKSurface.Create(new SKImageInfo((int)canvasView.CanvasSize.Width,
(int)canvasView.CanvasSize.Height));
var canvas = surface.Canvas;
canvas.Clear();
foreach (SKPath path in completedPaths)
canvas.DrawPath(path, paint);
foreach (SKPath path in inProgressPaths.Values)
canvas.DrawPath(path, paint);
canvas.Flush();
var snap = surface.Snapshot();
var pngImage = snap.Encode(SKEncodedImageFormat.Png, 100);
AnalyerResults analyerResults = mathclient.AnalyzeWork(pngImage);
try { ResultString = analyerResults.message; } catch { ResultString = "Error..."; }
How would I send the image to in C# to be able to be received and decoded like shown in part of my API?
I already tried:
HttpClient client = await GetClient();
var result = await client.PostAsync(Url + "analyzer", new ByteArrayContent(pngImage.ToArray()));
return JsonConvert.DeserializeObject<AnalyerResults>(await result.Content.ReadAsStringAsync());
I also tried:
var client = new RestClient(Url + "analyzer");
client.Timeout = -1;
var request = new RestRequest(Method.POST);
request.AddHeader("Content-Type", "image/png");
request.AddParameter("image/png", pngImage, ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
return JsonConvert.DeserializeObject<AnalyerResults>(response.Content);
However in both the content returned null. This question is related to How to Replicate this Postman Request which has a Binary Content Body and contains a .PNG File in C#?.

Node & python don't return the same hash256

My NodeJS & Python scripts don't return the same hash, what could cause this issue?
Node.js
const { createHmac } = require("crypto");
var message = 'v1:1583197109:'
var key = 'Asjei8578FHasdjF85Hfjkasi875AsjdiAas_CwueKL='
const digest = Buffer.from(key, "base64");
const hash = createHmac("sha256", digest)
.update(message)
.digest("hex");
console.log(hash)
> 7655b4f816dc7725fb4507a20f2b97823979ea00b121c84b76924fea167dcaf7
Python3
message = 'v1:1583197109:'
key = 'Asjei8578FHasdjF85Hfjkasi875AsjdiAas_CwueKL=' + '=' #add a "=" to avoid incorrect padding
digest = base64.b64decode(key.encode('utf-8'))
hash_ = hmac.new(digest, message.encode('utf-8'), hashlib.sha256)
hash_result = hash_.hexdigest()
print(hash_result)
> c762b612d7c56d3f9c95052181969b42c604c2d41b7ce5fc7f5a06457e312d5b
I guess it could be the extra = to avoid the incorrect padding but my key ends with a single =.

Node.js Buffer.from(..., 'base64') can consume the input in the "urlsafe" base64 (https://nodejs.org/api/buffer.html#buffer_buffers_and_character_encodings), and _ is not a valid Base64 character for python, while it is for node.
Adding altchars that correspond to the "urlsafe" version of Base64 to python code yields equal hashes.
const { createHmac } = require("crypto");
var message = 'v1:1583197109:'
var key = 'Asjei8578FHasdjF85Hfjkasi875AsjdiAas_CwueKL='
const digest = Buffer.from(key, "base64");
const hash = createHmac("sha256", digest)
.update(message)
.digest("hex");
console.log(hash) // 7655b4f816dc7725fb4507a20f2b97823979ea00b121c84b76924fea167dcaf7
message = 'v1:1583197109:'
key = 'Asjei8578FHasdjF85Hfjkasi875AsjdiAas_CwueKL=' + '=' #add a "=" to avoid incorrect padding
digest = base64.b64decode(key.encode('utf-8'), altchars='-_')
hash_ = hmac.new(digest, message.encode('utf-8'), hashlib.sha256)
hash_result = hash_.hexdigest()
print(hash_result) # 7655b4f816dc7725fb4507a20f2b97823979ea00b121c84b76924fea167dcaf7
Also, python's b64decode has validate kwarg, which would check the input string and "fail loud" instead of ignoring incorrect characters

Conversion of Text sentences to CONLL Format

I want to convert the Normal english text into CONLL-U format for maltparser for finding dependency in the text in Python. I tried in java but was failed to do so, below is the format I'm looking for-
String[] tokens = new String[11];
tokens[0] = "1\thappiness\t_\tN\tNN\tDD|SS\t2\tSS";
tokens[1] = "2\tis\t_\tV\tVV\tPS|SM\t0\tROOT";
tokens[2] = "3\tthe\t_\tAB\tAB\tKS\t2\t+A";
tokens[3] = "4\tkey\t_\tPR\tPR\t_\t2\tAA";
tokens[4] = "5\tof\t_\tN\tEN\t_\t7\tDT";
tokens[5] = "6\tsuccess\t_\tP\tTP\tPA\t7\tAT";
tokens[6] = "7\tin\t_\tN\tNN\t_\t4\tPA";
tokens[7] = "8\tthis\t_\tPR\tPR\t_\t7\tET";
tokens[8] = "9\tlife\t_\tR\tRO\t_\t10\tDT";
tokens[9] = "10\tfor\t_\tN\tNN\t_\t8\tPA";
tokens[10] = "11\tsure\t_\tP\tIP\t_\t2\tIP";
I have tried in java but I can not use the standford APIs, I want the same in python.
//This is the example of java code but here the tokens which is created needs to be parsed via code not manually-
MaltParserService service = new MaltParserService(true);
// in the CoNLL data format.
String[] tokens = new String[11];
tokens[0] = "1\thappiness\t_\tN\tNN\tDD|SS\t2\tSS";
tokens[1] = "2\tis\t_\tV\tVV\tPS|SM\t0\tROOT";
tokens[2] = "3\tthe\t_\tAB\tAB\tKS\t2\t+A";
tokens[3] = "4\tkey\t_\tPR\tPR\t_\t2\tAA";
tokens[4] = "5\tof\t_\tN\tEN\t_\t7\tDT";
tokens[5] = "6\tsuccess\t_\tP\tTP\tPA\t7\tAT";
tokens[6] = "7\tin\t_\tN\tNN\t_\t4\tPA";
tokens[7] = "8\tthis\t_\tPR\tPR\t_\t7\tET";
tokens[8] = "9\tlife\t_\tR\tRO\t_\t10\tDT";
tokens[9] = "10\tfor\t_\tN\tNN\t_\t8\tPA";
tokens[10] = "11\tsure\t_\tP\tIP\t_\t2\tIP";
// Print out the string array
for (int i = 0; i < tokens.length; i++) {
System.out.println(tokens[i]);
}
// Reads the data format specification file
DataFormatSpecification dataFormatSpecification = service.readDataFormatSpecification(args[0]);
// Use the data format specification file to build a dependency structure based on the string array
DependencyStructure graph = service.toDependencyStructure(tokens, dataFormatSpecification);
// Print the dependency structure
System.out.println(graph);

There is now a port of the Standford library to Python (with improvements) called Stanza. You can find it here: https://stanfordnlp.github.io/stanza/
Example of usage:
>>> import stanza
>>> stanza.download('en') # download English model
>>> nlp = stanza.Pipeline('en') # initialize English neural pipeline
>>> doc = nlp("Barack Obama was born in Hawaii.") # run annotation over a sentence

Write gpxlogger data to the same file

I have some very basic gpxlogger code that writes all the data to file quite well. (below)
gpxlogger -d -f /home/pi/Desktop/EPQ/temp_gps/gpslog
However I would like this code to always write to the same file without overwriting it. So if possible when it started logging it would go to the bottom of the file and start logging the data there, below what has already been logged.
Thanks,
Dan.
Javascript to read xml file
<script src="http://maps.google.com/maps?file=api&v=2&key=ABQIAAAA7_kD1t_m22HBF9feCaDPZxQwcATY4FXmxYwkk9LNWGtAQdNKTBS1kBsTEqrRPg2kWxuNdmf2JVCIkQ" type="text/javascript"></script>
<script src="http://gmaps-utility-library.googlecode.com/svn/trunk/markermanager/release/src/markermanager.js"> </script><script>
var map;
function initialize () {
if (GBrowserIsCompatible()) {
map = new GMap2(document.getElementById("map_canvas"));
map.setCenter(new GLatLng(53.423027, -1.523462), 10);
map.addControl(new GLargeMapControl());
map.addControl(new GMapTypeControl());
map.addMapType(G_PHYSICAL_MAP);
map.setMapType(G_PHYSICAL_MAP);
addMarkersFromXML();
}
}
function addMarkersFromXML(){
var batch = [];
mgr = new MarkerManager(map);
var request = GXmlHttp.create();
request.open('GET', 'gpslog.xml', true);
request.onreadystatechange = function() {
if (request.readyState == 4 && request.status == 200) {
var xmlDoc = request.responseXML;
var xmlrows = xmlDoc.documentElement.getElementsByTagName("trkpt");
for (var i = 0; i < xmlrows.length; i++) {
var xmlrow = xmlrows[i];
var xmlcellLatitude = parseFloat(xmlrows[i].getAttribute("lat"));
var xmlcellLongitude = parseFloat(xmlrows[i].getAttribute("lon"));
var point = new GLatLng(xmlcellLatitude,xmlcellLongitude);
//get the time of the pin plot
var xmlcellplottime = xmlrow.getElementsByTagName("time")[0];
var celltextplottime = xmlcellplottime.firstChild.data;
//get the elevation of the pin plot
var xmlcellplotelevation = xmlrow.getElementsByTagName("ele")[0];
var celltextplotelevation = xmlcellplotelevation.firstChild.data;
//get the number of satellites at the time of the pin plot
var xmlcellplotsat = xmlrow.getElementsByTagName("sat")[0];
var celltextplotsat = xmlcellplotsat.firstChild.data;
var htmlString = "Time: " + celltextplottime + "<br>" + "Elevation: " + celltextplotelevation + "<br>" + "Satellites: " + celltextplotsat;
//var htmlString = 'yes'
var marker = createMarker(point,htmlString);
batch.push(marker);
}
mgr.addMarkers(batch,10);
mgr.refresh();
}
}
request.send(null);
}
function createMarker(point,html) {
var marker = new GMarker(point);
GEvent.addListener(marker, "click", function() {
marker.openInfoWindowHtml(html);
});
return marker;
}
</script>
</head>
<body onload="initialize()" onunload="GUnload()">
<div id="map_canvas" style="width: 1350px; height: 800px"></div>
<div id="message"></div>
</body>
</html>

Here's another option.
Look at gps3.py, put it, and the following script into a directory.
It reads data from the gpsd; creates the gpx log file if it doesn't exist; appends "trackpoint" data to it when data exists; while maintaining the same file and appending "trackpoint" data after a restart.
Place both in the same directory and then have you javascript read the file..or put the entire structure in the same script.
#!/usr/bin/env python
# coding=utf-8
""" gpx logger to create and append a gpx formatted log of gpsd data """
import os
import time
import gps3
from datetime import datetime
the_connection = gps3.GPSDSocket()
the_fix = gps3.Fix()
the_log = '/tmp/gpx3.gpx'
creation = datetime.utcnow()
fmt = '%Y-%m-%d %H:%M:%S %Z%z'
genesis = creation.strftime(fmt)
if not os.path.isfile(the_log):
header = ('<?xml version = "1.0" encoding = "utf-8"?>\n'
'<gpx version = "1.1" '
'creator = "GPSD 3.9 - http://catb.org/gpsd" '
'client = "gps3.py - http://github.com/wadda/gps3"'
'xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance"'
'xmlns = "http://www.topografix.com/GPX/1/1"'
'xsi:schemaLocation = "http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd">\n '
'<metadata>\n '
' <time>{}\n'
'</metadata>\n').format(genesis)
f = open(the_log, 'w')
f.write(header)
f.close()
try:
for new_data in the_connection:
if new_data:
the_fix.refresh(new_data)
if not isinstance(the_fix.TPV['lat'], str): # lat determinate of when data is 'valid'
latitude = the_fix.TPV['lat']
longitude = the_fix.TPV['lon']
altitude = the_fix.TPV['alt']
time = the_fix.TPV['time']
mode = the_fix.TPV['mode']
tag = the_fix.TPV['tag']
sats = the_fix.satellites_used()
hdop = the_fix.SKY['hdop']
vdop = the_fix.SKY['vdop']
pdop = the_fix.SKY['pdop']
trackpoint = ('<trkpt lat = {} lon = {}>\n'
' <ele>{}</ele>\n'
' <time>{}</time>\n'
' <src>GPSD tag ="{}"</src>\n'
' <fix>{}</fix >\n'
' <sat>{}</sat>\n'
' <hdop>{}</hdop>\n'
' <vdop>{}</vdop>\n'
' <pdop>{}</pdop>\n'
'</trkpt>\n').format(latitude, longitude, altitude, time, tag, mode, sats[1], hdop, vdop, pdop)
addendum = open(the_log, 'a')
addendum.write(trackpoint)
addendum.close()
except Exception as error:
print('Danger-Danger',error)

You run into the problem of a daemonised gpxlogger requiring a -f flag for a file name and that will overwrite the file. This you know.
I see there are two options. Not run gpxlogger as a daemon
gpxlogger >> /home/pi/Desktop/EPQ/temp_gps/gpslog
or run it as a daemon and cat the file to an append-able file
gpxlogger -d -f /home/pi/Desktop/EPQ/temp_gps/gpslog & cat /home/pi/Desktop/EPQ/temp_gps/gpslog >> /home/pi/Desktop/EPQ/temp_gps/gpslog_concatenated

Another way to look at it would be to create sequential logs, and then concatenate them with gpsbable, but in order to do that you need to have a script, and an index.
Make the index echo 0 > ~/.gpxfilecount
Open a favourite editor and create a file including something like:
#! /usr/bin/bash
COUNT=`cat ~/.gpxfilecount`
echo $(($COUNT + 1 )) > ~/.gpxfilecount
filename="gpxlogfile${COUNT}.gpx"
exec gpxlogger -d -f $filename
Mark the script executable chmod +x ~/bin/gpxer.sh (or favourite name).
Every time you fire up the gpxlogger an incremented filename is created. Those file can then be concatenated without tears by gpsbable gpsbabel -i geo -f gpxlogfile1.gpx -f gpxlogfile2.gpx -f gpxlogfile3.gpx -o gpx -F biglogcat.gpx ...or however gpsbable works.

I was curious what building a gpx file from scratch would look like using only minidom. Unfortunately life intervened, sorry for the delay...
When this script (below) gpex3.py, still a little crude and inefficient (read/write every second), is placed in the same directory as gps3.py it creates an appendable gpx file at /tmp/gpx3.gpx
#! /usr/bin/python3
# coding=utf-8
"""banana"""
import xml.dom.minidom
import gps3
import time
from datetime import datetime, timezone, timedelta
import os
import sys
gps_connection = gps3.GPSDSocket()
gps_fix = gps3.Fix()
the_log = '/tmp/gpx3.gpx'
def start_time():
"""time in the beginning"""
timestart = str(datetime.utcnow().replace(tzinfo=(timezone(timedelta(0)))))
return timestart
def close(doc):
"""write file to disk and close"""
log_write = open(the_log, "w")
doc.writexml(log_write)
log_write.close()
if os.path.isfile(the_log):
doc = xml.dom.minidom.parse(the_log) # opens the pre-existing
gpx_element = doc.firstChild
else:
doc = xml.dom.minidom.Document()
gpx_element = doc.createElement("gpx")
doc.appendChild(gpx_element)
trk_element = doc.createElement("trkseg")
trk_element.setAttribute("began", start_time())
gpx_element.appendChild(trk_element)
utc = alt = hdop = vdop = pdop = mode = sats = tag = 'n/a'
try:
tpv_list = {'time': utc, 'ele': alt, 'tag': tag}
sky_list = {'hdop': hdop, 'vdop': vdop, 'pdop': pdop}
# misc_list = {'sat': sats, 'fix':mode} # just an account
element = {}
x = 1 # for the 'is it working?'
for new_data in gps_connection:
if new_data:
gps_fix.refresh(new_data)
if not isinstance(gps_fix.TPV['lat'], str):
trkpt_element = doc.createElement("trkpt")
trk_element.appendChild(trkpt_element)
trkpt_element.setAttribute('lat', str(gps_fix.TPV['lat']))
trkpt_element.setAttribute('lon', str(gps_fix.TPV['lon']))
# tpv_list[key]
for key in tpv_list:
if key == 'ele':
element[key] = '{}'.format(gps_fix.TPV['alt']) # because consistency with labels is a horrible.
else:
element[key] = '{}'.format(gps_fix.TPV[key])
# sky_list[key]
for key in sky_list:
element[key] = '{}'.format(gps_fix.SKY[key])
# Misc.
element['sat'] = '{}'.format(gps_fix.satellites_used()[1])
element['fix'] = '{}'.format(("ZERO", "NO_FIX", "2D", "3D")[gps_fix.TPV['mode']])
for key in element:
trkpt_data = doc.createElement(key)
trkpt_element.appendChild(trkpt_data)
new_value = doc.createTextNode(element[key])
trkpt_data.appendChild(new_value)
# print(doc.toprettyxml())
close(doc) # write to file with every trackpoint
print('Cycle', x) # Only an "is it working?"
x += 1
time.sleep(1)
except KeyboardInterrupt:
gps_connection.close()
print("\nTerminated by user\nGood Bye.\n")
if __name__ == '__main__':
pass

Retrieve User Entry IDs from MAPI

I extended the win32comext MAPI with the Interface IExchangeModifyTable to edit ACLs via the MAPI. I can modify existing ACL entries, but I stuck in adding new entries. I need the users entry ID to add it, according this C example
(Example Source from MSDN)
STDMETHODIMP AddUserPermission(
LPSTR szUserAlias,
LPMAPISESSION lpSession,
LPEXCHANGEMODIFYTABLE lpExchModTbl,
ACLRIGHTS frights)
{
HRESULT hr = S_OK;
LPADRBOOK lpAdrBook;
ULONG cbEid;
LPENTRYID lpEid = NULL;
SPropValue prop[2] = {0};
ROWLIST rowList = {0};
char szExName[MAX_PATH];
// Replace with "/o=OrganizationName/ou=SiteName/cn=Recipients/cn="
char* szServerDN = "/o=org/ou=site/cn=Recipients/cn=";
strcpy(szExName, szServerDN);
strcat(szExName, szUserAlias);
// Open the address book.
hr = lpSession->OpenAddressBook(0,
0,
MAPI_ACCESS_MODIFY,
&lpAdrBook );
if ( FAILED( hr ) ) goto cleanup;
// Obtain the entry ID for the recipient.
hr = HrCreateDirEntryIdEx(lpAdrBook,
szExName,
&cbEid,
&lpEid);
if ( FAILED( hr ) ) goto cleanup;
prop[0].ulPropTag = PR_MEMBER_ENTRYID;
prop[0].Value.bin.cb = cbEid;
prop[0].Value.bin.lpb = (BYTE*)lpEid;
prop[1].ulPropTag = PR_MEMBER_RIGHTS;
prop[1].Value.l = frights;
rowList.cEntries = 1;
rowList.aEntries->ulRowFlags = ROW_ADD;
rowList.aEntries->cValues = 2;
rowList.aEntries->rgPropVals = &prop[0];
hr = lpExchModTbl->ModifyTable(0, &rowList);
if(FAILED(hr)) goto cleanup;
printf("Added user permission. \n");
cleanup:
if (lpAdrBook)
lpAdrBook->Release();
return hr;
}
I can open the Address Book, but HrCreateDirEntryIdEx is not provided in the pywin32 mapi. I found it in the exchange extension, which does not compile on my system, the missing library problem. Do you have any idea to retrieve the users entry ID?
Thank.
Patrick

I got this piece of code and it works fine
from binascii import b2a_hex, a2b_hex
import active_directory as ad
# entry_type, see http://msdn.microsoft.com/en-us/library/cc840018.aspx
# + AB_DT_CONTAINER 0x000000100
# + AB_DT_TEMPLATE 0x000000101
# + AB_DT_OOUSER 0x000000102
# + AB_DT_SEARCH 0x000000200
# ab_flags, maybe see here: https://svn.openchange.org/openchange/trunk/libmapi/mapidefs.h
def gen_exchange_entry_id(user_id, ab_flags=0, entry_type = 0):
muidEMSAB = "DCA740C8C042101AB4B908002B2FE182"
version = 1
# Find user and bail out if it's not there
ad_obj = ad.find_user(user_id)
if not ad_obj:
return None
return "%08X%s%08X%08X%s00" % (
ab_flags,
muidEMSAB,
version,
entry_type,
b2a_hex(ad_obj.legacyExchangeDN.upper()).upper(),
)
data = gen_exchange_entry_id("myusername")
print data
print len(a2b_hex(data))