Accessing a memory-mapped file using Python - python

I am looking to take use of a memory mapped file from Guild Wars 2, which is designed to link into Mumble for positional audio. The file contains information on the characters coordinates and other useful information.
I have been able to access the coordinate information using this script,
import mmap
import struct
last=[]
while True:
shmem = mmap.mmap(0, 20, "MumbleLink", mmap.ACCESS_READ)
coord=struct.unpack("IL3f", shmem)[2:5]
shmem.close()
if last!=coord:
print(coord)
last = coord
X = coord[2]
Y = coord[0]
Z = coord[1])
My problem is I am having difficulty working out how to get more information out of the file. How would I go about accessing other information that is stored, such as character name and camera position.
There is information on the file here:
https://forum-en.guildwars2.com/forum/community/api/Map-API-Mumble-Mashup
http://mumble.sourceforge.net/Link
Any help would be greatly appreciated.
Cheers,
Ed.

You can try to map more than 20 bytes from the file in mmap call, say use 1024, unpack the whole thing according to the http://mumble.sourceforge.net/Link and then extract the name and camera position:
s = struct.unpack('IL3f3f3f512s3f')
name = s[11].decode('utf-16')
camera_pos_x,camera_pos_y,camera_pos_z = s[12:15]

For the names, create a character name in the game and make sure it gets written to disk - perhaps by exiting the game.
Then use a binary file editor to search for the name. I'm partial to http://sourceforge.net/projects/bpe/ , but there are many of these.
Finding camera positions could prove more difficult. I'd probably start out by finding character names, and then search nearby for things that could be camera positions.

Related

coordinate conversion script isn't giving me an accurate reading SVY21 to WGS84

I would like to convert my dataset of SVY21 coordinates, into WGS84 coordinates.
I am currently using this script from this repo I found but this script this yields inaccurate results with a discrepancy of up to 0.04, so the coordinates that I convert end up being on an entirely different geographical location in the same country.
Was wondering if there is anyone who can assist to help me with a script for converting a large dataset from SVY21 to WGS84?
E.G I want to convert
38816.0396118, 34379.9602051
but instead I get
1.36728713070321, 103.890645888016
when I should be getting
1.327235496598071, 103.93042021823591
I would do it on those online converters but my file sizes are pretty big(can go up to few GB) so it's better to run a script on my local computer instead using either Python or C++ or any other alternatives that will work. Also most online converters have a limit on file size as well.
Here's an accurate converter link: https://dominoc925-pages.appspot.com/webapp/calc_svy21/default.html but it doesnt accept my file size.
Appreciate the help :D Thanks~!
You probably have your coordinates the wrong way around. Consider the following:
import pyproj
xfm = pyproj.Transformer.from_crs('EPSG:3414', 'EPSG:4326')
x, y = 38816.0396118, 34379.9602051
print(xfm.transform(x, y))
# prints: (1.3673123058118237, 103.89064694097199)
print(xfm.transform(y, x))
# prints: (1.3271927478890677, 103.93050656742128)
Still about a ten-thousandth of a degree off, but I don't know how good pyproj's coordinate space definitions are in this case.

How to find objects in floor plan image in Tkinter python through svg file?

I have a vectorized floorplan image. I want to identify the objects in the image through the vector data in the SVG file of that image. The SVG code does not have any close points(z) in between them. So I am unable to understand when does the point moves to the other object? Can somebody help me, please?
I have very little knowledge about these SVG files and using them in Tkinter. So please somebody help me or suggest me what can I do?
This is the vector data of the image.
vector data of the image
use in conjunction with SO floorplan question.
Jump to z_final_floorplan.svg for final file.
A
Create 4 files:
w_original_floorplan.svg
x_rough_static_floorplan.svg
y_rough_live_floorplan.svg
z_final_floorplan.svg
w_original_floorplan.svg and x_rough_static_floorplan.svg are identical apart from filename.
y_rough_live_floorplan.svg and z_final_floorplan.svg are empty; to be populated.
Copy x_rough_static_floorplan.svg to y_rough_live_floorplan.svg.
Open y_rough_live_floorplan.svg on browser using server.
x_rough_static_floorplan.svg find all M and replace with two newlines / symbol M (case sensitive). shift + enter shift + enter /M
B
[this section takes the time]
Take away 1st '/' in path in y_rough_live_floorplan.svg [shows blackout_floorplan]
Label x_rough_static_floorplan.svg code section blackout_floorplan where code is.
(this file is used as rough-work, so being xml / svg valid is irrelevant)
In y_rough_live_floorplan.svg find next '/' and delete it [shows floorplan_top_left_whiteout]
Label x_rough_static_floorplan.svg code section floorplan_top_left_whiteout where code is.
Have x_rough_static_floorplan.svg and y_rough_live_floorplan.svg open in 2 windows, will be going back and forth to each of them. Keep repeating until at end.
(hint: find tool seems to be on switching from files in vscode, so you can use find / and next one cmd + g easily) Maybe handy to have a paper printout of original svg as reference and label the names of objects you create e.g.bath, sink, table, as you go along (don’t be fooled by this, one table is 'table'. Is 2nd chair chair2, chair_2, chair_two etc.?) etc..
C
Reorder the whole labels and corresponding code in path x_rough_static_floorplan.svg so the labels are ordered next to each other, but in the order they are found in the path:
e.g.
…
floorplan
bath
sink
table_chairs
sofa
…
Use the 'find' tool here. This process, itself will require a temp file to copy and paste to rather than reorder within the file working on. And rewrite temp to file working on. Might be good idea to create checklist of objects and cross-off as done.
E.g. floorplan, bath, table_chairs, sink…
D
Create path elements from your grouped objects, putting each id as id=“floorplan_main”, id=“bath”, id=“sink” etc.. etc..
Bear in mind, the data of how this is drawn is really, really bad. Really they should be drawn with rect elements for a rectangle when possible and a lot of the path data is very unnecessary, but that’s obviously how the application generates the svg.

Extract image position from .docx file using python-docx

I'm trying to get the image index from the .docx file using python-docx library. I'm able to extract the name of the image, image height and width. But not the index where it is in the word file
import docx
doc = docx.Document(filename)
for s in doc.inline_shapes:
print (s.height.cm,s.width.cm,s._inline.graphic.graphicData.pic.nvPicPr.cNvPr.name)
output
21.228 15.920 IMG_20160910_220903848.jpg
In fact I would like to know if there is any simpler way to get the image name , like s.height.cm fetched me the height in cm. My primary requirement is to get to know where the image is in the document, because I need to extract the image and do some work on it and then again put the image back to the same location
This operation is not directly supported by the API.
However, if you're willing to dig into the internals a bit and use the underlying lxml API it's possible.
The general approach would be to access the ImagePart instance corresponding to the picture you want to inspect and modify, then read and write the ._blob attribute (which holds the image file as bytes).
This specimen XML might be helpful:
http://python-docx.readthedocs.io/en/latest/dev/analysis/features/shapes/picture.html#specimen-xml
From the inline shape containing the picture, you get the <a:blip> element with this:
blip = inline_shape._inline.graphic.graphicData.pic.blipFill.blip
The relationship id (r:id generally, but r:embed in this case) is available at:
rId = blip.embed
Then you can get the image part from the document part
document_part = document.part
image_part = document_part.related_parts[rId]
And then the binary image is available for read and write on ._blob.
If you write a new blob, it will replace the prior image when saved.
You probably want to get it working with a single image and get a feel for it before scaling up to multiple images in a single document.
There might be one or two image characteristics that are cached, so you might not get all the finer points working until you save and reload the file, so just be alert for that.
Not for the faint of heart as you can see, but should work if you want it bad enough and can trace through the code a bit :)
You can also inspect paragraphs with a simple loop, and check which xml contains an image (for example if an xml contains "graphicData"), that is which is an image container (you can do the same with runs):
from docx import Document
image_paragraphs = []
doc = Document(path_to_docx)
for par in doc.paragraphs:
if 'graphicData' in par._p.xml:
image_paragraphs.append(par)
Than you unzip docx file, images are in the "images" folder, and they are in the same order as they will be in the image_paragraphs list. On every paragraph element you have many options how to change it. If you want to extract img process it and than insert it in the same place, than
paragraph.clear()
paragraph.add_run('your description, if needed')
run = paragraph.runs[0]
run.add_picture(path_to_pic, width, height)
So, I've never really written any answers here, but i think this might be the solution to your problem. With this little code you can see the position of your images given all the paragraphs. Hope it helps.
import docx
doc = docx.Document(filename)
paraGr = []
index = []
par = doc.paragraphs
for i in range(len(par)):
paraGr.append(par[i].text)
if 'graphicData' in par[i]._p.xml:
index.append(i)
If you are using Python 3
pip install python-docx
import docx
doc = docx.Document(document_path)
P = []
I = []
par = doc.paragraphs
for i in range(len(par)):
P.append(par[i].text)
if 'graphicData' in par[i]._p.xml:
I.append(i)
print(I)
#returns list of index(Image_Reference)

Digital cursor movement recorded in PDF creation visible in CorelDraw

I am using CairoSVG to turn an SVG into a PDF. I have found a bug where each line of text has a 1 dimensional mark at the end when the PDF is loaded into CorelDraw.
I find this is true even with the simplest and barest of SVGs. I then decided to dig into the python source of CairoSVG to fix the issue at it's root. I have found text.py to be the place where the issue occurs.
https://github.com/Kozea/CairoSVG/blob/master/cairosvg/text.py
Specifically, the for [x, y, dx, dy, r], letter in letters_positions: for loop. I can actually manipulate the mark by doing this after the for loop.
surface.context.move_to( 100, 100 )
surface.context.save()
surface.context.text_path(' ')
surface.context.restore()
Depending on the values I enter, it will move one of the points (in this case the lower right point of the mark). This suggests that when CairoSVG is done with a line of text, the surface will move the cursor to the start of the last letter it was drawing. For some reason, the last time it does this is actually recorded into the PDF, although for most PDF viewers it does not render. I have tried to fine tune this mark so the two points are the same, but because this location is different each time depending on letter and location and more, I can't fine tune this number to make the mark disappear, even when using available width and height variables
Note that surface.context is not a file or function and surface is pprinted as <cairosvg.surface.PDFSurface object at 0x7f647802ec90>, it's some sort of Cairo interface for interacting with the PDF file directly through some sort of PDF API, although I haven't found any documentation on it.
I have tried lots of cheap work arounds, like saving the PDF in another program then opening in Corel, however elements are lost or altered in ways that are not acceptable.
I have also of course tried commenting out this file 1 line at a time, then chunks of code at a time, but with no luck.
I have also looked into other files such as path.py, parser.py, and more, but text.py seems to be the most promising location.
How can I prevent this mark from appearing in my PDF files?
I have found I could get the desired results by going into the CairoSVG code and using a different function to create the text elements onto the PDF.
This collects the letters into a variable.
# Collect entire line instead of letter
text_line_letters = ''
for [x, y, dx, dy, r], letter in letters_positions:
text_line_letters = text_line_letters + letter
# Skip rest of for loop
continue
This uses the new function to creates the actual text onto the PDF.
# Create Text with glyph_path
surface.context.save()
font = surface.context.get_scaled_font()
glyphs, clusters, is_backwards = font.text_to_glyphs(
0, -5, text_line_letters, with_clusters=True)
surface.context.glyph_path(glyphs)
surface.context.restore()
The actual root cause of the issue is unknown, but likely deeper inside the Cairo library itself.
https://github.com/bonzini/cairo/blob/9099c7e7307a39bc630919faa65bba089fd15104/src/cairo.c#L3349

Trimming audio in python

ok so im using pyaudio as well but from what I been looking at the wave module could maybe help me out here.
So im trying to add a trimming function on my program, what I mean is im trying to allow the user to find parts of a wav. that he/she doesn't like and has the ability to trim the wave file to however he/she wants it.
so far i been using pyaudio for just simple playback and pyaudio is really easy when it comes to recording from an input device.
I been searching on pyaudio on anything I can do to trim audio but i really havent found anything that can help me. Though on the embedded wave module I see there are ways to set position.
Would I have to have a loop or if statement done so that the program would know which positions to record and then have either pyaudio or the wave module to record the song from user set positions (beginning, end)? Would my program run efficiently if I approached it this way?
Let's assume that you read in the wave file using scipy.
Then you need to have "edit points" These are in and out values (in seconds for example) that the user would like to keep. you could get these from a file or from displaying the wave file and getting clicks from the mouse. If the user gives parts of the audio file that should be removed then this would need to be first negated
This is not the most efficient solution but should be ok for many scenarios.
import scipy.io.wavfile
fs1, y1 = scipy.io.wavfile.read(filename)
l1 = numpy.array([ [7.2,19.8], [35.3,67.23], [103,110 ] ])
l1 = ceil(l1*fs1)#get integer indices into the wav file - careful of end of array reading with a check for greater than y1.shape
newWavFileAsList = []
for elem in l1:
startRead = elem[0]
endRead = elem[1]
if startRead >= y1.shape[0]:
startRead = y1.shape[0]-1
if endRead >= y1.shape[0]:
endRead = y1.shape[0]-1
newWavFileAsList.extend(y1[startRead:endRead])
newWavFile = numpy.array(newWavFileAsList)
scipy.io.wavfile.write(outputName, fs1, newWavFile)

Categories