Python library for computing spatial derivatives of optical flow - python

I'm trying to compute a differential image velocity invariants (e.g. curl, divergence, deformation, etc) from a video using OpenCV in Python. To do that, I need to compute the spatial derivatives in the x,y directions of the optical flow. Unfortunately, OpenCV only seems to supply the APIs for computing optical flow, not its derivative.
Are there any Python libraries out there for computing spatial derivatives of optical flow? I found this SO question that was somewhat similar Lucas Kanade Optical Flow, Direction Vector, and there is code the person wrote for computing spatial derivatives, but if at all possible I'd love a library rather than writing the code myself. Any suggestions would be appreciated!

This is the way I see it (I've worked with optical flow a little bit):
You want to compute the individual partial derivatives of the optical flow field; one for the x direction, and one for the y.
I'd attempt to solve the problem like so:
Split your flow array/matrix into two matrices: x and y flow.
For each of those, you could go the naive route and just do a simple difference: derivative = current_state - last_state. But this approach is very messy, as the derivative will be sensitive to the slightest bit of error.
To counter that, you could approximate one chunk of your data points (maybe a whole row?) with a regression curve that is easily differentiable, like a polynomial.
The just differentiate that approximated curve and you're good to go.
You could also just smooth individual matrices and do a naive difference, which should be much faster than approximating data points, but should be more tolerant to error.

Related

Solving a se(3) optimization problem using SciPy?

I want to use scipy.optimize package to solve a rigid body optimization problem (finding optimal translation and rotation of a object).
A way to parametrize this problem is through lie group theory and the lie algebra se(3) (often called twist coordinates). This means the state vector is composed of 6 parameters (i.e. pose = [rho_x, rho_y, rho_z, theta_0, theta_1, theta_3]) from which the SE(3) Transformation matrix (a composition of translation vector and rotation matrix) can be obtained through the so called 'wedge' or 'hat' operator and the matrix exponentiation. (I found this cheat-sheet to be an excellent overview)
Now my question: Can scipy be used to optimize a 3D pose that is parametrized by the twist coordinates? I read somewhere that such problems require 'on manifold optimization' methods. So can I use scipy's least_square() or minimize(method="BFGS") for this? or do these only work for flat euclidean spaces?
Alternatively it would also help me a lot if anyone knows of any example where scipy is used to solve a 3D pose optimization problem. I have been looking for examples for the last couple of days but without much success.
This question is a implementation specific part of a more general question I posted on the robotics stackexchange here.

How to get a time series based on a spectrogram in Python?

I have a time series and generate its spectrogram in Python with matplotlib.pyplot.specgram.
After I make some analysis and changes I need to convert the spectrogram back into time series.
Is there any function in matplotlib or in other library that I can use directly? Or if not, could you please elaborate on which direction I should work on?
Your warm help is appreciated.
Matplotlib is a library for plotting data. Generally if you're trying to do any computation you'd use a library suited for that.
numpy is a very popular library for doing numerical computation in Python. It just so happens they have a fairly extensive set of fft and ifft methods.
I would check them out here and see if they can solve your problem.
One thing commonly done (for example in the source separation community) is to use the phase data of the original signal (before transformation where applied to it) - the result is much better than null or random phase, and not so far from algorithms aiming at reconstructing the phase information from scratch.
A classic reconstruction algorithm is Griffin&Lim's, described in the paper "Signal estimation from modified short-time Fourier transform". This is an iterative algorithm, each iteration requires a full STFT / inverse STFT, which makes it quite costly.
This problem is indeed an active area of research, a search for STFT + reconstruction + magnitude will yield plenty of papers aiming at improving on Griffin&Lim in terms of signal quality and/or computational efficiency.
You can find detailed dicussion hereThread on DSP Stack Exchange

Standard procedure for geometric transformations

I'm trying to build a very basic tool for a problem in mechanical engineering and need to do simple transformations of coordinates and vectors in euclidian space which include translations + rotations.
For example
a component part with a local coordinate system (cs) is moved in respect to a world cs. I need to compute the new position of the (origin of) the local cs and its orientation.
a vector is shifted and rotated in respect to a cs. Its new position has to be computed.
I'm aware of different concepts for doing these computations:
Euler angles
Quaternions
Homogeneous coordinates
From my POV the use of homogeneous coordinates would be the simplest approach because it is possible to compute translations and rotations in one step.
Questions:
What is the most common approach in programming to implement this kind of computations?
Is there a python library which can handle these transformations? I found some smaller libraries like transformations.py but I guess transformations like these are very very common and so I wonder if this isn't part of scipy or something like that.
After all i assume i'm searching for the wrong terms and would be glad if someone could provide a hint for further reading, code examples, libraries (especially for python).
Use numpy and linear algebra to do the transformations as matrix multiplications

Python library for GPS trajectory pre-processing?

I'd like to know if there is any implemented python library for GPS trajectory pre-processing such as compression, smoothing, filtering, etc.
Expanding on my comment, a Kalman filter is the usual choice for estimating position and velocity from noisy sensor readings.
Here's what Wikipedia has to say on the topic (emphasis mine:)
The Kalman filter is an algorithm, commonly used since the 1960s for
improving vehicle navigation (among other applications, although
aerospace is typical), that yields an optimized estimate of the
system's state (e.g. position and velocity). The algorithm works
recursively in real time on streams of noisy input observation data
(typically, sensor measurements) and filters out errors using a
least-squares curve-fit optimized with a mathematical prediction of
the future state generated through a modeling of the system's physical
characteristics.
The Kalman filter is the basic version; there's also the extended Kalman filter and unscented Kalman filter (though my control systems lecturer never got around to telling us what those were actually used for.)
#stark has provided a link to an implementation of the Kalman filter in Python (not sure of the quality.) You may be able to find others, or roll your own with scipy.
Not GPS-specific, but numpy has general statistics and scientific algorithms. For example, if you want to make a best-fit line to a series of points, you would run a linear regression on the data.

How can I fit a Bézier curve to a set of data?

I have a set of data points (which I can thin out) that I need to fit with a Bézier curve. I need speed over accuracy, but the fit should be decent enough to be recognizable. I'm also looking for an algorithm I can use that doesn't make much use of libraries (specifically NumPy).
I've read several research papers, but none has enough detail to fully implement. Are there any open-source examples?
I have similar problem and I have found "An algorithm for automatically fitting digitized curves" from Graphics Gems (1990) about Bezier curve fitting.
Additionally to that I have found source code for that article.
Unfortunately it is written in C which I don't know very well. Also, the algorithm is quite hard to understand (at least for me). I am trying to translate it into C# code. If I will be successful, I will try to share it.
File GGVecLib.c in the same folder as FitCurves.c contains basic vectors manipulation functions.
I have found a similar Stack Overflow question, Smoothing a hand-drawn curve. The approved answer provide C# code for a curve fitting algorithm from Graphic Gems.
What's missing in a lot of these answers is that you may not want to fit a single cubic Bézier curve to your data. More generally, you would like to fit a sequence of cubic Bézier curves, i.e., a piecewise cubic Bézier fit, to an arbitrary set of data.
There's a nice thesis dating from 1995, complete with MATLAB code, that does this:
% Lane, Edward J. Fitting Data Using Piecewise G1 Cubic Bezier Curves.
% Thesis, NAVAL POSTGRADUATE SCHOOL MONTEREY CA, 1995
http://www.dtic.mil/dtic/tr/fulltext/u2/a298091.pdf
To use this, you must, at minimum, specify the number of knot points, i.e., the number data points that will be used by the optimization routines to make this fit. Optionally, you can specify the knot points themselves, which increases the reliability of a fit. The thesis shows some pretty tough examples. Note that Lane's approach guarantees G1 continuity (directions of adjacent tangent vectors are identical) between the cubic Bézier segments, i.e., smooth joints. However, there can be discontinuities in curvature (changes in direction of second derivative).
I have reimplemented the code, updating it to modern MATLAB (R2015b). Contact me if you would like it.
Here's an example of using just three knot points (chosen automatically by the code) the fits two cubic Bézier segments to a Lissajous figure.
If most of the data fits the model you could try RANSAC. It would be easy enough to pick 4 points and random and fit a bezier curve from those. I'm not sure off the top of my head how expensive it would be to evaluate the curve against all the other points (part of the RANSAC algorithm). But it would be a linear solution and RANSAC is really easy to write (and there are probably open source algorithms out there).
First of all, make sure what you ask for is actually what you want. Fitting the points to a Bezier curve will place them in the hull of the points. Using a spline will make sure your curve goes through all points.
That said, creating the function that draws either is not complicated at all. Wikipedia has a nice article that will explain the basics, Bézier curve.
I had a MATLAB solution for this problem.
I encountered the same problem, but my code is written in MATLAB.
I hope its not too hard to translate it into Python.
You can find the control points by this code FindBezierControlPointsND.m
For some reason, it does not have function "ChordLengthNormND" in its archive,
but it is called at line 45.
I replaced it by following lines:
[arclen,seglen] = arclength(p(:,1),p(:,2),'sp');
t = zeros(size(p,1),1);
sums = seglen(1);
for i = 2:size(p,1)-1
t(i) = sums / arclen;
sums = sums + seglen(i);
end
t(end) = 1;
arclength's MATLAB code can be obtained here.
After that, we have control points for Bezier curve, and there are lots of implementation of building Bezier curve by control points on the web.

Categories