I would like to perform a few basic machine vision tasks using Python and I'd like to know where I could find tutorials to help me get started.
As far as I know, the only free library for Python that does machine vision is PyCV (which is a wrapper for OpenCV apparently), but I can't find any appropriate tutorials.
My main tasks are to acquire an image from FireWire. Segment the image in different regions. And then perform statistics on each regions to determine pixel area and center of mass.
Previously, I've used Matlab's Image Processing Tootlbox without any problems. The functions I would like to find an equivalent in Python are graythresh, regionprops and gray2ind.
Thanks!
OpenCV is probably your best bet for a library; you have your choice of wrappers for them. I looked at the SWIG wrapper that comes with the standard OpenCV install, but ended up using ctypes-opencv because the memory management seemed cleaner.
They are both very thin wrappers around the C code, so any C references you can find will be applicable to the Python.
OpenCV is huge and not especially well documented, but there are some decent samples included in the samples directory that you can use to get started. A searchable OpenCV API reference is here.
You didn't mention if you were looking for online or print sources, but I have the O'Reilly book and it's quite good (examples in C, but easily translatable).
The FindContours function is a bit similar to regionprops; it will get you a list of the connected components, which you can then inspect to get their info.
For thresholding you can try Threshold. I was sure you could pass a flag to it to use Otsu's method, but it doesn't seem to be listed in the docs there.
I haven't come across specific functions corresponding to gray2ind, but they may be in there.
documentation: A few years ago I used OpenCV wrapped for Python quite a lot. OpenCV is extensively documented, ships with many examples, and there's even a book. The Python wrappers I was using were thin enough so that very little wrapper specific documentation was required (and this is typical for many other wrapped libraries). I imagine that a few minutes looking at an example, like the PyCV unit tests would be all you need, and then you could focus on the OpenCV documentation that suited your needs.
analysis: As for whether there's a better library than OpenCV, my somewhat outdated opinion is that OpenCV is great if you want to do fairly advanced stuff (e.g. object tracking), but it is possibly overkill for your needs. It sounds like scipy ndimage combined with some basic numpy array manipulation might be enough.
acquisition: The options I know of for acquisition are OpenCV, Motmot, or using ctypes to directly interface to the drivers. Of these, I've never used Motmot because I had trouble installing it. The other methods I found fairly straightforward, though I don't remember the details (which is a good thing, since it means it was easy).
I've started a website on this subject: pythonvision.org. It has some tutorials, &c and some links to software. There are more links and tutorials there.
You probably would be well served by SciPy. Here is the introductory tutorial for SciPy. It has a lot of similarities to Matlab. Especially the included matplotlib package, which is explicitly made to emulate the Matlab plotting functions. I don't believe SciPy has equivalents for the functions you mentioned. There are some things which are similar. For example, threshold is a very simple version of graythresh. It doesn't implement "Otsu's" method, it just does a simple threshold, but that might be close enough.
I'm sorry that I don't know of any tutorials which are closer to the task you described. But if you are accustomed to Matlab, and you want to do this in Python, SciPy is a good starting point.
I don't know much about this package Motmot or how it compares to OpenCV, but I have imported and used a class or two from it. Much of the image processing is done via numpy arrays and might be similar enough to how you've used Matlab to meet your needs.
I've acquired image from FW camera using .NET and IronPython. On CPython I would checkout ctypes library, unless you find any library support for grabbing.
Foreword: This book is more for people who want a good hands on introduction into computer or machine vision, even though it covers what the original question asked.
[BOOK]: Programming Computer Vision with Python
At the moment you can download the final draft from the book's website for free as pdf:
http://programmingcomputervision.com/
From the introduction:
The idea behind this book is to give an easily accessible entry point to hands-on
computer vision with enough understanding of the underlying theory and algorithms
to be a foundation for students, researchers and enthusiasts.
What you need to know
Basic programming experience. You need to know how to use an editor and run
scripts, how to structure code as well as basic data types. Familiarity with Python or other scripting style languages like Ruby or Matlab will help.
Basic mathematics. To make full use of the examples it helps if you know about
matrices, vectors, matrix multiplication, the standard mathematical functions
and concepts like derivatives and gradients. Some of the more advanced mathe-
matical examples can be easily skipped.
What you will learn
Hands-on programming with images using Python.
Computer vision techniques behind a wide variety of real-world applications.
Many of the fundamental algorithms and how to implement and apply them your-
self.
Related
Basically, I want to recreate the conceptual results from the paper "Learning to Branch in Mixed Integer Programming" by Khalil, et al, at the same time avoiding, if possible:
1)The necessity of obtaining an academic license for CPLEX (which was used in the paper) or similar serious commercial solver
2)The necessity of using C based API. This is not a strict requirement, but Python has the benefit of having good and very accessible ML libraries, which seems like a great advantage for this specific goal
I am aware, that there is a great number of open source Python based MILP solvers, but a lot of them focus on the end-to-end solution of relatively simple problems in their presentation and, if we also consider, that a lot of them (if not all) hook up to other C based solvers, it is highly non-obvious to judge, which ones actually have needed customization potential.
So, if anyone has more in-depth experience with trying to customize Python solvers for their highly specific needs, I would appreciate the advice.
I'm afraid you will hit a roadblock at some point there. It's really hard to do that without doing C/C++ work (imho).
Python-way
I only know three projects with some low-level functionality (and it's still hard to say if those fit your needs).
https://github.com/coin-or/python-mip
relatively new
promises interactive cut-gen
has a chapter Developing Customized Branch-&-Cut algorithms
but i'm not sure if there is enough freedom for your task (seems to focus on cuts for now)
build around open-source solver Cbc/Clp (besides Gurobi)
https://github.com/coin-or/CyLP
not much develeopment for years now
the whole python-3 dev was sad (see issues; pull-request not processed for years; it's a resource problem: the maintainers are nice people!)
was designed to research pivoting
but it also says: For example, you may
.. define cut generators, branch-and-bound strategies
hard to see how to achieve what you look for except for abstract LP-relax - fix - resolve
might be hard to control specifics (warm-start vs. hot-start)
build around open-source Cbc/Clp
https://github.com/SCIP-Interfaces/PySCIPOpt
basic docs show more high-level usage
but it's internal code at least has entries for branchexeclp and co.
maybe it's ready to use (maybe not)
raw list of interface classes
as those things (maybe) wrap the original C-API, there is a lot of good documentation in the parent-project!
build around open-source solver SCIP
easier to grab the solver in academic setting, but by no means free (i'm not a lawyer and won't try to find the right words)
at least one developer of it is active on StackOverflow
Alternative: C++
If trying to get full-control; which might be needed, with minimal need for understanding the underlying solver in all it's details, you probably want to use C/C++ within Coin OSI. Sadly the Cbc part is unmaintained, but depending on your exact task, you might only need Clp for example.
Alternative: Julia
I did not follow the recent developments there, but the language did have a strong early focus on Mathematical Optimization (driven by a big group of people) surpassing python even in it's early days (imho!).
But i'm not sure if MathOptInterface is fine-grained enough for your task.
I am having a trouble picking the right data structure as/library . I lack in experience in the area of image processing / pattern recognition . The aim is to building a simple prototype to learn recognizing particular shapes from construction plans. I would be great full for any indication about the data structure as I know It will be hard to switch it later on during the project and thus I am not entirely sure which one to pick.
The problem is , I plan to use a kind of neural network / algorithm later on so the performance of processing of the data structure may happen to be my bottle neck.
I was thinking about NumPy / SciPy / PIL / MatPlotLib
I will be extremely grateful for expertise of anyone who has tackled similar problem
If you're planning on using something like PyTorch later on (which would tie in to the sort of neural-network functionality you're pursuing), I'd become familiar with how NumPy operates, as it bridges pretty well into Torch data structures. If it helps, a lot of SciPy and Matplotlib functions work beautifully with Numpy structures right out of the box.
It's hard to tell exactly what you're looking for in these non-neural data structures; where are you worried about performance concerns and bottlenecks?
I'd recommend starting out with some PyTorch (or other deep learning framework) tutorials regarding image recognition and classification; it will get you closer to where you want to end up, and you'll be better able to make decisions about what your eventual program structure needs will be.
sorry if this all seem nooby and unclear, but I'm currently learning Netlogo to model agent-based collective behavior and would love to hear some advice on alternative software choices. My main thing is that I'd very much like to take advantage of PyCuda since, from what I understand, it enables parallel computation. However, does that mean I still have to write the numerical script in some other environment and implement the visuals in yet another one???
If so, my questions are:
What numerical package should I use? PyEvolve, DEAP, or something else? It appears that PyEvolve is no longer being developed and DEAP is just a wrapper on the outdated(?) EAP.
Graphic-wise, I find mayavi2 and vtk promising. The problem is, none of the numerical package seems to bind to these readily. Is there no better alternative than to save the numerical output to datafile and feed them into, say, mayavi2?
Another option is to generate the data via Netlogo and feed them into a graphing package from (2). Is there any disadvantage to doing this?
Thank you so much for shedding light on this confusion.
You almost certainly do not want to use CUDA unless you are running into a significant performance problem. In general CUDA is best used for solving floating point linear algebra problems. If you are looking for a framework built around parallel computations, I'd look towards OpenCL which can take advantage of GPUs if needed..
In terms of visualization, I'd strongly suggest targeting a a specific data interchange format and then letting some other program do that rendering for you. The only reason I'd use something like VTK is if for some reason you need more control over the visualization process or you are looking for a real time solution.
Probably the best choice for visualization would be to use an intermediate format and do it in another program. But for performance, i'd rather configure a JVM for a cluster and run NetLogo on it. I've not tried it yet but i'm thinking seriously to try NetLogo on a Beowulf style cluster.
BTW, there is an ABM platform called Repast that is said to have Python interface if you're planning to implement your code in Python.
So, I use SPM to register fMRI brain images between the same patient; however, I am having trouble registering images between patients.
Essentially, I want to register a brain atlas to a patient-specific scan, so that I can do some image patching. So register, then apply that warping and transformation to any number of images.
SPM was unsuccessful in such a registration. It cannot warp the atlas to be in the same brain shape as the patient brain.
Would software such as freesurfer be good for this?? Or is there something better out there in either matlab or python (but preferably python)??
Thanks!
tylerthemiler
There is a bulk of tools for image registration, e.g. look at http://www.nitrc.org under "Spatial transformation" -> "Registration". Nipype is indeed a nice Python module which wraps many of those (e.g. FSL, Freesurfer, etc) so you could explore different available tools within somewhat unified interface.
Besides those well known (SPM, FSL, AFNI) also you could give a try to somewhat less known but very powerful CMTK (http://www.nitrc.org/projects/cmtk) which comes with non-linear registration(s), population-based template construction, many other features and SRI24 atlas. Such script as asegment_sri24 could be used for a quick start with registering/reslicing each subject using labels available in SRI24 atlas.
To start using CMTK (or dozens of other neuroimaging software) in a matter of minutes I would recommend you to look at http://neuro.debian.net -- the platform to allow very easy deployment of (maintained) neuroscience software. FSL, AFNI, CMTK, SRI24 atlas etc are available there upon your demand ;)
Freesurfer segments and annotates the brain in the patient's native space, resulting in patient-specific regions, like so.
I'm not sure what you mean by patching, or to what other images you'd like to apply this transformation, but it seems like the software most compatible for working with individual patient data, rather than normalized data across patients.
I think ITK is made for this kind if purpose. A Python wrapper exists (Paul Novotny distributes binaries for Ubuntu on his site), but this is mainly C++. If you work under Linux then it is quite simple to compile if you are familiar with cmake.
As this toolkit is a very low-level framework I can advise you to try elastix which is a command line utility allowing one to make registration on picture using multiscale Bspline dense registration.
Another interesting tool based on Maxwell demons and improved with diffeomorphic capabilities is MedINIRA.
Along SPM's lines, you can use IBSPM. It was developed to solve exactly that problem.
You can use ANTs software, or u can use Python within 3dSclicer for template registration.
However, I did mane template registration in SPM and I recommend it for fMRI data better than ITK or Slicer.
I found these links very helpful :) let me know if you need more help.
https://fmri-training-course.psych.lsa.umich.edu/wp-content/uploads/2017/08/Preprocessing-of-fMRI-Data-in-SPM-12-Lab-1.pdf
https://nipype.readthedocs.io/en/latest/users/examples/fmri_spm.html
I am C/C /Java programmer, but lately I have started learn Python.
Moreover I have 3D Graphics on my studies. I have to create 3D model of my apartment, with dynamic camera. I am wondering if this is a good idea to merge this two issues, by writing this 3D model in python.
However as I said, I am a python beginner, so I don't know possibilities, which python can give me in this area. Which libraries/engine will be the best for a start?
You can also checkout Pyglet, which is a higher-level library for using with OpenGL.
IMHO C++ is a better option for 3D graphics, and you have experience in it.
Though you can have a look at PyOpenGL, python bindings for OpenGl.
http://pyopengl.sourceforge.net/
If you're solely trying to learn how to do a 3d model go for the language you're the most familiar with. I'd recommend C++ or C# in that case (whichever of the 2 you meant with the second C).
If you also want to learn more about the language Python is the better choice.
But pure language wise I wouldn't say that C++/c#/python beats the other.
There are a lot of options and libraries regardless of the language you choose. But if you need a quick and dirty model you might want to check out VPython, which is python plus a simple 3D library (http://vpython.org/index.html. There are video tutorials at: http://www.youtube.com/vpythonvideos
The programming part would likely be very easy using VPython.
The hard part is creating the object for your apartment, which you might have to do by hand, with hand coding of the coordinates to define each object. The VPython site has a utility for importing .stl files, and many 3D programs, e.g., Blender, will export .stl files. (I've never used this, so I can't speak to how well it works).
If you just need to model the walls, floor, and ceiling, and maybe a simple object or two, it wouldn't be hard to hand-code the coordinates. Sketch it out on graph paper and make notes of the z axis values.