I am wondering if there are any python-based applications and examples about the 3D bin packing problem? I am facing a problem of planning the loading of thousands of items/boxes/pallets into the ocean containers (mainly 40HC).
Mannual planning, as most companies currently are doing, is very inefficient and painful. I am very interested to know if there is any python-based optimization toolkit that helps to solve such problem in a relatively accurate manner.
Allocating the items to containers are relatively easy, as I see from an example here: https://developers.google.com/optimization/bin/bin_packing#complete_programs
What is challenging is to generate a concrete plan telling people how to load each items into a specific container, like which box needs to be placed inside first, which item needs to be placed on top of others, so that the container storage space is maximally utilized.
And I hope to have a visual ouput like below. Are there any good coding examples?
I work on the Bin Packing problem and the only open-source python implementations that I have seen are those of academic research papers or heuristics. These have various limitations though such as not handling a large number of items or various physical constraints, and lacking a practical simulation guide for packing.
If you don't necessarily need access to the code, there is a growing number of sophisticated software for your use case, such as DeepPack by InstaDeep which is currently open for free access. It determines an optimal packing sequence using reinforcement learning, and shows a 3D simulation of the item packing order.
Related
I am writing code in Python to analyze social networks with node and edge attributes. Currently, I am using the NetworkX package to generate the graphs. Is there any limit to the size (in terms of the number of nodes, edges) of the graph which can be generated using this package?
I am new to coding social network problems in Python and have recently come across another package called NetworKit for large networks, but am not sure at what size should NetworKit be a better option, could you please elaborate on difference in performance and functionality between the two packages?
Thanks for your reply in advance.
My suggestion:
Start w/ Networkx as it has a bigger community, it's well mainteined and documented... and the best of all... you can easily understand what it does as it's 100% done in Python.
It's true it's not exactly fast, but it will be fast enough for most of the calculations. If you are running calculations from your laptop it can be slow for intensive calculations (Eg: sigma/omega small worldness metrics) in big networks (> 10k nodes and >100k vertexes).
If you need to speed it up, then you can easily incorporate networKit in your code as it integrates very easily to networkx and pandas, but it has a much more limited library of algorithms.
Compare yourself:
NetworkX algorithms: https://networkx.github.io/documentation/stable/reference/algorithms/index.html
VS
NetworKit algorithms: https://networkit.github.io/dev-docs/python_api/modules.html
Is there any limit to the size (in terms of the number of nodes, edges) of the graph which can be generated using this package?
No. There is no limit. It is all dependent on your memory capacity and size.
could you please elaborate on difference in performance and functionality between the two packages?
I personally don't have any experience with NetworkKit, however here (by Timothy Lin) you can find a very good benchmarking analysis on different tools including networkx and networkkit. Check out its conclusion section:
As for recommendations on which package people should learn, I think picking up networkx is still important as it makes network science very accessible with a wide range of tools and capabilities. If analysis starts being too slow (and maybe that’s why you are here) then I will suggest taking a look at graph-tool or networkit to see if they contain the necessary algorithms for your needs.
Basically, I want to recreate the conceptual results from the paper "Learning to Branch in Mixed Integer Programming" by Khalil, et al, at the same time avoiding, if possible:
1)The necessity of obtaining an academic license for CPLEX (which was used in the paper) or similar serious commercial solver
2)The necessity of using C based API. This is not a strict requirement, but Python has the benefit of having good and very accessible ML libraries, which seems like a great advantage for this specific goal
I am aware, that there is a great number of open source Python based MILP solvers, but a lot of them focus on the end-to-end solution of relatively simple problems in their presentation and, if we also consider, that a lot of them (if not all) hook up to other C based solvers, it is highly non-obvious to judge, which ones actually have needed customization potential.
So, if anyone has more in-depth experience with trying to customize Python solvers for their highly specific needs, I would appreciate the advice.
I'm afraid you will hit a roadblock at some point there. It's really hard to do that without doing C/C++ work (imho).
Python-way
I only know three projects with some low-level functionality (and it's still hard to say if those fit your needs).
https://github.com/coin-or/python-mip
relatively new
promises interactive cut-gen
has a chapter Developing Customized Branch-&-Cut algorithms
but i'm not sure if there is enough freedom for your task (seems to focus on cuts for now)
build around open-source solver Cbc/Clp (besides Gurobi)
https://github.com/coin-or/CyLP
not much develeopment for years now
the whole python-3 dev was sad (see issues; pull-request not processed for years; it's a resource problem: the maintainers are nice people!)
was designed to research pivoting
but it also says: For example, you may
.. define cut generators, branch-and-bound strategies
hard to see how to achieve what you look for except for abstract LP-relax - fix - resolve
might be hard to control specifics (warm-start vs. hot-start)
build around open-source Cbc/Clp
https://github.com/SCIP-Interfaces/PySCIPOpt
basic docs show more high-level usage
but it's internal code at least has entries for branchexeclp and co.
maybe it's ready to use (maybe not)
raw list of interface classes
as those things (maybe) wrap the original C-API, there is a lot of good documentation in the parent-project!
build around open-source solver SCIP
easier to grab the solver in academic setting, but by no means free (i'm not a lawyer and won't try to find the right words)
at least one developer of it is active on StackOverflow
Alternative: C++
If trying to get full-control; which might be needed, with minimal need for understanding the underlying solver in all it's details, you probably want to use C/C++ within Coin OSI. Sadly the Cbc part is unmaintained, but depending on your exact task, you might only need Clp for example.
Alternative: Julia
I did not follow the recent developments there, but the language did have a strong early focus on Mathematical Optimization (driven by a big group of people) surpassing python even in it's early days (imho!).
But i'm not sure if MathOptInterface is fine-grained enough for your task.
I am having a trouble picking the right data structure as/library . I lack in experience in the area of image processing / pattern recognition . The aim is to building a simple prototype to learn recognizing particular shapes from construction plans. I would be great full for any indication about the data structure as I know It will be hard to switch it later on during the project and thus I am not entirely sure which one to pick.
The problem is , I plan to use a kind of neural network / algorithm later on so the performance of processing of the data structure may happen to be my bottle neck.
I was thinking about NumPy / SciPy / PIL / MatPlotLib
I will be extremely grateful for expertise of anyone who has tackled similar problem
If you're planning on using something like PyTorch later on (which would tie in to the sort of neural-network functionality you're pursuing), I'd become familiar with how NumPy operates, as it bridges pretty well into Torch data structures. If it helps, a lot of SciPy and Matplotlib functions work beautifully with Numpy structures right out of the box.
It's hard to tell exactly what you're looking for in these non-neural data structures; where are you worried about performance concerns and bottlenecks?
I'd recommend starting out with some PyTorch (or other deep learning framework) tutorials regarding image recognition and classification; it will get you closer to where you want to end up, and you'll be better able to make decisions about what your eventual program structure needs will be.
Now I want to train my own image data in caffe using SegNet.
But at the first step we need label our own image like these:
I have tried to search github but cannot find anything. So my question is anyone know which tool can make semantic label image?
Check out a tool called sloth: https://github.com/cvhciKIT/sloth, which is an open-source tool written in Python with PyQt for creating ground truth computer vision datasets for a wide array of applications, such as semantically creating data like you have above.
If you don't like sloth, you can use any image editing software, like GIMP where you would make one layer per label and use polygons and flood fill of different hues to create your data. You would then merge all of the layers together to make a final image that you would use for your purposes.
However, as user Miki mentioned (see discussion thread below), creating new datasets from the beginning will take a considerable amount of effort. It is highly advisable that you don't create this on your own as you need a lot of data to ensure your algorithms are performing correctly. You'll need the help of other (hopefully willing) PhD students, preferably those you know personally or work with you in your lab or workplace to help manually curate this data for you.
If this isn't an option, you can use crowd sourced funded places like Amazon Mechanical Turk where you can outsource the work to willing individuals where you inform them of the task at hand and you pay a small amount per image. This would be something to consider if you can't find many people to help you.
All in all, this will take a considerable amount of effort, not only in terms of time but in terms of people if you want to create a large data set within a short span of time. I would recommend you simply use established datasets, such as what you have referenced from Cambridge, or Miki suggested LabelMe by Antonio Torralba which not only is a toolbox for annotating images from his LabelMe dataset but it also allows you to do the same for your own images.
Good luck!
As answer by #rayryeng a tool called sloth is great to finish these task in simple way. However, if I have more than 20 object waiting for me to classify, sloth is not a ideal tools. Thus I develop a simple tool which call IsLabel to finish these problem with few algorithms.
And the result look like these while using IsLabel just took me 40s:
INPUT:
OUTPUT:
I know its not perfect but it work fine for me.
I would recommend using https://www.labelbox.io/. They open sourced a lot of their code and have a hosting platform to manage the whole labeling process end to end.
Here is an example of segmentation
And you can export labels with a mask.
I am searching for a python package that I can use to simulate molecular dynamics in non-equilibrium situations. I need a setup that can handle a fairly large number of molecules in a primarily kinetic theory manner, and that can handle having solid surfaces present. With regards to the surfaces, I would need to be able to create arbitrary shapes and monitor pressure and other variables resulting from the molecular action. Alternatively, I could add the surface parts myself if I had molecules that could handle it.
Does anyone know of any packages that might be suitable?
Have you considered SimPy? SimPy is a rather generic Discrete Event Simulation package, but could feasibly meet your needs.
Better yet the Molecular Modelling ToolKit (MMTK) seems more specialized...
I have used neither, but this sounds like fun. Python, as a language, seems to be in privileged position for use in simulation software, whereby people can script the specific details of their model while relying on the framework for all the common logic, such as scheduling, visualization, monitoring etc. The unknown is how well such toolkits scale when fed with agent counts commensurate with biology models (BTW, how "big" is that?)
Lampps and gromacs are two well known molecular dynamics codes. These codes both have some python based wrapper stuff, but I am not sure how much functionality the wrappers expose. They may not give you enough control over the simulation.
Google for "GromacsWrapper" or google for "lammps" and "pizza.py"
Digital material and ASE are two molecular dynamics codes that expose a lot of functionality, but last time I looked, they were both fairly specialized. They may not allow you to use the force potentials that you want
Google for "digital material" and "cornell" or google for "ase" and dtu
Note to MJV: Normal MD-codes take one time step at a time, and they move all particles in each time step. Most of the time is spend calculating the total force on each atom. This involves iterating over a list of pairs of neighboring atoms. I think the best idea is to do the force calculation and a few more basics in c++ or fortran and then wrap that functionality in python. (But it could be fun to see how far one can get by using numpy matrices)
The following programs can be used to run MD symulations:
Gromacs
AMBER
charmm
OpenMM
many others...
The following Python packages are useful for preparing and analysing MD trajectories:
MDtraj and the OMNIA ecosystem
MDAnalysis
ProDy
MMTK
Another generic simulations framework is my own GarlicSim. You can try that. I could help you get a simpack up if you're serious about it.
I don't know if that programs does all the features you need but there is avogadro in the kde programs, i think it is extendable and since it is open source you could do anything with it. http://www.kde-apps.org/content/show.php/Avogadro?content=59521
It is really advanced and programmed by a friend of mine
I second MMTK, but take a look at VMD, which is the best MD software I'm aware of, and is Python-scriptable (in addition to Tk). See this for examples and tutorials.
I recommend to use molecular dynamics software to run MD simulations like Gromacs. This software is highly optimized for that particular purpose. You can also run on GPU's and you will be able to run larger systems in less time.
Afterwards, you run only the analysis with python packages using the generated trajectories.
mdtraj
pmx