This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Learning efficient algorithms
I recently came across an problem that was solved by applying the correct algorithm: Calculating plugin dependencies
While I was eventually able to understand the logic of the prescribed algorithm, it was not an easy task for me. The only reason I was able to come up with code that worked was because of the logic example on the wikipedia page.
Being entirely self taught, without any CS or math background, I'd like to at least get some practical foundation to being able to apply algorithms to solve problems.
That said, are there any great books / resources (something akin to 'algorithms for dummies') that doesn't expect you have completed college Algebra 9 or Calculus 5 that can teach the basics? I don't expect to ever be a wizard, just expand my problem solving tool-set a little bit.
Doing an amazon search turns up a bunch of books, but I'm hoping you guys can point me to the truly useful resources.
The only language I have any real experience with is Python (a tiny bit of C) so whatever I find needs to be language agnostic or centred around Python/C.
"Art of Computer Programming" by Donald Knuth is a Very Useful Book.
A great book is "Introduction to Algorithms" by Cormen, Leiserson, Rivest and Stein.
Probably not the easiest one but it is very good indeed.
I found useful for myself the following sources:
"Analysis of Algorithms : An Active Learning Approach" by Jeffrey J. McConnell;
"Python Algorithms: Mastering Basic Algorithms in the Python Language"(Expert's Voice in Open Source) by Magnus Lie Hetland. - this books seems to me to be a very like a previous book but from python developer point of view;
http://en.wikipedia.org/wiki/Structure_and_Interpretation_of_Computer_Programs
Steve Skiena's Algorithm Design Manual is very good. It doesn't assume very much background knowledge, and covers several important topics in algorithms.
Personally I found Algorithms and Complexity to be super helpful. I'm also without CS degree or anything.
Related
Basically, I want to recreate the conceptual results from the paper "Learning to Branch in Mixed Integer Programming" by Khalil, et al, at the same time avoiding, if possible:
1)The necessity of obtaining an academic license for CPLEX (which was used in the paper) or similar serious commercial solver
2)The necessity of using C based API. This is not a strict requirement, but Python has the benefit of having good and very accessible ML libraries, which seems like a great advantage for this specific goal
I am aware, that there is a great number of open source Python based MILP solvers, but a lot of them focus on the end-to-end solution of relatively simple problems in their presentation and, if we also consider, that a lot of them (if not all) hook up to other C based solvers, it is highly non-obvious to judge, which ones actually have needed customization potential.
So, if anyone has more in-depth experience with trying to customize Python solvers for their highly specific needs, I would appreciate the advice.
I'm afraid you will hit a roadblock at some point there. It's really hard to do that without doing C/C++ work (imho).
Python-way
I only know three projects with some low-level functionality (and it's still hard to say if those fit your needs).
https://github.com/coin-or/python-mip
relatively new
promises interactive cut-gen
has a chapter Developing Customized Branch-&-Cut algorithms
but i'm not sure if there is enough freedom for your task (seems to focus on cuts for now)
build around open-source solver Cbc/Clp (besides Gurobi)
https://github.com/coin-or/CyLP
not much develeopment for years now
the whole python-3 dev was sad (see issues; pull-request not processed for years; it's a resource problem: the maintainers are nice people!)
was designed to research pivoting
but it also says: For example, you may
.. define cut generators, branch-and-bound strategies
hard to see how to achieve what you look for except for abstract LP-relax - fix - resolve
might be hard to control specifics (warm-start vs. hot-start)
build around open-source Cbc/Clp
https://github.com/SCIP-Interfaces/PySCIPOpt
basic docs show more high-level usage
but it's internal code at least has entries for branchexeclp and co.
maybe it's ready to use (maybe not)
raw list of interface classes
as those things (maybe) wrap the original C-API, there is a lot of good documentation in the parent-project!
build around open-source solver SCIP
easier to grab the solver in academic setting, but by no means free (i'm not a lawyer and won't try to find the right words)
at least one developer of it is active on StackOverflow
Alternative: C++
If trying to get full-control; which might be needed, with minimal need for understanding the underlying solver in all it's details, you probably want to use C/C++ within Coin OSI. Sadly the Cbc part is unmaintained, but depending on your exact task, you might only need Clp for example.
Alternative: Julia
I did not follow the recent developments there, but the language did have a strong early focus on Mathematical Optimization (driven by a big group of people) surpassing python even in it's early days (imho!).
But i'm not sure if MathOptInterface is fine-grained enough for your task.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
So I'm making a TicTacToe "AI" and the code itself doesn't have any deep learning implications such as Tensor flow in min-max algorithms. The code simply consists of a jumble of if/else statements. My question is: can my code be considered an Artificial Intelligence because I'm literally training the code to play tic tac toe in a certain way when a player selects an area of play?
This IEEE publication has the following to say on the subject:
The term artificial intelligence denotes behavior of a
machine which, if a human behaves in the same way, is
considered intelligent.
It is difficult to extend this definition, because the definition
of what factors describe human intelligence is not clear.
Tic-Tac-Toe is a very simple game, so it is very easy to make a simple application behave exactly the same as an intelligent human would. So, if this is the definition of artificial intelligence to which you subscribe, then yes, you would be justified in calling your "jumble of if/else statements" an AI.
However, if you tell someone you wrote a Tic-Tac-Toe AI, they will immediately know that it is either (a) too complicated for what you're applying it to, or (b) not really very intelligent at all, but functional for its task.
The problem arises when people use "artificial intelligence" as a buzzword (buzzphrase?) that obscures how the AI is making decisions, which could potentially be because it's an ML black-box and they can't tell you exactly how it works either, or because they want to sell a product that sounds fancy but is actually just a jumble of if/else statements (or simply would not benefit from explaining it in detail in place of just saying "it's AI").
In this case I feel like it is a safe, if somewhat useless, application of the term.
This is really a question about word definition more than programming, but per artificial-intelligence:
This tag may also refer to the part of a computer game application that controls the behavior of the virtual characters with which the player may interact during the game.
So I guess in the context on Stack Overflow: yes, a tic-tac-toe decision tree is an AI.
In artificial intelligence, an intelligent agent (IA) is an autonomous
entity which observes through sensors and acts upon an environment
using actuators (i.e., it is an agent) and directs its activity
towards achieving goals.
Some agents are simple, and some agents are complex. A simple if/else statement is a very simple intelligent agent.
AI is not new. John McCarthy coined the term in 1955 and organized the first Artificial Intelligence conference in 1956.
In the last few decades, we have cracked building scalable intelligent agents that solve hard problems very fast; thanks to the technological improvement, faster & cheaper hardware & years of research by great people.
So a simple if/else is an intelligent agent considered as AI by the pioneers of AI whose tremendous work & years of research been open sourced resulted in building complex intelligent agents faster and simpler.
The definition of an intelligent agent coined by Stuart Russell and Peter Norvig in the book Artificial Intelligence: A Modern Approach. In the introduction of the book, the authors cited this example of simple if/else and confirmed that it's a simple, intelligent agent.
The credibility of the Book:
The leading textbook in Artificial Intelligence.
Used in over 1400 universities in over 125 countries.
The 22nd most cited computer science publication on Citeseer (and 4th most cited publication of this century).
The credibility of the Authors:
Stuart Russell, Ph.D., Computer Science, Stanford University
Peter Norvig, Director of Research at Google
My references:
Inteligent Agent link
John McCarthy link
The credibility of the Book aima.cs.berkeley.edu
Norvig Resume link
Russell Resume link
I am using Latent Dirichlet Allocation with a corpus of news data from six different sources. I am interested in topic evolution, emergence, and want to compare how the sources are alike and different from each other over time. I know that there are a number of modified LDA algorithms such as the Author-Topic model, Topics Over Time, and so on.
My issue is that very few of these alternate model specifications are implemented in any standard format. A few are available in Java, but most exist as conference papers only. What is the best way to go about implementing some of these algorithms on my own? I am fairly proficient in R and jags, and can stumble around in Python when given long enough. I am willing to write the code, but I don't really know where to start and I don't know C or Java. Can I build a model in JAGS or Python just having the formulas from the manuscript? If so, can someone point me at an example of doing this? Thanks.
My friend's response is below, pardon the language please.
First I wrote up a Python implementation of the collapsed Gibbs sampler seen here (http://www.pnas.org/content/101/suppl.1/5228.full.pdf+html) and fleshed out here (http://cxwangyi.files.wordpress.com/2012/01/llt.pdf). This was slow as balls.
Then I used a Python wrapping of a C implementation of this paper (http://books.nips.cc/papers/files/nips19/NIPS2006_0511.pdf). Which is fast as f*ck, but the results are not as great as one would see with NMF.
But NMF implementations I've seen, with scitkits, and even with the scipy sparse-compatible recently released NIMFA library, they all blow the f*ck up on any sizable corpus. My new white whale is a sliced, distributed implementation of the thing. This'll be non-trivial.
In Python, do you know of PyMC? It's flexible in specifying both the model and the fitting algorithm.
Also, when starting with R and JAGS, there is this tutorial on "Using JAGS in R with the rjags Package" together with a collection of examples.
I am looking for references (tutorials, books, academic literature) concerning structuring unstructured text in a manner similar to the google calendar quick add button.
I understand this may come under the NLP category, but I am interested only in the process of going from something like "Levi jeans size 32 A0b293"
to: Brand: Levi, Size: 32, Category: Jeans, code: A0b293
I imagine it would be some combination of lexical parsing and machine learning techniques.
I am rather language agnostic but if pushed would prefer python, Matlab or C++ references
Thanks
You need to provide more information about the source of the text (the web? user input?), the domain (is it just clothes?), the potential formatting and vocabulary...
Assuming worst case scenario you need to start learning NLP. A very good free book is the documentation of NLTK: http://www.nltk.org/book . It is also a very good introduction to Python and the SW is free (for various usages). Be warned: NLP is hard. It doesn't always work. It is not fun at times. The state of the art is no where near where you imagine it is.
Assuming a better scenario (your text is semi-structured) - a good free tool is pyparsing. There is a book, plenty of examples and the resulting code is extremely attractive.
I hope this helps...
Possibly look at "Collective Intelligence" by Toby Segaran. I seem to remember that addressing the basics of this in one chapter.
After some researching I have found that this problem is commonly referred to as Information Extraction and have amassed a few papers and stored them in a Mendeley Collection
http://www.mendeley.com/research-papers/collections/3237331/Information-Extraction/
Also as Tai Weiss noted NLTK for python is a good starting point and this chapter of the book, looks specifically at information extraction
If you are only working for cases like the example you cited, you are better off using some manual rule-based that is 100% predictable and covers 90% of the cases it might encounter production..
You could enumerable lists of all possible brands and categories and detect which is which in an input string cos there's usually very little intersection in these two lists..
The other two could easily be detected and extracted using regular expressions. (1-3 digit numbers are always sizes, etc)
Your problem domain doesn't seem big enough to warrant a more heavy duty approach such as statistical learning.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
Duplicate
What are some good resources for learning about Artificial Neural Networks?
I'm looking for a good (beginner level) reference book (or website) on different types of Neural Nets/their applications/examples. I don't have any particular application in mind, I'm just curious as to how I can make use of them. I'm specifically interested in using them with Python, but any language, or even just theory would do fine.
There is quite a extensive series of courses avaliable at Heaton Research. The course is for C# (Avaliable also for Java) however it explains the concepts at length, so I suggest you take a look at it even if you will code in python yourself.
The courses are in video format, however most important concepts are also writen down.
See the below three links for Neural Networks using Python:
An Introduction to Neural Networks
Weave a Neural Net with Python
Neural Networks in Pyro
Ron Stephens
"Programming collective intelligence" by Toby Segaran has a chapter about NN and also some examples in Python.
You might want to try out A Brief Introdruction to Neural Networks by David Kriesel. It's a richly illustrated ebook, and it's available for free. It covers lots of Network paradigms and is less theoretical than the ebook of Rojas. Seems to be the best on the web that you can get for free
AI-Junkie has a very good intuitive tutorial about neural networks. The site is designed to minimize the required mathematics so that the tutorial is accessible.
I am currently using this site as a primer - not python, but a good feed-forward network example and pretty straightforward to follow.
At the same time I have been reading The Essence of Neural Networks by Robert Callan (ISBN 0-13-908732-X) which has a wide range of network architectures and applications and is an easy read.
Since you mention python I should direct you to this IBM site, which I found very useful and the underlying code is in python here. Citation should go to Neil Schemenauer.
I should also mention that I took the python code and ported it to numpy because it ran very slowly. I was unsuccessful, but before I rubbish numpy I have to say that I suspect my implementation was not very good and I'm sure there is a vectorised way of doing forward passes and backpropagations, I just didn't find it.
What I have ended up doing is implementing in Java by a simple port of the python code. This only took an hour or so and it runs about 100 times faster. I think this is more proof that I don't know what I'm doing with numpy, but if you are starting from scratch I would question whether raw python is the right language for you. You may be better coding this sort of thing in C or C++ if you have to use python.
Best of luck.
I think you have the python bit covered with the answers given above. As for the "or even just theory", Raúl Rojas has a hefty ebook you can download from his wiki page.
The best reference is "Neural Networks for Pattern Recognition", by Bishop. Another good book is "Neural Networks and Learning Machines", by Haykin.
More practical references include the user guides of the Neural Network Toolbox for Matlab or the Open Source Neural Networks C++ Library Flood.
FAQ ANN
newsgroup comp.ai.neural-nets archieves usable online or offline