We Keep Coding

ESB for Python? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
At work we're debating choosing an off-the-shelf full-stack ESB (Enterprise Service Bus -- http://en.wikipedia.org/wiki/Enterprise_service_bus ) versus rolling our own around an AMQP system such as RabbitMQ.
Part of the concern is that our ecosystem is PHP moving as fast as possible to Python, and what APIs we already have all pass messages in JSON. The search space on ESBs and 'SOA solutions' is really, really polluted; either that or this just isn't a Python Thing(tm). Thus I ask here: it seems that all full-stack ESB solutions that we've considered are primarily focused on integrating SOAP-based components. That seems like a lot of double wrapping and unwrapping of messages that we may not want to do. Also, we'd really like to write our service-to-ESB adapters in Python if we can.
I should note here that the PHP/Python ecosystem is internal -- many of our clients are fairly hidebound organizations using big monolithic enterpriseware with which our apps integrate. They would be happy with SOAP, XML-RPC etc for our customer-facing APIs. We've already had requests. So, at the border, we may end up eventually providing both JSON-based and SOAP-based services; just not internally.
tl;dr: Does anybody have a favored full-stack ESB for use with Python apps? What about a non-favored one that you've worked with (and how terrible was it, anyway)? Do you think that wrapping something like RabbitMQ is a better idea for a PHP/Python ecosystem?

It really is a big question, more like a couple of questions in one, if I may. So please excuse me if I misunderstood you.
An ESB is, by definition, not really coupled to your choice of language in implementing your applications. So no matter if your apps that you want to connect in a SOA manner are written in PHP or Python, you should be able to expose them in a SOA. JSON on the other hand is a bit of an off choice in the SOA world and this choice is really important.
Many of the COTS ESBs offer some level of REST support. Mule, ServiceMix, WSO2, JBoss are some open source ones. If you prefer to go with a commercial vendor I believe Tibco and Oracle have either out-of-the-box support or can be implemented with a bit of coding.
Your choice of using any of the message brokers such as RabbitMQ or AMQP is really about the transport - which is once again not really directly related. Then again, if your goal is to add messaging among your applications it is a valid question which I cannot really answer.
If you can write your main motivation for your interest in an ESB it would be helpful for a better answer.
HTH

Wow...that's quite the question, and there's no way that I can answer it fully...but since you asked here's the proverbial 2 cents:
RabbitMQ is pretty good. I'm using it for real-time messaging in a multi-agent system that implements caching on a scientific computing grid. One of the nice features of RabbitMQ is that you can push arbitrary (including binary) messages, which might be a nice feature for performance. (Perhaps using the serialization / deserialization from the Hessian protocol?)
My only concern with RabbitMQ would be the continued support of the python libraries for AMQP. The current libraries are functional, I've used them...but they haven't been around long, and I'm not sure how big the community is that's interested in and supporting such work. That said, it seems like RabbitMQ (AMQP, and messaging in general) are becoming a hotter topic as of late, and hopefully that would broaden the base that's working on the python - AMQP interface libraries.

How can I learn more about Python’s internals? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I have been programming using Python for slightly more than half an year now and I am more interested in Python internals rather than using Python to develop applications. Currently I am working on porting a few libraries from Python2 to Python3. However, I have a rather abstract view on how to make port stuff over from Python2 to Python3 as most of the changes deal with design issues in Python2.x
I'd like to learn more about Python internals; should I go for a top-down or a bottom-up approach? Are there any references you could recommend?

It sounds like you want to know more about the rationale behind the design of the language, rather than internals. "internals" to me means things like how objects are laid out in memory, how reference counting works, and so on.
If you're looking for a deeper understanding of the design decisions, try reading the PEPs: they are the proposals for changes in the language, and often include detailed discussions of the reasons for the changes, rejected alternatives, and so on. Even the rejected PEPs are useful, because they show the thinking that has shaped the language.
For example:
3105: Making print a function
3110: Catching exceptions in Python 3.x
3131: Supporting non-ASCII identifiers
and so on..
If you really want to learn about Python internals, then start by reading about the Python C API, which is used to build Python itself: my talk A Whirlwind Excursion through Python C Extensions is one place to start. Then you can dive into the Python source code itself for anything you need to learn about.

To someone who is stumbling upon this question from related links or search, there is a documentation written Yaniv Aknin on Python Internals. It starts from the scratch and is highly readable.

I find the series of Yaniv Aknin's Pythons Innards series
fantastic, too
I discovered it thanks to Planet Python
.
You may be also interested by the answer of TryPyPy in this SO thread

I would first read the What's New document for Python 3. It gives a good high-level overview and touches on the detailed changes.
You might also do a search for 'porting to python 3' or similar. There are lots of good resources and tools.
One tool that's new and hard to find is six, by Benjamin Peterson. It enables writing of code that is compatible across the Python 2*3 gap.
The part I found most difficult about maintaining Python 2 and Python 3 -compatible code was deployment. I could write code that would run just fine, but when I went do package and deploy, it was unclear when the conversion should happen. I ultimately found a distutils command build_py_2_to_3 that would do the trick. By using that command in my setup.py, I could release a source distribution that would deploy on either Python 2 or Python 3. An example can be found in jaraco.util.
You also asked about the internals. If you really want to get at the internals, you can view the source for Python 2.x and Python 3.x, though honestly, I would stick with reading the tutorials and maybe some of the .py files in the Python libs.

should I go for a top-down or a bottom-up approach?
Both! Seriously.

Have you tried this?
Automated Python 2 to 3 code
translation

Simple non-web based bug tracker [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
There's a ton and and a half of questions and even more answers here concerning people looking for bug trackers. However all of them (that I found) seem to be about web based solutions. Since I'm working on a local project where I don't want to set up a web / DB server, and I don't want to use a hosted tracker either, I'm looking for something that runs locally.
very preferably open-source
pure Python or (at least) Windows executable
no need for a database server (sqlite is obviously fine)
Doesn't have to be fancy, just the basic bug / issue tracking functionality; just a little bit more than my current TODO text file or an Excel table.
Any suggestions?

I'm surprised nobody has mentioned Roundup.
It meets all your criteria, including not requiring a web-based interface (as per your specification, and unlike the accepted answer which suggested Trac).
Roundup is:
Open source
Pure Python
Supports SQLite
Not fancy, focuses on solid bug tracking
And as a significant point of differentiation, it has command-line and email interfaces in addition to a web interface.
It's very easy to get started - I suggest you take it for a spin.

Trac might be a bit too over engineered, but you could still run it locally via tracd on localhost.
It's:
opensource.
pure Python
uses sqlite
But as I said, might be too complex for your use case.
Link: http://trac.edgewall.org

If you don't need to share your bug tracker system with a team (i.e., it's okay to have it isolated to your computer) I would recommend using Tiddlywiki. Technically it's web-based, but it's entirely encapsulated within a single HTML document and requires no database or server whatsoever (only a web browser) so I think it follows the spirit of what you are wanting. It's extremely customizable since it's 100% HTML/CSS/javascript. I have been using a tiddlywiki as a project notebook for years, keeping track of my to-do list, bug list, and general project documentation in one centralized, cross-referenced place. You can also find all sorts of tiddlywikis that you can download pre-configured for productivity (for example, TeamTasks, MonkeyGTD, or GTDTiddlyWiki Plus).

Maybe Fossil is of any use to you?
It is actually a DVCS but it also integrates a bugtracker and wiki, very much like trac (although I like trac, don't get me wrong). And its webbased, on the other hand the installation is supossedly dead simple.

Proprietary TestTrack (http://www.seapine.com/ttpro.html) has a client edition that will those things. We use it at work and I'm very happy using it.
Maybe you can check out this wikipedia article for hints
http://en.wikipedia.org/wiki/Comparison_of_issue_tracking_systems

Do yourself a favor. Get over this "must not be web based" obsession, Install a local WAMP stack on your PC or on a LAN server. Now, you can install your own wiki. And something like Trac. I'd like to find an implementation of google code's bugtracker and integrated wiki thats runnable locally - Trac seems to be the closest.
You have also installed a local SVN server? Even for personal projects the ability to track changes over time. revert etc. and integration with Trac are too good to pass up even for purely 1 man projects.

What's the best online tutorial for starting with Spring Python [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
Spring Python seems to be the gold-standard for how to define good quality APIs in Python - it's based on Spring which also seems to be the gold-standard for Java APIs.
My manager has complained (with good reason) that our APIs are in a mess - we need to impose some order on them. Since we will be re-factoring it makes sense to take advantage of what is considered best practice - so we would like to consider Spring.
Could somebody point me to the best learning resources for getting started with Spring? I've googled for a while and not found anything which seems to start from first principles. I'm looking for something which assumes good knowledge of Python but zero knowledge of Spring on other platforms or it's principles.

How did you come to decide on Spring Python as your API of choice? Spring works well on Java where there's a tradition of declarative programming; defining your application primarily using XML to control a core engine is a standard pattern in Java.
In Python, while the underlying patterns like Inversion of Control are still apposite (depending on your use case), the implementation chosen by Spring looks like a classic case of something produced by a Java programmer who doesn't want to learn Python. See the oft-referenced article Python is Not Java.
I applaud your decision to introduce order and thoughtfulness to your codebase, but you may wish to evaluate a number of options before making your decision. In particular, you may find that using Spring Python will make it difficult to hire good Python programmers, many of whom will run the other way when faced with 1000-line XML files describing object interactions.
Perhaps start by re-examining what you really want to accomplish. The problem cannot simply be that "you need a framework". There are lots of frameworks out there, and it's hard to evaluate a) if you truly need one and b) which one will work if you haven't identified what underlying software problems you need to solve.
If the real problem is that your code is an unmaintainable mess, introducing a framework probably won't fix the issue. Instead of just messy code, you'll have code that is messy in someone else's style :-) Perhaps rigour in the dev team is where you should recommend starting first: good planning, code reviews, stringent hiring practices, a "cleanup" release, etc...
Good luck with the research.

I won't go so far as to suggest that Spring Python is bad (because I don't know enough about it). But, to call Spring Python the "gold standard for Python APIs" is a stretch. To me, it seems that Spring Python is more of a way to allow Python apps to interact with Java Apps using Spring.
At any rate, after taking a precursory glance at the official documentation, it seems fairly easy to understand for me having decent knowledge of Python but no knowledge of spring. Aside from the fact that it almost looks like Java code where the author forgot the typenames, semicolons, and curly braces. :-)

Which of these scripting languages is more appropriate for pen-testing? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
First of all, I want to avoid a flame-war on languages. The languages to choose from are Perl, Python and Ruby . I want to mention that I'm comfortable with all of them, but the problem is that I can't focus just on one.
If, for example, I see a cool Perl module, I have to try it out. If I see a nice Python app, I have to know how it's made. If I see a Ruby DSL or some Ruby voodoo, I'm hooked on Ruby for a while.
Right now I'm working as a Java developer, but plan on taking CEH in the near future. My question is: for tool writing and exploit development, which language do you find to be the most appropriate?
Again, I don't want to cause a flame-war or any trouble, I just want honest opinions from scripters that know what they're doing.
One more thing: maybe some of you will ask "Why settle on one language?". To answer this: I would like to choose only one language, in order to try to master it.

You probably want Ruby, because it's the native language for Metasploit, which is the de facto standard open source penetration testing framework. Ruby's going to give you:
Metasploit's framework, opcode and shellcode databases
Metasploit's Ruby lorcon bindings for raw 802.11 work.
Metasploit's KARMA bindings for 802.11 clientside redirection.
Libcurl and net/http for web tool writing.
EventMachine for web proxy and fuzzing work (or RFuzz, which extends the well-known Mongrel webserver).
Metasm for shellcode generation.
Distorm for x86 disassembly.
BinData for binary file format fuzzing.
Second place here goes to Python. There are more pentesting libraries available in Python than in Ruby (but not enough to offset Metasploit). Commercial tools tend to support Python as well --- if you're an Immunity CANVAS or CORE Impact customer, you want Python. Python gives you:
Twisted for network access.
PaiMei for program tracing and programmable debugging.
CANVAS and Impact support.
Dornseif's firewire libraries for remote debugging.
Ready integration with WinDbg for remote Windows kernel debugging (there's still no good answer in Ruby for kernel debugging, which is why I still occasionally use Python).
Peach Fuzzer and Sully for fuzzing.
SpikeProxy for web penetration testing (also, OWASP Pantera).
Unsurprisingly, a lot of web work uses Java tools. The de facto standard web pentest tool is Burp Suite, which is a Java swing app. Both Ruby and Python have Java variants you can use to get access to tools like that. Also, both Ruby and Python offer:
Direct integration with libpcap for raw packet work.
OpenSSL bindings for crypto.
IDA Pro extensions.
Mature (or at least reasonable) C foreign function interfaces for API access.
WxWindows for UI work, and decent web stacks for web UIs.
You're not going to go wrong with either language, though for mainstream pentest work, Metasploit probably edges out all the Python benefits, and at present, for x86 reversing work, Python's superior debugging interfaces edge out all the Ruby benefits.
Also: it's 2008. They're not "scripting languages". They're programming languages. ;)

[Disclaimer: I am primarily a Perl programmer, which may be colouring my judgement. However, I am not a particularly tribal one, and I think on this particular question my argument is reasonably objective.]
Perl was designed to blend seamlessly into the Unix landscape, and that is why it feels so alien to people with a mainly-OO background (particularly the Java school of OOP). For that reason, though, it’s incredibly widely installed on machines with any kind of Unixoid OS, and many vendor system utilities are written in it. Also for the same reason, servers that have neither Python nor Ruby installed are still likely to have Perl on them, again making it important to have some familiarity with. So if your CEH activity includes extensive activity on Unix, you will have to have some amount of familiarity with Perl anyway, and you might as well focus on it.
That said, it is largely a matter of preference. There is not much to differentiate the languages; their expressive power is virtually identical. Some things are a little easier in one of the languages, some a little easier in another.
In terms of libraries I do not know how Ruby and Python compare against each other – I do know that Perl has them beat by a margin. Then again, sometimes (particularly when you’re looking for libraries for common needs) the only effect of that is that you get deluged with choices. And if you are only looking to do things in some particular area which is well covered by libraries for Python or Ruby, the mass of other stuff on CPAN isn’t necessarily an advantage. In niche areas, however, it matters, and you never know what unforeseen need you will eventually have (err, by definition).
For one-liner use on the command line, Python is kind of a non-starter.
In terms of interactive interpreter environment, Perl… uhm… well, you can use the debugger, which is not that great, or you can install one from CPAN, but Perl doesn’t ship a good one itself.
So I think Perl does have a very slight edge for your needs in particular, but only just. If you pick Ruby you’ll probably not be much worse off at all. Python might inconvenience you a little more noticeably, but it too is hardly a bad choice.

I could make an argument for all three :-)
Perl has all of CPAN - giving you a huge advantage in pulling together functionality quickly. It also has a nice flexible testing infrastructure that means you can plug lots of different automated testing styles (including tests in other languages) in the same framework.
Ruby is a lovely language to learn - and lacks some of the cruft in Perl 5. If you're doing web based testing it also has the watir library - which is trez useful (see http://wtr.rubyforge.org/)
Python - nice language and (while it's not to my personal preference) some folk find the way its structured easier to get to grips with.
Any of them (and many others) would be a great language to learn.
Instead of looking at the language - I'd look at your working environment. It's always easier to learn stuff if you have other folk around who are doing similar stuff. If you current dev/testing folk are already focussed on one of the above - I'd go for that. If not, pick the one that would be most applicable/useful to your current working environment. Chat to the rest of your team and see what they think.

That depends on the implementation, if it will be distributed I would go with Java, seeing as you know that, because of its portability. If it is just for internal use, or will be used in semi-controlled environments, then go with whatever you are the most comfortable maintaining, and whichever has the best long-term outlook.
Now to just answer the question, I would go with Perl, but I'm a linux guy so I may be a bit biased in this.

If you plan on using Metasploit for pen-testing and exploit development I would recommend ruby as mentioned previously Metasploit is written in ruby and any exploit/module development you may wish to do will require ruby.
If you will be using Immunity CANVAS for pen testing then for the same reasons I would recommend Python as CANVAS is written in python. Also allot of fuzzing frameworks like Peach and Sulley are written in Python.
I would not recommend Perl as you will find very little tools/scripts/frameworks related to pen testing/fuzzing/exploits/... in Perl.
As your question is "tool writing and exploit development" I would recommend Ruby if you choose Metasploit or python if you choose CANVAS.
hope that helps :)

Speaking as a CEH, learn the CEH material first. This will expose you to a variety of tools and platforms used to mount various kinds of attacks. Once you understand your target well, look into the capabilities of the tools and platforms already available (the previously mentioned metasploit framework is very thorough and robust). How can they be extended to meet your needs? Once you know that, you can compare the capabilities of the languages.
I would also recommend taking a look at the tools available on the BackTrack distro.

All of them should be sufficient for that. Unless you need some library that is only available in one language, I'd let personal preference guide me.

If you're looking for a scripting language that will play well with Java, you might want to look at Groovy. It has the flexibility and power of Perl (closures, built in regexes, associative arrays on every corner) but you can access Java code from it thus you have access to a huge number of libraries, and in particular the rest of the system you're developing.

metasploit is a great framework for penetration testing. It's mainly written in Ruby, so if you know that language well, maybe you can hook in there. However, to use metasploit, you don't need to know any language at all.

If you are interested in CEH, I'd take a look at Grey Hat Python. It shows some stuff that is pretty interesting and related.
That being said, any language should be fine.

Well, what kind of exploits are you thinking about? If you want to write something that needs low level stuff (ptrace, raw sockets, etc.) then you'll need to learn C. But both Perl and Python can be used. The real question is which one suits your style more?
As for toolmaking, Perl has good string-processing abilities, is closer to the system, has good support, but IMHO it's very confusing. I prefer Python: it's a clean, easy to use, easy to learn language with good support (complete language/lib reference, 3rd party libs, etc.). And it's (strictly IMHO) cool.

I'm with tqbf. I've worked with Python and Ruby. Currently I'm working with JRuby. It has all the power of Ruby with access to the Java libraries so if there is something you absolutely need a low-level language to solve you can do so with a high-level language. So far I haven't needed to really use much Java as Ruby has had the ability to do everything I've needed as an API tester.