We Keep Coding

How we can automate python client application which is used an an interface for user to call APIs

We have made an python client which is used as an interface for user. some function is defined in the client which internally calls the APIs and give output to users.
My requirement is to automate the python client - functions and validate the output.
Please suggest tools to use.

There are several ways to do that:
You can write multiple tests for your application as the test cases which are responsible to call your functions and get the result and validate them. It calls the "feature test". To do that, you can use the python "unittest" library and call the tests periodically.
If you have a web application you can use "selenium" to make automatic test flows. (Also you can run it in a docker container virtually)
The other solution is to write another python application to call your functions or send requests everywhere you want to get the specific data and validate them. (It's the same with the two other solutions with a different implementation)

The most straightforward way is using Python for this, the simplest solution would be a library like pytest. More comprehensive option would be something like Robot framework
Given you have jmeter in your tags I assume that at some point you will want to make a performance test, however it might be easier to use Locust for this as it's pure Python load testing framework.
If you still want to use JMeter it's possible to call Python programs using OS Process Sampler

How (in what form) to share (deliver) a Python function?

The final outcome of my work should be a Python function that takes a JSON object as the only input and return another JSON object as output. To keep it more specific, I am a data scientist, and the function that I am speaking about, is derived from data and it delivers predictions (in other words, it is a machine learning model).
So, my question is how to deliver this function to the "tech team" that is going to incorporate it into a web-service.
At the moment I face few problems. First, the tech team does not necessarily work in Python environment. So, they cannot just "copy and paste" my function into their code. Second, I want to make sure that my function runs in the same environment as mine. For example, I can imagine that I use some library that the tech team does not have or they have a version that differ from the version that I use.
ADDED
As a possible solution I consider the following. I start a Python process that listen to a socket, accept incoming strings, transforms them into JSON, gives the JSON to the "published" function and returns the output JSON as a string. Does this solution have disadvantages? In other words, is it a good idea to "publish" a Python function as a background process listening to a socket?

You have the right idea with using a socket but there are tons of frameworks doing exactly what you want. Like hleggs, I suggest you checkout Flask to build a microservice. This will let the other team post JSON objects in an HTTP request to your flask application and receive JSON objects back. No knowledge of the underlying system or additional requirements required!
Here's a template for a flask app that replies and responds with JSON
from flask import Flask, request, jsonify
app = Flask(__name__)
#app.route('/', methods=['POST'])
def index():
json = request.json
return jsonify(your_function(json))
if __name__=='__main__':
app.run(host='0.0.0.0', port=5000)
Edit: embeded my code directly as per Peter Britain's advice

My understanding of your question boils down to:
How can I share a Python library with the rest of my team, that may not be using Python otherwise?
And how can I make sure my code and its dependencies are what the receiving team will run?
And that the receiving team can install things easily mostly anywhere?
This is a simple question with no straightforward answer... as you just mentioned that this may be integrated in some webservice, but you do not know the actual platform for this service.
You also ask:
As a possible solution I consider the following. I start a Python process that listen to a socket, accept incoming strings, transforms them into JSON, gives the JSON to the "published" function and returns the output JSON as a string. Does this solution have disadvantages? In other words, is it a good idea to "publish" a Python function as a background process listening to a socket?
In the most simple case and for starting I would say no in general. Starting network servers such as an HTTP server (which is built-in Python) is super easy. But a service (even if qualified as "micro") means infrastructure, means security, etc.
What if the port you expect is not available on the deployment machine? - What happens when you restart that machine?
How will your server start or restart when there is a failure?
Would you need also to eventually provide an upstart or systemd service (on Linux)?
Will your simple socket or web server support multiple concurrent requests?
is there a security risk to expose a socket?
Etc, etc. When deployed, my experience with "simple" socket servers is that they end up being not so simple after all.
In most cases, it will be simpler to avoid redistributing a socket service at first. And the proposed approach here could be used to package a whole service at a later stage in a simpler way if you want.
What I suggest instead is a simple command line interface nicely packaged for installation.
The minimal set of things to consider would be:
provide a portable mechanism to call your function on many OSes
ensure that you package your function such that it can be installed with all the correct dependencies
make it easy to install and of course provide some doc!
Step 1. The simplest common denominator would be to provide a command line interface that accepts the path to a JSON file and spits JSON on the stdout.
This would run on Linux, Mac and Windows.
The instructions here should work on Linux or Mac and would need a slight adjustment for Windows (only for the configure.sh script further down)
A minimal Python script could be:
#!/usr/bin/env python
"""
Simple wrapper for calling a function accepting JSON and returning JSON.
Save to predictor.py and use this way::
python predictor.py sample.json
[
"a",
"b",
4
]
"""
from __future__ import absolute_import, print_function
import json
import sys
def predict(json_input):
"""
Return predictions as a JSON string based on the provided `json_input` JSON
string data.
"""
# this will error out immediately if the JSON is not valid
validated = json.loads(json_input)
# <....> your code there
with_predictions = validated
# return a pretty-printed JSON string
return json.dumps(with_predictions, indent=2)
def main():
"""
Print the JSON string results of a prediction, loading an input JSON file from a
file path provided as a command line argument.
"""
args = sys.argv[1:]
json_input = args[0]
with open(json_input) as inp:
print(predict(inp.read()))
if __name__ == '__main__':
main()
You can process eventually large inputs by passing the path to a JSON file.
Step 2. Package your function. In Python this is achieved by creating a setup.py script. This takes care of installing any dependent code from Pypi too. This will ensure that the version of libraries you depend on are the ones you expect. Here I added nltk as an example for a dependency. Add yours: this could be scikit-learn, pandas, numpy, etc. This setup.py also creates automatically a bin/predict script which will be your main command line interface:
#!/usr/bin/env python
# -*- encoding: utf-8 -*-
from __future__ import absolute_import, print_function
from setuptools import setup
from setuptools import find_packages
setup(
name='predictor',
version='1.0.0',
license='public domain',
description='Predict your life with JSON.',
packages=find_packages(),
# add all your direct requirements here
install_requires=['nltk >= 3.2, < 4.0'],
# add all your command line entry points here
entry_points={'console_scripts': ['predict = prediction.predictor:main']}
)
In addition as is common for Python and to make the setup code simpler I created a "Python package" directory moving the predictor inside this directory.
Step 3. You now want to package things such that they are easy to install. A simple configure.sh script does the job. It installs virtualenv, pip and setuptools, then creates a virtualenv in the same directory as your project and then installs your prediction tool in there (pip install . is essentially the same as python setup.py install). With this script you ensure that the code that will be run is the code you want to be run with the correct dependencies. Furthermore, you ensure that this is an isolated installation with minimal dependencies and impact on the target system. This is tested with Python 2 but should work quite likely on Python 3 too.
#!/bin/bash
#
# configure and installs predictor
#
ARCHIVE=15.0.3.tar.gz
mkdir -p tmp/
wget -O tmp/venv.tgz https://github.com/pypa/virtualenv/archive/$ARCHIVE
tar --strip-components=1 -xf tmp/venv.tgz -C tmp
/usr/bin/python tmp/virtualenv.py .
. bin/activate
pip install .
echo ""
echo "Predictor is now configured: run it with:"
echo " bin/predict <path to JSON file>"
At the end you have a fully configured, isolated and easy to install piece of code with a simple highly portable command line interface.
You can see it all in this small repo: https://github.com/pombredanne/predictor
You just clone or fetch a zip or tarball of the repo, then go through the README and you are in business.
Note that for a more engaged way for more complex applications including vendoring the dependencies for easy install and not depend on the network you can check this https://github.com/nexB/scancode-toolkit I maintain too.
And if you really want to expose a web service, you could reuse this approach and package that with a simple web server (like the one built-in in the Python standard lib or bottle or flask or gunicorn) and provide configure.sh to install it all and generate the command line to launch it.

Your task is (in generality) about productionizing a machine learning model, where the consumer of the model may not be working in the same environment as the one which was used to develop the model. I've been trying to tackle this problem since past few years. The problem is faced by many companies and it is aggravated due to skill set, objectives as well as environment (languages, run time) mismatch between data scientists and developers. From my experience, following solutions/options are available, each with its unique advantages and downsides.
Option 1 : Build the prediction part of your model as a standalone web service using any lightweight tool in Python (for example, Flask). You should try to decouple the model development/training and prediction part as much as possible. The model that you have developed, must be serialized to some form so that the web server can use it.
How frequently is your machine learning model updated? If it is not done very frequently, the serialized model file (example: Python pickle file) can be saved to a common location accessible to the web server (say s3), loaded in memory. The standalone web server should offer APIs for prediction.
Please note that exposing a single model prediction using Flask would be simple. But scaling this web server if needed, configuring it with right set of libraries, authentication of incoming requests are all non-trivial tasks. You should choose this route only if you have dev teams ready to help with these.
If the model gets updated frequently, versioning your model file would be a good option. So in fact, you can piggyback on top of any version control system by checking in the whole model file if it is not too large. The web server can de-serialize (pickle.load) this file at startup/update and convert to a Python object on which you can call prediction methods.
Option 2 : use predictive modeling markup language. PMML was developed specifically for this purpose: predictive modeling data interchange format independent of environment. So data scientist can develop model, export it to a PMML file. The web server used for prediction can then consume the PMML file for doing predictions. You should definitely check the open scoring project which allows you to expose machine learning models via REST APIs for deploying models and making predictions.
Pros: PMML is standardized format, open scoring is a mature project with good development history.
Cons: PMML may not support all models. Open scoring is primarily useful if your tech team's choice of development platform is JVM. Exporting machine learning models from Python is not straightforward. But R has good support for exporting models as PMML files.
Option 3 : There are some vendors offering dedicated solutions for this problem. You will have to evaluate cost of licensing, cost of hardware as well as stability of the offerings for taking this route.
Whichever option you choose, please consider the long term costs of supporting that option. If your work is in a proof of concept stage, Python flask based web server + pickled model files will be the best route. Hope this answer helps you!

As already suggested in other answers the best option would be creating a simple web service. Besides Flask you may want to try bottle which is very thin one-file web framework. Your service may looks as simple as:
from bottle import route, run, request
#route('/')
def index():
return my_function(request.json)
run(host='0.0.0.0', port=8080)
In order to keep environments the same check virtualenv to make isolated environment for avoiding conflicts with already installed packages and pip to install exact version of packages into virtual environment.

I guess you have 3 possibilities :
convert python function to javascript function:
Assuming the "tech-team" use Javascript for web-service, you may try to convert your python function directly to a Javascript function (which will be really easy to integrate on web page) using empythoned (based on emscripten)
The bad point of this method is that each time you need update/upgrade your python function, you need also to convert to Javascript again, then check & validate that the function continue to work.
simple API server + JQuery
If the conversion method is impossible, I am agree with #justin-bell, you may use FLASK
getting JSON as input > JSON to your function parameter > run python function > convert function result to JSON > serve the JSON result
Assuming you choose the FLASK solution, "tech-team" will only need to send an async. GET/POST request containing all the arguments as JSON obj, when they need to get some result from your python function.
websocket server + socket.io
You can also use take a look on Websocket to dispatch to the webservice (look at flask + websocket for your side & socket.io for webservice side.)
=> websocket is really usefull when you need to push/receive data with low cost and latency to (or from) a lot of users (Not sure that websocket will be the best fit to your need)
Regards

How to interface with opentsdb from Python

I'd like to interface with an opentsdb data store with Python. I only see a java client library for it. How would I go about this?

Unless you want a standalone client (in which case the Twisted Python OpenTSDB Client looks great), the easiest way is to run tcollector, and simply drop your Python script under /usr/local/tcollector/collector/0 – your script is expected to never return and print one data point per line in that format: metric timestamp value tag1=value1 tag2=value2 ....
tcollector takes care of connecting to OpenTSDB, pushing your data points out, etc. So you can focus on collecting the data you want to collect and write your data collection script in Python or any other scripting language you may like.

You can use Python request module and OpenTSDB HTTP API, too.

Give that library a try.
Twisted Python OpenTSDB Client
http://code.google.com/p/totsdb/source/browse/tostdb.py

Python: How to use platform.win32_ver() on a remote machine?

So of course I'm new to Python and to programming in general...
I am trying to get OS version information from the network. For now I only care about the windows machines.
using PyWin32 I can get some basic information, but it's not very reliable. This is an example of what I am doing right now: win32net.NetWkstaGetInfo(myip, 100)
However, it appears as though this would provide me with more appropriate information: platform.win32_ver()
I have no idea how get the info from a remote machine using this. I need to specify an IP or a range of IP's... I intend on using Google's ipaddr to get a list of network ranges to scan. I will eventually need to scan a large network for this info.
Can someone provide an example?

A good way is to use WMI. The following links from Microsoft contain enough information to write code for your purposes:
Connecting to WMI on a Remote Computer
WMI Tasks: Operating Systems
The missing piece is how to do this in Python. For that, consult Tim Golden's site:
WMI for Python
WMI Cookbook
By the way, if you're OK with using a command line program and parsing the output, then I would suggest the PsTools available freely. In particular, psinfo can do what you want.

I had to use remote registry...
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion
ProductName, EditionID, CurrentVersion, CurrentBuild

SOAP 1.2 python client

I am looking for a python SOAP 1.2 client but it seems that it does not exist . All of the existing clients are either not maintainted or only compatible with SOAP 1.1:
suds
SOAPpy
ZSI

Even though this question has an accepted answer, there's a few notes I'd like regarding suds.
I'm currently writing some code for interfacing with .tel community hosting for work and I needed a Python SOAP library, and suds was pretty much ideal except for its lack of support for SOAP 1.2.
I managed to hack around the problem as for my purposes, SOAP 1.1 and SOAP 1.2 share enough in common that I was able to simply patch suds to use the SOAP 1.2 envelope namespace. I outlined what I did in this gist: https://gist.github.com/858851
As it's worth reproducing here, here's the code:
from suds.client import Client
from suds.bindings import binding
import logging
USERNAME = 'username'
PASSWORD = 'password'
# Just for debugging purposes.
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.client').setLevel(logging.DEBUG)
# Telnic's SOAP server expects a SOAP 1.2 envelope, not a SOAP 1.1 envelope
# and will complain if this hack isn't done.
binding.envns = ('SOAP-ENV', 'http://www.w3.org/2003/05/soap-envelope')
client = Client('client.wsdl',
username=USERNAME,
password=PASSWORD,
headers={'Content-Type': 'application/soap+xml'})
# This will now work just fine.
client.service.someRandomMethod()
If I've time, I'm planning on submitting a patch to suds to allow the version of SOAP to be used to be specified, and to add enough missing functionality to make it useful.

The zeep library supports both SOAP 1.1 and 1.2 as long as the service's WSDL properly indicates it.
WSF/Python is supporting SOAP 1.2.
INTRODUCTION
WSF/Python is the Python language extension to WSO2 WSF/C
[http://www.wso2.org/projects/wsf/c].
This version enables you to
consume/provide Web Services both with
REST and SOAP.
Support for REST
Support for SOAP 1.1
Support for SOAP 1.2
For downloading, you don't have to register. Just click "submit" at the very bottom.
Samples can be found within the downloaded archive, eg:
LOG_DIR = '/tmp/'
LOG_LEVEL = 4
WSFC_HOME = '/opt/wso2/wsf_c'
END_POINT = 'http://localhost:9090/axis2/services/echo/echoString'
if __name__ == '__main__':
message = """
<ns1:echoString xmlns:ns1="http://ws.apache.org/axis2/services/echo">
<text>Hello World!</text>
</ns1:echoString>
"""
try:
client = wso2.wsf.WSClient({
'to':END_POINT,
'WSF_LOG_DIR':LOG_DIR,
'WSF_LOG_LEVEL':LOG_LEVEL,
'WSFC_HOME':WSFC_HOME,
})
print 'Sending: ' + message
response = client.request(message)
if response is not None:
print 'Respose: ' + response + '\n'
else:
print 'Error occurred!'
except wso2.wsf.WSFault, e:
print 'Exception occurred:'
print e

If you are really wanting to use SOAP 1.2 even though it is not used as a standard as yet, I reckon I can post an answer that requires some work (all for the greater good :)).
I recommend that you use gSOAP:
gSOAP - a easy-use, cross-platform
toolkit for C/C++ lovers to develop
XML-based Web services and XML
parsers. Although it is well-known as
a Web service development toolkit and
has been proved its good performance,
it can also be used to create
high-performance XML parsers,
serializers and deserializers from XML
schemas or C/C++ structs/classes. My
experimental results demonstrate that
the XML parsers generated using gSOAP
toolkit run several times faster than
xerces-c parsers in either DOM or SAX
mode.
Now, I wish it were that easy. Due to gSOAP being a C++ library, you are going to have to wrap it to be able to use it in Python.
One way of wrapping the library is to use a tool by the name of SWIG (Simplified Wrapper and Interface Generator). This tool automatically wraps C/C++ libraries for use in high level languages, for example (you guessed it) Python.
I also recommend you read this PDF file (from page 14) on implementing gSOAP with C++. It is very helpful.
Using this solution, you can utilize a well looked after library, SOAP 1.2 and a very nice performance ratio. I think you will be quite happy with the results.

I have had the very similar problem some years ago and I have solved it by using Jython.
If there is no existed implementation of SOAP 1.2 for Python, you may be interested in Jython, which seamlessly integrates you with the Java platform. It means, you can use any of the existed SOAP 1.2 Java classes and just import it into your Jython program. Your Jython program is just your Python program, but you can import Java classes.
Jython itself includes almost all of the modules in the standard Python programming language distribution, but be sure that your program does not use any special non-standard Python library.
Example:
say, you have Jython installed (it is free and Open Source) and your Python program is called myprog.py and you want use Java class CLASSNAME:
1) import required Java class inserting import CLASSNAME into your myprog.py
2) run jython myprog.py