We Keep Coding

Trying to run python code on jenkins in ubuntu

all.
I recently started working with Jenkins, in an attempt to replace cronjob with Jenkins pipeline. I have really a bit knowledge of programming jargon. I learned what I learned from questions on stackoverflow. So, if you guys need any more info, I would really appreciate if you use plain English.
So, I installed the lastest version of Jenkins and suggested plugins plus all the plugins that I could find useful to python running.
Afterwards, I searched stackoverflow and other websites to make this work, but all I could do was
#!/usr/bin/env python
from __future__ import print_function
print('Hello World')
And it succeeded.
Currently, Jenkins is running on Ubuntu 16.04, and I am using anaconda3's python (~/anaconda3/bin/python).
When I tried to run a bit more complicated python code (by that I mean import pandas), it gives me import error.
What I have tried so far is
execute python script build: import pandas - import error
execute shell build: import pandas (import pandas added to the code that worked above)
python builder build: import pandas - invalid interpreter error
pipeline job: sh python /path_to_python_file/*.py - import error
All gave errors. Since 'hello world' works, I believe that using anaconda3's python is not an issue. Also, it imported print_function just fine, so I want to know what I should do from here. Change workspace setting? workdirectory setting? code changes?
Thanks.

Since 'hello world' works, I believe that using anaconda3's python is not an issue.
Your assumption is wrong.
There are multiple ways of solving the issue but they all come down to using the correct python interpreter with installed pandas. Usually in ubuntu you'll have at least two interpreters. One for python 2 and one for python 3 and you'll use them in shell by calling either python pth/to/myScript.py or python3 pth/to/myScript.py. python and python3 are in this case just a sort of labels which point to the correct executables, using environmental variable PATH.
By installing anaconda3 you are adding one more interpreter with pandas and plenty of other preinstalled packages. If you want to use it, you need to tell somehow your shell or Jenkins about it. If import pandas gives you an error then you're probably using a different interpreter or a different python environment (but this is out of scope here).
Coming back to your script
Following this stack overflow answer, you'll see that all the line #!/usr/bin/env python does, is to make sure that you're using the first python interpreter on your Ubuntu's environment path. Which almost for sure isn't the one you installed with anaconda3. Most likely it will be the default python 2 distributed with ubuntu. If you want to make sure which interpreter exactly is running your script, instead of 'Hello World' put inside:
#!/usr/bin/env python
import sys
print(sys.executable) # this line will give you the exact path to the interpreter
print(sys.version) # this one will give you the version
Ok, so what to do?
Well, run your script using the correct interpreter. Remove #!/usr/bin/env python from your file and if you have a pipeline, add there:
sh "/home/yourname/anaconda3/bin/python /path_to_python_file/myFile.py"
It will most likely solve the issue. It's also quite flexible in the sense that if you ever want to use this python file on a different machine, you won't have your username hardcoded inside.

Error in calling a Python script from a Basic function

I am new in Python and Basic. I am trying to replicate the IMPORTHTML function from Google Sheets in LibreOffice (LO) Calc. In a nutshell, I want to create a GetHtmTable( Url, Table Index) Basic function in Calc which will call a Python script to do the heavy work.
So based on Villeroy's great example, I implemented in LO 5.1.6.2. the Basic SOUNDEX function which calls the Python script sheetFunctions.py to get familiar with the process. My environment is Linux Mint 18, I use Python 3, I imported all kind of libraries such as Uno, PIP etc.. I use PycharmProjects as a Python editor.
I see clearly under the LO Calc menu tools->macro->organize macros->python the sheetFunctions.py Python script, which indeed is in the folder /usr/lib/libreoffice/share/Scripts/python.
Whenever, I run the SOUNDEX Basic function I see the following error message:
BASIC runtime error. An exception occurred Type:
com.sun.star.script.provider.ScriptFrameworkErrorException Message:
: an
error occurred during file opening
/usr/lib/libreoffice/program/pythonscript.py:429 in function
getModuleByUrl() [lastRead = self.sfa.getDateTimeModified( url )]
/usr/lib/libreoffice/program/pythonscript.py:993 in function
getScript() [mod = self.provCtx.getModuleByUrl( fileUri )]
I tried to debug the SOUNDEX basic function and found out the blocking point is when the program runs getScript("vnd.sun.star.script:sheetFunctions.py$soundex?language=Python&location=user").
I've been trying for days now to overcome this error, unsuccessfully I must confess.
I wonder if I need to bring some extra extensions in the Basic environment or a missing add-in in the Linux/Python one?
I changed the location=user by location=document and got stuck again. I added recently libreoffice-script-provider-python thanks to the command sudo apt-get install libreoffice-script-provider-python but this did not help. I also embedded in the Calc document the Python script but same this did not solve the issue.

The location name does not match. The standard place for self-written scripts is under the user directory. This is location=user, for example ~/.config/libreoffice/4/user/Scripts/python.
Then there is location=share, which refers to the path in your question. These parameters are described under Python Script in the URI Specification.
See also my answer to this question. Be sure to try the APSO extension if you haven't yet. Especially, APSO helps when using location=document, because embedding requires several steps including editing manifest.xml.

In Shiny, Python Virtual environment PERMISSION DENIED (Error 126)

We are building a User Interface APP (predicting a continuous variable through a machine learning model) through R Shiny.
Since we built the machine learning model in Python3 sklearn module, we hope that we could write python codes in R Shiny to call that model and corresponding functions.
We used R-package "reticulate" to create virtual python environment where it would save python packages, and through which we could call python3 functions.
We created the virtual environment using the following line of code (the function in R package "reticulate")
use_virtualenv("env", required = TRUE)
Where we indeed have the following directory "env/bin" in which there are python and python3 to execute.
The Shiny APP worked perfectly locally. HOWEVER, when we made attempts to publish, it gave the following error (please see picture) (after the APP was successfully deployed and on shinyapps.io, it said the APP was running).
The issue was "Error 126", which denied the permission for our APP to access the virtual environment. This issue had no previous (similar) case on Stackoverflow, and therefore we spent a long time to debug (issue not resolved).
If anyone knows how to solve this problem, would it be possible for you to kindly mark your solution tips below? (We hope your solution would not modify our basic layout, i.e. "calling python-made model in Shiny and publish through Shiny") We really appreciate your efforts to help us out!
Thank you so much!

Could you share the code where actual call to python script is being made? is it a python module function that you are calling from Rshiny? what does the python module/function do and return? I have used reticulate inside shiny to call Python scripts and it works fine. Didn't require to set the environment. Just provide the source to python script and call it like any other R function.

If you're trying to deploy to shinyapps.io, you may need to set the RETICULATE_PYTHON env variable so that reticulate uses the correct version of Python when running your app:
VIRTUALENV_NAME = 'env'
Sys.setenv(RETICULATE_PYTHON = paste0('/home/shiny/.virtualenvs/',
VIRTUALENV_NAME,
'/bin/python'))
Full example here demonstrates one method for configuring a Shiny + reticulate app so that it can easily run both locally and on shinyapps.io.

Communications between Blender and Tensorflow

I have a simulation inside Blender that I would like to control externally using the TensorFlow library. The complete process would go something like this:
while True:
state = observation_from_blender()
action = find_action_using_tensorflow_neural_net(state)
take_action_inside_blender(action)
I don't have much experience with the threading or subprocess modules and so am unsure as to how I should go about actually building something like this.

Rather than mess around with Tensorflow connections and APIs, I'd suggest you take a look at the Open AI Universe Starter Agent[1]. The advantage here is that as long as you have a VNC session open, you can connect a TF based system to do reinforcement learning on your actions.
Once you have a model constructed via this, you can focus on actually building a lower level API system for the two things to talk to each other.
[1] https://github.com/openai/universe-starter-agent

Thanks for your response. Unfortunately, trying to get Universe working with my current setup was a bit of a pain. I'm also on a fairly tight deadline so I just needed something that worked straight away.
I did find a somewhat DIY solution that worked well for my situation using the pickle module. I don't really know how to convert this approach into proper pseudocode, so here is the general outline:
Process 1 - TensorFlow:
load up TF graph
wait until pickled state arrives
while not terminated:
Process 2 - Blender:
run simulation until next action required
pickle state
wait until pickled action arrives
Process 1 - TensorFlow:
unpickle state
feedforward state through neural net, find action
pickle action
wait until next pickled state arrives
Process 2 - Blender:
unpickle action
take action
This approach worked well for me, but I imagine there are more elegant low level solutions. I'd be curious to hear of any other possible approaches that achieve the same goal.

I did this to install tensorflow with the python version that comes bundled with blender.
Blender version: 2.78
Python version: 3.5.2
First of all you need to install pip for blender's python. For that I followed instructions mentioned in this link: https://blender.stackexchange.com/questions/56011/how-to-use-pip-with-blenders-bundled-python. (What I did was drag the python3.5m icon into the terminal and then further type the command '~/get-pip.py').
Once you have pip installed you are all set up to install 3rd party modules and use them with blender.
Navigate to the bin folder insider '/home/path to blender/2.78/' directory. To install tensorflow, drag the python3.5m icon into terminal, then drag pip3 icon into terminal and give the command install tensorflow.
I got an error mentioning module lib2to3 not found. Even installing the module didn't help as I got the message that no such module exists. Fortunately, I have python 3.5.2 installed on my machine. So navigate to /usr/lib/python3.5 and copy the lib2to3 folder from there to /home/user/blender/2.78/python/lib/python3.5 and then again run the installation command for tensorflow. The installation should complete without any errors. Make sure that you test the installation by importing tensorflow.

pexpect: How to interact() over file object using newer version of pexpect library

pexpect has gone through big changes since the version provided by Ubuntu 15.04 (3.2). When installing newest version of pexpect using pip3, this minimal program that previously successfully gave terminal emulation over serial console does not work any more:
#!/usr/bin/python3
import serial
import pexpect.fdpexpect
ser = serial.Serial("/dev/ttyS0", baudrate=115200)
spawn = pexpect.fdpexpect.fdspawn(ser)
spawn.interact()
The newer API is missing interact() method in class pexpect.fdpexpect.fdspawn which used to be there.
Question: how is the newer version of pexpect (currently 4.2.1) supposed to be used to provide free manual interaction with file object (serial port in this case)?
Alternatively, question/work-around: I recognize I am using a bit heavy machinery for such a simple use case, any suggestions for other python libraries that can do the same as earlier version of pexpect could?
Code reading: Examples use pexpect.spawn(command_str) to get a spawn object which has interact() method; actually this pexpect.spawn() is the same as directly creating a pexpect.pty_spawn.spawn object which has this method. On the other hand, pexpect.fdpexpect.fdspawn() will construct a pexpect.fdpexpect.fdspawn class which is missing interact() method. Both of these spawn classes derive from pexpect.spawnbase.SpawnBase class. Based on my quick reading, this looks like regression that resulted from refactoring on the way to version 4.x.

Browsing the pexpect issues, found #351, #356, and the newly submitted #377. By my quick reading, it seems this is a regression that was brought by uncompleted refactoring along the way towards new major release version 4. There are three possible avenues:
Make sure to install older version:
$ pip3 install "pexpect<3"
(or make sure that the version installed by other system is 3.x).
Fix pexpect by yourself.
Use some other python library.
These avenues were all more or less hinted by github user #takluyver, apparently maintainer of pexpect in comment to issue #351:
#jquast any thoughts on what to do for this (and #356)? I feel bad that we've broken things people were doing with fdspawn, but I really don't want to make it inherit from the pty spawn class, and now that it works on Windows, I think it's important to preserve that too.
Maybe we should just say:
If you have legacy code that broke with Pexpect 4, downgrade to Pexpect 3.x (pip install pexpect<4)
If you're writing new code, use streamexpect instead of pexpect.