We Keep Coding

Azure DevOps pipeline microsoft hosted image with specific Python version

I want to make use of Python 3.4 x86 as my build target. Microsoft hosted agents don't have that as an option - 3.5 is the lowest version 3x branch.
Can I just install python during run process - still use Microsoft hosted agent but with python version I like?
The UsePythonVersion doesn't install it.

It seems that you are using Hosted agent vs2017-win2016 and we can see that the Python version installed by the agent does not contain Python 3.4
Check the task prerequisites:
A Microsoft-hosted agent with side-by-side versions of Python installed, or a self-hosted agent with Agent.ToolsDirectory configured (see FAQ).
This task will fail if no Python versions are found in Agent.ToolsDirectory. Available Python versions on Microsoft-hosted agents can be found here.
As a workaround, we should install the Python 3.4 via command line and then use the task Use Python version to specify the Python version. Check this doc for more details.
Update1
It seems that we cannot install 3.4 in the hosted agent.
TEST1 install python via power shell script:
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
Invoke-WebRequest -Uri "{URL}" -OutFile "$(Build.SourcesDirectory)/{python install file .exe}"
$(Build.SourcesDirectory)/{python install file .exe} /quiet InstallAllUsers=0 PrependPath=1 Include_test=0
But cannot found python 3.4 exe file in the download website.
TEST2 install the python via extension.
Extension: Python build tools
Task: Install Python on Windows
According to the json file, we can see that 3.5.2 is the lowest version.

Hmm. Exe isn’t there but msi’s are. They take same options as EXE I believe

If you are using a customized agent You need to add an environment for AGENT_TOOLSDIRECTORY and a python folder structure (https://go.microsoft.com/fwlink/?linkid=871498).
In my case, I did in docker file used to generate the agent.
ENV AGENT_TOOLSDIRECTORY="/AGENT_DEVOPS"
RUN mkdir /AGENT_DEVOPS
RUN mkdir /AGENT_DEVOPS/Python
# python 3.8
RUN mkdir /AGENT_DEVOPS/Python/3.8.0
RUN apt-get install python3.8-venv
RUN python3.8 -m venv /AGENT_DEVOPS/Python/3.8.0/x64
RUN touch /AGENT_DEVOPS/Python/3.8.0/x64.complete
# python 3.7
RUN mkdir /AGENT_DEVOPS/Python/3.7.5
RUN apt-get install python3.7-venv
RUN python3.7 -m venv /AGENT_DEVOPS/Python/3.7.5/x64
RUN touch /AGENT_DEVOPS/Python/3.7.5/x64.complete

Install Python of specific version system-wide with pyenv

I am researching possibility to upgrade to Python 3.6 in our project.
Right now we are using Python 3.5.2 from ppa:fkrull/deadsnakes on Ubuntu 14.04. The PPA doesn't have Python 3.6 yet and it's not clear when it will be available.
I don't want to install yet another PPA.
And I am trying to find a more general approach.
I found people suggesting to use pyenv which compiles Python from source, which sounds interesting, because I can upgrade Python any time without waiting until repo maintainer adds it. Also I can easily install other Python flavors like PyPy.
I am not ready to use pyenv as virtual environment yes, so I am wondering if it's possible to use it to compile and install Python globally so that I can just use it.

The documentation is a little confusing because there is no python-build binary added in PATH after installation.
python-build is a pyenv plugin (installed by default). Documentation and more info is here: https://github.com/pyenv/pyenv/tree/master/plugins/python-build.
How to install system-wide Python for all users: 1) Login as root and 2) install required Python version to /usr/local/python-X.Y.Z.
sudo ~/.pyenv/plugins/python-build/bin/python-build 3.6.1 /usr/local/python-3.6.1/
Now you can use this Python version as a normal user, for example you can create virtualenv for your project:
/usr/local/python-3.6.1/bin/python -m venv /var/www/my-app/.env/

https://github.com/yyuu/pyenv/wiki/Common-build-problems#installing-a-system-wide-python
Installing a system-wide Python
If you want to install a Python interpreter that's available to all
users and system scripts (no pyenv), use /usr/local/ as the install
path. For example:
sudo python-build 3.3.2 /usr/local/

I've contributed a package for python3.6 in deadsnakes for trusty / xenial :)
https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes/+packages?field.name_filter=python3.6&field.status_filter=published&field.series_filter=

By combining the hints from the other answers and reading through the documentation, I found a nice way to do exactly what you want that should work well in a CI system or in a Docker container or on a developer machine if they haven't already installed python3.x via Apt or Yum or Homebrew.
Assuming you have all the dependencies required to build your desired version of Python 3.x (anything above 3.4 requires some extra packages the pyenv-installer doesn't always warn you about), you can run the commands below to get a new system wide Python that should be executable by all users, which makes it easy to pass to virtualenv creations with python3.6 -m venv yourvenv.
curl https://pyenv.run | bash # or
wget -O - https://pyenv.run | bash
export PATH="$HOME/.pyenv/bin:$PATH"
$(pyenv which python-build) 3.6.10 /usr/local/
which python3.6
python3.6 --version
# If you get an error running the above commands, it probably means
# /usr/local/bin isn't in your PATH yet
# on Debian/Ubuntu and maybe others the /etc/environment or
# /etc/login.defs file puts it in the path when a user logs in
echo $PATH
export PATH="/usr/local/bin:$PATH"
python3.6 --version

Scapy installation fails due to invalid token

I have recently taken up learning networks, and I want to install scapy.
I have downloaded the latest version (2.2.0), and have two versions of python on my computer- 2.6.1 and 3.3.2. My OS is windows 7 64 bit.
After extracting scapy and navigating to the correct folder in the terminal, I was instructed to run "python setup.py install". I get the following error-
File "setup.py", line 35
os.chmod(fname,0755)
................................^
......................invalid
token
(dots for alignment)
How do I solve this problem?

Update: scapy-python3 is deprecated (2018) and will no longer be updated. scapy>=2.4.0 has merged python 3 compatibility.
The most up-to-date installation method is now
pip3 install scapy>=2.4.0
You may check the installation page in the documentation for other installation methods
Original answer:
Perhaps you are trying to install the package scapy for Python 2, but you need the one for Python 3.
pip install scapy
gave this error:
os.chmod(fname,0755)
^
SyntaxError: invalid token
while
pip3 install scapy-python3
did a proper install.
This error means the octal number is not recognized by Python 3, see PEP 3127:
octal literals must now be specified with a leading "0o" or "0O" instead of "0";

The following works for me on Python 3.5
pip3.5 install scapy-python3

Change os.chmod(fname,0755) to os.chmod(fname,0o755) and re-run

If pip installation is causing problem. You can download using wget and try to install.
$ cd /tmp
$ wget --trust-server-names scapy.net
$ unzip scapy-x.x.x.zip
$ cd scapy
$ sudo python setup.py install
Check here for all ways of installing scapy.

Scapy mainly used on uinx-liked OS, and can't install by pip. But they offered msi installer for windows:
http://www.secdev.org/projects/scapy/doc/installation.html
The error also occurs on Linux, but virtualenv saves me.Virtualenv is a really good solution of using different version python or librarys on one OS.
virtualenv -p $python_bin_path $virtualenv_directory_name
Creating a virtual env with python2 and python3:
virtualenv -p `which python` project_with_python2
virtualenv -p `which python3` project_with_python3
Then active the env, and install the requirements.
cd project_with_python2
source bin/activate
pip install scapy
pip install -r requirements.txt
And using deactivate to exit env.

How to use Python virtualenv

The scenario is: I am on Ubuntu 11 which comes with Python 2.7, I want to run Mozilla JetPack which supports Python 2.5/2.6 and Google App Engine which only supports Python 2.5.
Read that its not a good idea to remove Python 2.7 as Ubuntu maybe using it. So the correct way is to use virtualenv. But I am quite lost using it. I installed Python 2.5 in /usr/local/python25 following this guide
I tried
jiewmeng#JM:/usr/local/python25/bin$ ./python --version
Python 2.5.5
jiewmeng#JM:/usr/local/python25/bin$ ./python virtualenv /works/tmp/test
./python: can't open file 'virtualenv': [Errno 2] No such file or directory
then the below works but I will be using Python 2.7
jiewmeng#JM:/usr/local/python25/bin$ virtualenv /works/tmp/test
New python executable in /works/tmp/test/bin/python
Installing distribute.................................................................................................................................................................................done.
jiewmeng#JM:/usr/local/python25/bin$ cd /works/tmp/test/bin
jiewmeng#JM:/works/tmp/test/bin$ ls
activate activate_this.py easy_install easy_install-2.7 pip python
jiewmeng#JM:/works/tmp/test/bin$ ./python --version
Python 2.7.1+
Also, how do I then run Mozilla JetPack or Google App Engine with this version of Python? Sorry I am new to Python (and Linux/Ubuntu)

Outline:
First cd to /usr/local/python25/bin
Download setuptools for Python2.5 (setuptools-0.6c11-py2.5.egg)
Install it (sh setuptools-0.6c11-py2.5.egg).
Now install pip (easy_install pip).
Install virtualenv and virtualenvwrapper using pip (pip install v... etc.).
Configure WORKON_HOME for virtualenv wrapper to work (export WORKON_HOME = $HOME/.virtualenvs). You can use any other directory you want (not just $HOME/.virtualenvs). Just make sure to use the full path.
Now create a virtualenv (mkvirtualenv foobar).
Switch to the new virtualenv (workon foobar).
Now install GAE, JetPack and whatever you want using pip install blah
Why did your install not work?
Looks like you did not install virtualenv for Python2.5. Hence this will not work.
jiewmeng#JM:/usr/local/python25/bin$ ./python virtualenv /works/tmp/test
You can check by running ls command in that directory. I suspect you won't find virtualenv file there.
However this worked for you.
jiewmeng#JM:/usr/local/python25/bin$ virtualenv /works/tmp/test
Because it is using the virtualenv file for system default Python2.7. You can check which virtualenv and opening the virtualenv script. You'll see that the #! will point to system default python.
So you need to install the easy_install and pip for Python 2.5 before you can create virtualenv for Python 2.5. Just follow the steps outlined above.

You don't need to do anything fancy outside the virtualenv wrapper. Just use the --python=python2.5 flag (check out the man page for virtualenv form more). It does not matter what version you install it with, you just have to select the right executable for python in the virtual environment.
e.g. mkvirtualenv --python=python2.5 --distribute python25 if the python flag fails, either add a symlink (ln -s) to python25 in your $PATH or use the full path name on the python flag.
Also, default for multiple python installations is to have, for all 'altinstall' versions, a separate python and easy_install. So, for example: python2.5 ,easy_install-2.5 ,python2.6, easy_install-2.6 etc.

Unable to install Python without sudo access

I extracted, configured and used make for the installation package in my server.
However, I could not use make install. I get the error
[~/wepapps/python/Python-2.6.1]# make install
/usr/bin/install -c python /usr/local/bin/python2.6
/usr/bin/install: cannot create regular file `/usr/local/bin/python2.6': Permission denied
make: *** [altbininstall] Error 1
I run the folder with
chmod +x Python-2.6.1
I get still the same error.
How can I run make install without sudo access?

How can I install to a path under my home directory?
mkdir /home/masi/.local
cd Python-2.6.1
make clean
./configure --prefix=/home/masi/.local
make
make install
Then run using:
/home/masi/.local/bin/python
Similarly if you have scripts (eg. CGI) that require your own user version of Python you have to tell them explicitly:
#!/home/masi/.local/bin/python
instead of using the default system Python which “#!/usr/bin/env python” will choose.
You can alter your PATH setting to make just typing “python” from the console run that version, but it won't help for web apps being run under a different user.
If you compile something that links to Python (eg. mod_wsgi) you have to tell it where to find your Python or it will use the system one instead. This is often done something like:
./configure --prefix=/home/masi/.local --with-python=/home/masi/.local
For other setup.py-based extensions like MySQLdb you simply have to run the setup.py script with the correct version of Python:
/home/masi/.local/bin/python setup.py install

As of year 2020, pyenv is the best choice for installing Python without sudo permission, supposing the system has necessary build dependencies.
# Install pyenv
$ curl https://pyenv.run | bash
# Follow the instruction to modify ~/.bashrc
# Install the latest Python from source code
$ pyenv install 3.8.3
# Check installed Python versions
$ pyenv versions
# Switch Python version
$ pyenv global 3.8.3
# Check where Python is actually installed
$ pyenv prefix
/home/admin/.pyenv/versions/3.8.3
# Check the current Python version
$ python -V
Python 3.8.3

Extending bobince answer, there is an issue if you don't have the readline development package installed in your system, and you don't have root access.
When Python is compiled without readline, your arrow keys won't work in the interpreter. However, you can install the readline standalone package as follows: Adding Readline Functionality Without Recompiling Python
On the other hand, if you prefer to compile python using a local installation of readline, here's how.
Before doing as bobince was telling, compile and install readline. These are the steps to do so:
wget ftp://ftp.cwru.edu/pub/bash/readline-6.2.tar.gz
tar -zxvf readline-6.2.tar.gz
cd readline-6.2
./configure --with-prefix=$HOME/.local
make
make install
Then, add this line to your .bash_profile script:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.local/lib
Last, but not least, execute the following command
export LDFLAGS="-L$HOME/.local"
I hope this helps someone!

You can't; not to /usr, anyway. Only superusers can write to those directories. Try installing Python to a path under your home directory instead.