My Django site is hosted on Azure.
It allows for users to upload photos. I need a way for the system to resize, and possibly rotate photos.
Seems simple, and I tried to use the Pillow library but while it works locally it will not deploy to Azure for a number of reasons. I can be specific if needed but this is well documented like here.
I even tried buiding a wheel of Pillow and deploying that but Azure refuses to load it saying it is the wrong platform (even though I matched the Python 2.7 version - and 32 bit). I tried to upload 64 bit versions as well, and nothing works. So at this point I just want to leave Pillow behind me and ask for another way to achieve this in Python without Pillow. Is there any other way to do this?
Notes of things I tried:
1) Installing Pillow the normal way gives this familiar error message:
ValueError: zlib is required unless explicitly disabled using --disable-zlib, aborting
2) I then created a wheel by doing: pip wheel Pillow --wheel-dir=requirements
This however yields the following error in the pip.log:
Pillow-3.4.2-cp27-cp27m-win32.whl is not a supported wheel on this platform.
Pillow-4.1.1-cp27-cp27m-win32.whl is not a supported wheel on this platform.
I am certain that I'm runing Python 2.7 on a 32bit platform so not sure why its complaining.
After days wasted, I've discovered the reason why Pillow isn't installing. It's not because the wheel is incompatible to the platform, but rather that pip is too old.
Azure is using pip version 1.5.6 at the moment - shame on them. This version doesn't recognise wheels.
Here is how I fixed this:
Goto the Kudu DebugConsole:
https://[site_name].scm.azurewebsites.net/DebugConsole
Activate your VirtualEnv:
env\Scripts\activate
Note that if you run pip --version how old that version number is.
Now upgrade this by running:
python -m pip install -U pip
Note that you cannot upgrade the default pip in D:\Python27 as you don't have access to it but you can upgrade your local pip inside fo the virtual environment.
Now run pip --version to ensure you are running the latest version (i.e. >=9.0.1).
Now inside of requirements.txt you can tell pip to look for wheels in specific folders by adding a line at the top such as:
--find-links requirements (which means it will search the requirements folder).
Here is how you create the Pillow wheel. You can run this locally or on the Kudu Console. If you run it locally ensure your python version matches what you use on Azure (2.7 or 3.X) and by default make sure you use a 32bit version.
pip install wheel (Only if you don't have wheel installed)
pip wheel Pillow --wheel-dir=requirements
This will copy two files into your requirements folder: Pillow-X.whl and olefile-X.whl. Ensure these are added to your source control if you are deploying via git push. Push these to the server.
Now in the Kudu DebugConsole you can test the .whl files are there (after deploying) and test the installing by running:
pip install --no-index -r requirements.txt
This should now work and install Pillow!
When deploying pay close attention to if it says Found compatible virtual environment. or Creating python 2-7 virtual environment.. The former is what you want. But if you see the latter it means that the deploy has blasted your env folder and reset you back to pip 1.5.6. I don't know why it does this sometimes, but try to make as few changes to the env folder as possible after deploying (i.e. just upgrade pip and thats it) to avoid this.
I can't help you much with installing Pillow on Azure platform.
But my days of using manually resizing and other stuff is long gone.
I have been using thumbor https://thumbor.org/ for quite some time.
Just setup a secured instance of the same and use it resize, crop and manage your images dynamically.
Hope it helps
There is the other SO thread Microsoft Azure Django Python setup error Pillow, which has the similar issue about installing Pillow on Azure. I think my answer for that is helpful for resolving your issue. Any concern for my solution, please feel free to let me know.
Related
I'm trying to install tensorflow-gpu==1.7.0 in the docker image frolvlad/alpine-python2, using the command pip install tensorflow-gpu==1.7.0 but it seems like pip can't find it, I have tried with pip install tensorflow to install the latest but still no result.
i should mention that it works fine for pandas and numpy.
I just wanna know if it has something to do the deprecation message, and is there anyway to solve this.
Have a look for tensorflow-gpu==1.7.0 in pypi which you are looking for here, you could see the package meets the python2.7 requirement is tensorflow_gpu-1.7.0-cp27-cp27mu-manylinux1_x86_64.whl.
Unfortunately, the name cp27 means CPython, here I guess it's restricted to glibc, while alpine is using musl libc, if you download this wheel to your alpine container and have a install it will reports next error:
ERROR: tensorflow_gpu-1.7.0-cp27-cp27mu-manylinux1_x86_64.whl is not a supported wheel on this platform.
So, if you are not limited to use alpine, you could use debian based container, e.g. python:2, then you could successfully find the package just like you did in your local host machine.
# pip install tensorflow-gpu==1.7.0
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting tensorflow-gpu==1.7.0
Downloading tensorflow_gpu-1.7.0-cp27-cp27mu-manylinux1_x86_64.whl (256.2 MB)
If stick on alpine, I guess you may need to build from source code by yourselves, and may results in a lots effort to conquer issues, may refers to this to have a luck.
I am looking to download a Python library (specifically this one) from GitHub.
I had already downloaded it using pip install espnff but it appears changes have been made to it and the only way to get the updated version is through GitHub. I should also mention that I use Python with the Anaconda distribution, if that affects anything.
How do I download and update what I already have?
First, you should make sure that pip actually uses you anaconda python distribution, and not e.g. the one that comes as default on your OS. You can use which pip to do that.
After that, it is as easy as
pip install espnff --upgrade
If the latest changes have not yet been made available on pip, you could also try to install it manually from source. Taken from the repository you linked:
git clone https://github.com/rbarton65/espnff
cd espnff
python setup.py install
To make sure that you're installing the latest version available, you should use git pull to fetch and merge the latest changes before installing.
On some occasions, you might also have to delete the existing build directory first, or use
python setup.py build --force
python setup.py install
I'm on Ubuntu Server 14.04.
I'm trying to use pypy 3.2 to run a django application.
But whenever I try pypy3 manage.py runserver 0:8000, it says:
(fields.E210) Cannot use ImageField because Pillow is not installed.
HINT: Get Pillow at https://pypi.python.org/pypi/Pillow or run command "pip install Pillow".
But if I try to install Pillow:
pypy3 -m pip install Pillow
It says:
Requirement already up-to-date: Pillow in /usr/local/lib/python3.4/dist-packages/Pillow-2.9.0-py3.4-linux-x86_64.egg
Then I downloaded the source and tried:
pypy3 setup.py install
And it says:
Attribute error install_layout
I don't know what to do!!
_<
You should create a proper virtualenv with pypy3 as your interpreter t run your django install - inside the virtualenv, pip should just work for both django and Pillow - without strange maneuvers like pypy3 -m pip which are subject to breakrage like teh one you've got.
To create a virtualenv inside which the default Python interpreter is pypy3 just do virtualenv -p pypy3 myprojectdir - and activate it with source myprojectdir/bin/activate - for the lazy types there is virtualenv wrapper which creates even more shortcuts. From there on, pip install anything will install thins in this directory, without messing (or checking) your system packages from any other Python.
That is the way to have a sane environment- however it seens you are just trying to be beyond the edge for what is available now - there are mentions on the internet of Pillow being compatible with Pypy (and, of course, with python3) - but I could not find a single mention about running Pillow with pypy3.
I can get it to build without errors, by installing the pypy3-devel on my Linux system, and manually setting the CFLAGS variable - (I am using fedora here - on Ubunut you should have a pypy3-dev package). But even though it does build, it does fail to run on pypy, raising ImportError: unable to load extension module 'PIL/_imaging.pypy3-24.so': PIL/_imaging.pypy3-24.so: undefined symbol: PyExc_SystemError; (On fedora 22I set CFLAGS=-I/usr/lib64/pypy3-2.4.0/include/ check your pypy3-dev package to see where do these include files are on Ubuntu).
TL;DR Either use pypy2.x or regular cPython3.4 (of course this is preferred) for your project - one of these days the dependencies will catch up. Either way Django should not benefit that much from Pypy's JIT - a heavily loaded webserver will have to deal with database connection scalability (not Python) and good caching (which can be properly configured on layers above Python as well - take a look at "varnish"). If you have that much processing on requests that your site is taking a hit for not using Pypy, consider run the bottleneck algorithms in another process using celery/RPCs - and write that part of the program to run with Pypy/Cython/pure C.
I am still relatively new to python packaging, each time I think I find "the" solution, I am thrown another curve ball, here is my problem followed by what I've tried:
I have CentOS and Ubuntu systems with Python 2.7.3 installed that is partitioned from the net so I have to create an "all in one package"
The target system does NOT have setuptools, easy_install, pip, virtualenv installed (this is the problem I'm trying to solve here)
The requirements.txt (or setup.py install_dependencies) is fairly heavy (Flask, etc...) for the application (though really, this isn't the problem)
My packaging sophistication has progressed slowly:
For connected systems, I had a really nice process going with
packaging: python2.7 setup.py sdist
installation: create a virtualenv, untar the distribution, python setup.py install
For the disconnected system, I've tried a few things. Wheels seem to be appropriate but I can't get to the "final" installation that includes setuptools, easy_install, pip. I am new to wheels so perhaps I am missing something obvious.
I started with these references:
Python on Wheels, this was super helpful but I could not get my .sh scripts, test data, etc... installed so I am actually using a wheel/sdist hybrid right now
Wheel, the Docs, again, very helpful but I am stuck on "the final mile of a disconnected system"
I then figured out I could package virtualenv as a wheel :-) Yay
I then figured out I could package easy_install as a python program :-) Yay, but it depends on setuptools, boo, I can't find how to get these packaged / installed
Is there a reference around for bootstrapping a system that has Python, is disconnected, but does not have setuptools, pip, wheels, virtualenv? My list of things a person must do to install this simple agent is becoming just way too long :/ I suppose if I can finish the dependency chain there must be a way to latch in a custom script to setup.py to shrink the custom steps back down ...
Your process will likely vary according to what platform you are targeting, but in general, a typical way to get what you are trying to achieve is to download packages on an online machine, copy them over to the offline one, and then install them from a file rather than from a URL or repository).
A possible workflow for RPM-based distros may be:
Install python-pip through binary packages (use rpm or yum-downloadonly, to download the package on an online machine, then copy it over and install it on the offline one with rpm -i python-pip.<whatever-version-and-architecture-you-downloaded>).
On your online machine, use pip install --download <pkgname> to download the packages you need.
scp or rsync the packages to a given directory X onto your offline machine
Use pip install --find-links=<your-dir-here> <pkgname> to install packages on your offline machine.
If you have to replicate the process on many servers, I'd suggest you set up your own repositories behind a firewall. In case of pip, it is very easy, as it's just a matter of telling pip to use a directory as its own index:
$ pip install --no-index --find-links=file:///local/dir/ SomePackage
For RPM or DEB repos is a bit more complicated (but not rocket science!), but possibly also not that necessary, as you really only ought to install python-pip once.
The pip install --download option that #mac mentioned has been deprecated and removed. Instead the documentation states that the pip download method should be used instead. So the workflow should be:
Download the python package or installer using your online machine.
Install python using the offline method used by your package manager or the python installer for windows on the offline machine.
On the online machine use pip download -r requirements.txt where "requirments.txt" contains the packages you will be needing the proper format
Use pip install --find-links=<your-dir-here> <pkgname> to install packages on your offline machine.
I am new to python just a few weeks back i started using python(Classic Noob-Disclaimer)
Now whenever i install a module by copying the unzipped folder in site-packages under Lib and running the source install by using "c:\python27\lib\site-packages\tweepy-1.2\setup.py install" in command prompt it installs without any errors.
But now when i make a python script (*.py)
and store it on the desktop it wont work
and it gives out an error "No module found"
but when i store it in the same folder as the source it works perfectly.
also if i open the IDLE GUI it also returns the same error.
But this doesnt happen with the win32com module which i use for TTS.
I missing something..but i cudnt find the answer to it.
Plz help me!
i need to use many of these modules..they work great differently but not together as the modules are always missing!
Copying an unzipped folder to site-packages does not install a Python package.
To install manually, unzip the package to a temporary directory, then run:
python setup.py install
in this directory, after that you can remove the directory.
To download and install a pure Python package automatically, run:
pip install tweepy
if you have pip installed.
The simplest way to install Python packages that have C extensions is to use binary installers (*.exe, *.msi files).
To avoid all this use VirtualEnv
Virtualenv is a tool to create isolated Python environments.
The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform's standard location is), it's easy to end up in a situation where you unintentionally upgrade an application that shouldn't be upgraded.
Or more generally, what if you want to install an application and leave it be? If an application works, any change in its libraries or the versions of those libraries can break the application.
Also, what if you can't install packages into the global site-packages directory? For instance, on a shared host.
The easiest way to install python packages is by using pip. First you need to install pip as explained here if you use windows. Then you can query some packages, from command line, for example
> pip search twitter
Then to install certain packages, just use pip something like this:
> pip install tweepy