I am deploying a custom pytorch model on AWS sagemaker, Following this tutorial.
In my case I have few dependencies to install some modules.
I need pycocotools in my inference.py script. I can easily install pycocotool inside a separate notebook using this bash command,
%%bash
pip -g install pycocotools
But when I create my endpoint for deployment, I get this error that pycocotools in not defined.
I need pycocotools inside my inference.py script. How I can install this inside a .py file
At the beginning of inference.py add these lines:
from subprocess import check_call, run, CalledProcessError
import sys
import os
# Since it is likely that you're going to run inference.py multiple times, this avoids reinstalling the same package:
if not os.environ.get("INSTALL_SUCCESS"):
try:
check_call(
[ sys.executable, "pip", "install", "pycocotools",]
)
except CalledProcessError:
run(
["pip", "install", "pycocotools",]
)
os.environ["INSTALL_SUCCESS"] = "True"
Related
I have python program that imports schedule (import schedule) at the beginning. The code executes without a problem with python3 command, but starting it from other python file with call("sudo python3 ProgramWithSchedule.py", shell=True) returns error ModuleNotFoundError: No module named 'schedule'. And I can't figure out why...
I have library schedule installed with pip, pip3 AND apt-get (tried all three just to be sure :)
Thanks!
Because you are using a different interpreter/virtual environment for each project, which is generally considered the best practice.
You can apply the command below to create a file with all your installed modules, so you can use them whenever you want, by a single command to install all.
To keep/save all modules in a file:
pip freeze > requirements.txt
To install all of them with a single command in a new interpreter/virtual environment:
pip install requirements.txt
In case you tried installing a package and get an output:>>Requirement already satisfied.
You will find a path in your output where it says Requirement already satisfied, copy the path. Now go back to your working environment.
import sys
sys.path.append("/the/path/you/copied")
import schedule
You can try to force the usage of the same python interpreter with :
call(f"sudo {os.getenv('PYTHON3')} ProgramWithSchedule.py", shell=True)
and call your-script.py with :
PYTHON3=$(type python3) your-script.py ...
I want to run a bash shell script before running setup is called in a setup.py:
from numpy.distutils.core import setup, Extension
from subprocess import call
err = call('sh dependencies.sh',shell=True)
if err:
raise Exception('The dependencies failed to compile.')
extensions = [...]
setup(name = 'Package',
packages=['Package'],
ext_modules=extensions)
When I run python -m pip install . -v, everything works. HOWEVER, the script dependencies.sh is run two times, compiling the dependencies two times. How do I do this properly? Thanks!
Is there a way to make certain dependencies for a python package optional ? Or conditional on the installation of another dependency failing.
The two cases I have are:
I want to install dependency x AND y. However, in case either of the install fails, the package can probably work alright with just one of them, so installation should complete with a warning.
I want to install dependency x IF the installation of y failed.
You can have conditional dependencies, but not based on the success/failure of the installation of other dependencies.
You can have optional dependencies, but an install will still fail if an optional dependency fails to install.
The simplest way to do have dependencies be optional (i.e. won't fail the main installation if they fail to install) or conditional (based on the success/failure of other packages) is by adding a custom install command that manually shells out to pip to install the individual packages, and checks the result of each call.
In your setup.py:
import sys
import subprocess
from setuptools import setup
from setuptools.command.install import install
class MyInstall(install):
def run(self):
# Attempt to install optional dependencies
try:
subprocess.check_call([sys.executable, "-m", "pip", "install", "dependency-y"])
except subprocess.CalledProcessError:
subprocess.check_call([sys.executable, "-m", "pip", "install", "dependency-x"])
# Continue with regular installation
install.run(self)
setup(
...
cmdclass={
'install': MyInstall,
},
)
Note that this will only work if you publish a source distribution (.tar.gz or .zip). It will not work if you publish your package as a built distribution (.whl).
I am trying to build an exe file using cx_Freeze from target.py which has an import of keyring in the code. I succeeded in building the exe file, but calls an error saying "No recommended backend was available. Install the keyrings.alt package if you want to use the non-recommended backends. See README.rst for details." I used PyInstaller, but got the same error. I have found the link for ketrings.alt (https://github.com/jaraco/keyrings.alt) but have no idea how to use it.
So, my question is:
Is it possible to use keyring in cx_Freeze ?
How do I use the keyrings.alt ?
If keyring cannot be used in cx_Freeze, is there anyway of converting py files to exe file that has keyring import in them ?
My setup code for cx_Freeze is below.
import sys
import os from cx_Freeze
import setup, Executable
build_exe_options = {"packages":["keyring","selenium"]}
setup(name = "Name",version = "0.1",description = "Description",options = {"build_exe": build_exe_options},executables = [Executable(script="target.py")])
Following code worked for me with cx_freeze.
import keyring
from keyring.backends import Windows
keyring.set_keyring(Windows.WinVaultKeyring())
In setup.py script for cx_freeze add "keyring" in "packages" list.
On Ubuntu 18.04.6 I solved this issue first taking a look at what was failing with:
python -c "import keyring.backends.SecretService as SS; SS.Keyring.priority"
(...)
RuntimeError: The Secret Service daemon is neither running nor activatable through D-Bus
Ref: https://github.com/jaraco/keyring/issues/258
And then these were the steps I followed (most probably you just need to do 3.):
[1.] sudo apt-get install -y python-dbus.
[2.] pip install secretstorage.
3. sudo apt install gnome-keyring.
I am installing a package using python2 setup.py install.
This package is a tkinter application which contains traditional
if __name__ == '__main__':
main()
I tried running python2 -m my_app or python2 -m my_app.__main__ and python2 -c "import my_app" but I either get an error or nothing happens.
I can run it ./my_app.py from console.
How can I run my python application after installation with setup.py?
Import module with main function and call it
python -c "from some_module import main; main()"
but mostly modules/apps, simply expose bin/scripts, look in bin dir of your virtualenv or setup.py.
More info about how to expose (scripts, entry_points):
http://python-packaging.readthedocs.org/en/latest/command-line-scripts.html
https://pythonhosted.org/setuptools/setuptools.html#automatic-script-creation