Im using the following travis-ci configuration
language: python
env:
- DJANGO=1.4
- DJANGO=1.5
- DJANGO=1.6
python:
- "2.6"
- "2.7"
install:
- sudo pip install Django==$DJANGO
- sudo pip install .
script:
- cd autotest
- python manage.py test ...
But in everytime the tests are executed, I run into the following issue:
$ python manage.py test ...
Traceback (most recent call last):
File "manage.py", line 8, in <module>
from django.core.management import execute_from_command_line
ImportError: No module named django.core.management
The command "python manage.py test ..." exited with 1.
As i said on irc,
You are running pip install as root. More than that, sudo will reset the environment before finding and running pip. This will mean your pip install is not into the virtualenv that travis provides, but into the global site-packages.
When you do python manage.py test you are using the python binary provided by a virtualenv. However virtualenv will not look in the system site-packages. So it cannot see the Django you installed into the system site-packages.
Related
First of all I ran poetry update which seemed to work. To verify if it actually updated my poetry version I ran poetry --version which resulted in the following error:
Traceback (most recent call last):
File "C:\Users\XXX\.poetry\bin\poetry", line 17, in <module>
from poetry.console import main
ModuleNotFoundError: No module named 'poetry.console'
To solve this error, uninstalling poetry appeared the best option. So I tried uninstalling poetry by python get-poetry.py --uninstall and python install-poetry.py --uninstall. Both resulted in [Errno 2] No such file or directory. Furthermore, I tried poetry --uninstall which resulted also in the ModuleNotFoundError.
How can I uninstall poetry and why are my commands not working?
get-poetry.py and install-poetry.py are the installer scripts and have to be downloaded first. You can combine downloading and running in one line, e.g.:
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - --uninstall
or
curl -sSL https://install.python-poetry.org | python3 - --uninstall
for uninstalling poetry. For uninstalling you have to use the same script you have used for installing.
I am trying to run a python script in EC2 user-data. The EC2 instance I am launching uses a custom AMI image that I repaired. I installed the 2 packages that I need - boto3 and pyodbc packages by executing these commands (notice, I am installing those as root):
sudo yum -y install boto3 pyodbc
My user-data script:
#!/bin/bash
set -e -x
# set AWS region
echo "export AWS_DEFAULT_REGION=${region}" >> /etc/profile
source /etc/profile
# copy python script from the s3 bucket
aws s3 cp s3://${bucket_name}/ /home/ec2-user --recursive
/usr/bin/python3 /home/ec2-user/my_script.py
After launching a new EC2 Instance (using the my custom AMI) and checking /var/log/cloud-init-output.log I see that error:
+ python3 /home/ec2-user/main_rigs_stopgap.py
Traceback (most recent call last):
File "/home/ec2-user/my_script.py", line 1, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [1]
cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failed
Any suggestions, please?
To make sure that you installed modules to correct version of python, use builtin pip module of the python version you are using:
/usr/bin/python3 -m pip install module_name
I am building a python module. In order to define its path, a .pth file has been defined as follows:
# creation of the virtual environment
python -v venv env
# activation of the newly creation virtual environment
source env/bin/activate
To set the path of my module (my module is located in packages/regression_model/regression_model) I created this .pth file env/lib/python3.7/site-packages/regression_model.pth which contains:
# env/lib/python3.7/site-packages/regression_model.pth
../../../../packages/regression_model
Now, any where in my project, I can import my module regression_model through this command:
import regression_model
Actually my objective is to use CircleCI for the deployment of my project.
CircleCI is configured as follows:
version: 2
jobs:
test_regression_model:
working_directory: ~/project
docker:
- image: circleci/python:3.7.6
environment: # environment variables for primary container
PYTHONPATH: ~/project/packages/regression_model:~/project/packages/ml_api
steps:
- checkout
- run:
name: Runnning tests
command: |
virtualenv venv
. venv/bin/activate
pip install --upgrade pip
pip install -r packages/regression_model/requirements.txt
chmod +x ./scripts/fetch_kaggle_dataset.sh
./scripts/fetch_kaggle_dataset.sh
python packages/regression_model/regression_model/train_pipeline.py
py.test -vv packages/regression_model/tests
workflows:
version: 2
test-all:
jobs:
- test_regression_model
The problem I am facing is that CircleCI is indicating that my module can not be imported
Traceback (most recent call last):
File "packages/regression_model/regression_model/train_pipeline.py", line 4, in <module>
from regression_model import pipeline
ModuleNotFoundError: No module named 'regression_model'
To solve the problem, the path to that module regression_model has to be defined exactly as it was done locally. The question is then: how to define path in the CircleCI?
I tried to do it through the use of the environment variable PYTHONPATH but without success.
Any suggestions?
I found out the solution. Similarly to what it has been done manually on my local machine, I just define 2 command lines to get it done in CircleCI:
echo "../../../../packages/regression_model" >> env/lib/python3.7/site-packages/extra.pth
echo "../../../../packages/ml_api" >> env/lib/python3.7/site-packages/extra.pth
And below the full yml file just in case it could help others.
version: 2
jobs:
test_regression_model:
working_directory: ~/project
docker:
- image: circleci/python:3.7.6
steps:
- checkout
- run:
name: Runnning tests
command: |
virtualenv env
. env/bin/activate
pip install --upgrade pip
pip install -r packages/regression_model/requirements.txt
echo "../../../../packages/regression_model" >> env/lib/python3.7/site-packages/extra.pth
echo "../../../../packages/ml_api" >> env/lib/python3.7/site-packages/extra.pth
chmod +x ./scripts/fetch_kaggle_dataset.sh
./scripts/fetch_kaggle_dataset.sh
sudo apt-get install unzip
unzip packages/regression_model/regression_model/datasets/house-prices-advanced-regression-techniques.zip -d packages/regression_model/regression_model/datasets/
python packages/regression_model/regression_model/train_pipeline.py
py.test -vv packages/regression_model/tests
workflows:
version: 2
test-all:
jobs:
- test_regression_model
Im trying to run a package wal-e which I have installed as user root with sudo python3 -m pip install wal-e[aws,azure,google,swift].
I can run this command perfectly as user root using envdir /etc/wal-e.d/env wal-e backup-fetch /var/lib/postgresql/9.6/main LATEST.
However, when I sudo su - postgres and then run envdir /etc/wal-e.d/env wal-e backup-fetch /var/lib/postgresql/9.6/main LATEST, I get the error
Traceback (most recent call last):
File "/usr/local/bin/wal-e", line 7, in <module>
from wal_e.cmd import main
ImportError: No module named 'wal_e.cmd'
I gave user postgres full sudo permissions with usermod -aG sudo postgres. Also the wal-e package is installed in the same location.
When I run ls -la I get
-rwxr-xr-x 1 root root 211 Sep 20 14:24 /usr/local/bin/wal-e
Im also on Ubuntu 16.04.3
How can I run the command just like the root user?
I had to run a strict setup process for wal-e in order for the package to function properly.
Virtually what it boiled down to was installing all necessary dependencies on the machine that I was working with before installing and creating the user postgres. If the user was created before all the dependencies were installed, I got permissions errors.
I have project in django 1.4 and I need to run django test in contious integration system (GitLab 6.8.1 with Gitlab CI 4.3).
Gitlab Runner have installed on server with project.
When I run:
cd project/app/ && ./runtest.sh test some_app
I get:
Traceback (most recent call last):
File "manage.py", line 2, in <module>
from django.core.management import execute_manager
ImportError: No module named django.core.management
How I may run tests?
Do you have Django installed on the testrunner?
If not, try to configure a virtualenv for your testsuite. Best might be (if you have changing requirements) to make the setup and installation of this virtualenv part of your testsuite.
Change your job script in gitlab-ci with the following:
#!/bin/bash
export DISPLAY=:10
virtualenv env
source env/bin/activate
pip install -r requirements.txt
python manage.py test
Before doing this,. install virtualenv and xvfb (for selenium test) for GitLab runners.