I am trying to use aws_cdk in my project.
I am using visual studio as a IDE and I setup my virtualenv doing this:
Install https://nodejs.org/en/.
Through npm, install cdk from cmd window: npm install -g aws-cdk
Install python3
Using terminal in Visual Studio Code: go to the project folder
Run python -m venv .venv to create the virtual env.
Activate it: source .venv/bin/activate
Run pip install: pip install -r requirements.txt
Run another pip install: pip install -r requirements-dev.txt.
Check with cdk diff in the cmd
This last step returns:
Traceback (most recent call last):
File "C:......\app.py", line 4, in <module>
import aws_cdk as cdk
ModuleNotFoundError: No module named 'aws_cdk'
My App.py looks like this:
#!/usr/bin/env python3
import os
import aws_cdk as cdk
import module_config
from myproject.myproject_stack import MyStack
app = cdk.App()
tags = module_config.with_tags(service_name="my_project")
prefix = module_config.with_prefix(service_name="my_project")
MyStack(
app,
construct_id=prefix,
prefix=prefix,
tags=tags,
env=module_config.env,
)
app.synth()
My requirements.txt looks like:
aws-cdk-lib==2.56.1
constructs>=10.0.0,<11.0.0
The requirement-dev.txt is:
bandit>=1.7.4
black>=22.10.0
coverage>=6.4.4
pylint==2.15.5
pytest
pytest-cov
yamllint
pre-commit
If in the vistual studio I do ctrl+mouse click it opens the aws-cdk code, therefore I know it is installed, but seems like the virtual env is not able to find it.
In my local repository, the folder .venv has a folder Lib\site-packages and this one has another called aws_cdk.
I see everything corret, but when running the cdk diff it breaks.
Related
When trying to synthesize my CDK app, I receive the following error:
`
Traceback (most recent call last):
File "C:\Users\myusername\PycharmProjects\rbds-cdk_testing\app.py", line 2, in <module>
from aws_cdk.core import App, Environment
File "C:\Users\myusername\PycharmProjects\rbds-cdk_testing\.venv\lib\site-packages\aws_cdk\__init__.py", line 1260, in <module>
from .cloud_assembly_schema import (
ImportError: cannot import name 'AssetManifestOptions' from 'aws_cdk.cloud_assembly_schema' (C:\Users\myusername\PycharmProjects\rbds-cdk_testing\.venv\lib\site-packages\aws_cdk\cloud_assembly_schema\__init__.py)
I am using node version 18.0.0. Here's the steps I've done in creating my CDK app:
(FROM c:\Users\myusername\)
installed nvm
installed npm
nvm use 18.0.0
npm install -g yarn
npm install -g aws-cdk
cdk bootstrap aws://account-number/region
cd .\PyCharmProjects\mycdkapp
cdk init app --language python
.venv\Scripts\activate.bat
python -m pip install aws-cdk.aws-glue
python -m pip install aws-cdk
I error out even when executing cdk ls as the runtime tries to run app.py which contains
\
import yaml
from aws_cdk.core import App, Environment
from pipeline import PipelineCDKStack
In checking whether the init.py file for aws_cdk contains AssetManifestOptions, I've discovered it is completely missing:
Am I missing something here or is this a unique bug that I am experiencing? Any help much appreciated! I am banging my head on this one.
Its the same here, I think the issue can be in wrong package version.
cloud-assembly-schema==2.50.0 contains AssetManifestOptions.
Can you please paste here output of
pip list -v | grep aws
Iam able to install 2.50.0, however it depends on other packages of the same version (see attach)
And I cant set up core package because there is no CDKv2 matching distribution at the moment
I am trying to run a python script in EC2 user-data. The EC2 instance I am launching uses a custom AMI image that I repaired. I installed the 2 packages that I need - boto3 and pyodbc packages by executing these commands (notice, I am installing those as root):
sudo yum -y install boto3 pyodbc
My user-data script:
#!/bin/bash
set -e -x
# set AWS region
echo "export AWS_DEFAULT_REGION=${region}" >> /etc/profile
source /etc/profile
# copy python script from the s3 bucket
aws s3 cp s3://${bucket_name}/ /home/ec2-user --recursive
/usr/bin/python3 /home/ec2-user/my_script.py
After launching a new EC2 Instance (using the my custom AMI) and checking /var/log/cloud-init-output.log I see that error:
+ python3 /home/ec2-user/main_rigs_stopgap.py
Traceback (most recent call last):
File "/home/ec2-user/my_script.py", line 1, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [1]
cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failed
Any suggestions, please?
To make sure that you installed modules to correct version of python, use builtin pip module of the python version you are using:
/usr/bin/python3 -m pip install module_name
I'm trying to make an open source contribution to a python module that is hosted on Github (pypika).
I cloned the repo from github and ran pip in editable install mode such that any future imports would point to my version of the code.
But when I try running a test file within the repo, I get an error when trying to import the module. What am I doing wrong? How can I make it so that the import will use the modified module that I'm working on?
$ cd Dev
$ git clone https://github.com/kayak/pypika.git
$ pip install -e /Users/me/Dev/pypika
Obtaining file:///Users/me/Dev/pypika
Installing collected packages: PyPika
Running setup.py develop for PyPika
Successfully installed PyPika
$ python3 ./pypika/pypika/tests/test_functions.py
Traceback (most recent call last):
File "./pypika/pypika/tests/test_functions.py", line 3, in <module>
from pypika import (
ImportError: No module named 'pypika'
PyPika maintainer here. In order to contribute it's best to make a fork on Github and put your contributions in a separate branch which you can then pull request.
In order to run the tests you simply need to execute python -m unittest in the project folder of PyPika. (Or use the Python test runner functionality of your favourite IDE.)
pip install -e /Users/me/Dev/pypika will work if executed in the environment of the project in which you want to use PyPika.
I am following along with the O'Riley Head First Python (2nd Edition) Course.
At one point you will create a webapp and deploy it to pythonanywhere (chapter5).
The webapp uses two functions, imported from a module, created earlier.
The module is called vsearch.py. I also created a readme.txt and a setup.py and used setuptools to create a source distribution file using :
python3 setup.py sdist
The code of the setup.py read as follows:
from setuptools import setup
setup(
name = "vsearch",
version = "1.0",
description = "The Head First Python Seach Tools",
author = "HF Python 2e",
author_email = "hfpy2e#gmail.com",
url = "headfirstlabs.com",
py_modules = ["vsearch"],
)
The source distribution file gets created without errors and creates a file called vsearch-1.0.tar.gz
The file then gets uploaded to pythonanywhere and installed via console using:
python3 -m pip install vsearch-1.0.tar.gz --user
Console outputs:
15:36 ~/mysite $ python3 -m pip install vsearch-1.0.tar.gz --user
Looking in links: /usr/share/pip-wheels
Processing ./vsearch-1.0.tar.gz
Building wheels for collected packages: vsearch
Running setup.py bdist_wheel for vsearch ... done
Stored in directory: /home/Mohr/.cache/pip/wheels/85/fd/4e/5302d6f3b92e4057d341443ed5ef0402eb04994663282c12f7
Successfully built vsearch
Installing collected packages: vsearch
Found existing installation: vsearch 1.0
Uninstalling vsearch-1.0:
Successfully uninstalled vsearch-1.0
Successfully installed vsearch-1.0
Now when I try to run my webapp I get the following error:
2020-03-24 16:18:14,592: Error running WSGI application
2020-03-24 16:18:14,592: ModuleNotFoundError: No module named 'vsearch'
2020-03-24 16:18:14,593: File "/var/www/mohr_eu_pythonanywhere_com_wsgi.py", line 16, in <module>
2020-03-24 16:18:14,593: from vsearch4web import app as application # noqa
2020-03-24 16:18:14,593:
2020-03-24 16:18:14,593: File "/home/Mohr/mysite/vsearch4web.py", line 3, in <module>
2020-03-24 16:18:14,593: from vsearch import search4letters
Judging from this error I assume that "vsearch" can not be found because it was installed as "vsearch-1.0". However when I try to change this line to:
from vsearch-1.0 import search4letters
I rightfully get a synthax error since I can not adress modules this way. So what can I do about this? When creating the module in the beginning I added a version number to the setup.py file because according to the lecture it is good practice. Setuptools then automatically creates the source distribution file with the "-1.0" at the end. Also when importing it using the command shown above i automatically gets importet as "vsearch-1.0" which in turn I am unable to reference in my python code because of bad synthax.
Am I doing something wrong? Is there a way to import this under another namespace? Is there a way to reference "vsearch-1.0" in my python code without getting a synthax error?
There are different python3 versions installed on PythonAnywhere. When you install something using python3 -m pip or pip3 you use default python3 that is probably not matching python version setting of your web app. Use python3.7 and pip3.7 or python3.6 and pip3.6 etc. for --user installations to be sure.
pip install --user (with emphasized --user) installed the package into your user directory: /home/Mohr/.local/lib/pythonX.Y/site-packages/.
To run your WSGI application you probably use a virtual environment in which the user-installed modules are not available. To use modules in the venv you have to install everything in the venv. So activate the venv in a terminal and install the module with the venv's pip:
pip install vsearch-1.0.tar.gz
I am deploying a web application onto an AWS EC2 instance, and I'm getting an error. The logs indicate that I do not have cv2 installed.
ModuleNotFoundError: No module named 'cv2'
However, if I ssh into my instance, and run python from the shell I can import no problem.
https://drive.google.com/drive/folders/1-w3BN9pMAhkiDM40fODCPdjvU1Nx71UT?usp=sharing
I have already installed opencv onto the Linux server and checked that it is available for import.
From my application.py file
import cv2
File "/opt/python/current/app/localize.py", line 9, in
but from the command line:
>>> import cv2
>>> cv2.__version__
'4.1.0'
I expected the import to work since it works from the command line.
Check if your python package is available for the root/admin user but not accessible for the user trying to run the code?
If you can import that module in your EC2, then it is installed, but more importantly for which user it is installed, and for which version.
First try :
chmod 755 on all directories in python path for your default python and see if it works.(This will provide permissions for all import libraries in Python)
If your script is running Python3.7 and Default is Python2.7 then you might have to do --
sudo pip3 install opencv-python
the way to check the version defaults is:
which python ---Will provide default python path and version
which pip ---- Will Provide default PIP details
As #rayryeng suggested, I'm running Python 3.x from Elastic Beanstalk, and Python 2.x from command line. I fixed it by installing the correct version of cv2 for Python 3 and including the following before my import:
import sys
sys.path.append('/usr/local/lib64/python3.6/site-packages')
Try this:
cd
wget https://github.com/opencv/opencv/archive/3.2.0.zip
virtualenv project
source project/bin/activate
pip install numpy
mkdir local
unzip opencv-3.2.0.zip
cd opencv-3.2.0
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D BUILD_SHARED_LIBS=NO -D WITH_FFMPEG=ON -D BUILD_opencv_python2=ON -D CMAKE_INSTALL_PREFIX=~/local ~/opencv-3.2.0
make
make install
cp ~/local/lib/python2.7/site-packages/cv2.so ~/project/lib64/python2.7/site-packages/
Read more from here: Source