I've grown tired of trying to get elastic beanstalk to run python 3.5. Instead, I want to create a custom ami which establishes a separate virtualenv for the application (with python 3.5) and knows enough to launch the application using that virtualenv.
The problem is that once I ssh into the ec2 instance in order to create my custom ami, I am left wondering where the scripts are which govern the elastic beanstalk deployment behavior.
For example, when deploying via travis to elastic beanstalk, EB knows enough to look in a specific folder for the file application.py and to execute the file using a specific virtualenv (or maybe even the shudder root python installation of the machine). It even knows to execute a pip install -r requirements. Can anyone point me to where the script(s) are which govern this behavior?
UPDATE
Please see Elastic beanstalk require python 3.5 for those referencing the .ebextensions option. So far, it has not proved able to handle this problem due to the interdependency between the EB image operating system and the python environment used to run the application.
All of the EB files can be found in /opt/elasticbeanstalk - /opt/elasticbeanstalk/hooks is probably most relevant for what you're looking for.
You can use the ebextensions to run scripts you want when starting your ami.
Related
I just deployed a flask-python app with elastic beanstalk on AWS but cannot locate my app source files like application.py or templates/index.html etc
I've looked at looked at /var/../.. or /opt/../.. etc but nowhere to be found.
Is there an ebs command like $ eb find 'filename.py' etc?
/opt/python – Root of where you application will end up.
/opt/python/current/app – The current application that is hosted in the environment.
/opt/python/on-deck/app – The app is initially put in on-deck and then, after all the deployment is complete, it will be moved to current. If you are getting failures in yourcontainer_commands, check out out the on-deck folder and not the current folder.
/opt/python/current/env – All the env variables that eb will set up for you. If you are trying to reproduce an error, you may first need to source /opt/python/current/env to get things set up as they would be when eb deploy is running.
opt/python/run/venv – The virtual env used by your application; you will also need to run source /opt/python/run/venv/bin/activate if you are trying to reproduce an error
As of today, using the default AWS Linux option when creating eb (Python 3.7 running on 64bit Amazon Linux 2/3.1.1), I found the application files in:
/var/app/current
If that doesn't work you can search for a file that you know to be unique in your app, e.g
sudo find / -name hello.html
In my case the above returns
/var/app/current/templates/hello.html
Say I have a file "main.py" and I just want it to run at 10 minute intervals, but not on my computer. The only external libraries the file uses are mysql.connector and pip requests.
Things I've tried:
PythonAnywhere - free tier is too limiting (need to connect to external DB)
AWS Lambda - Only supports up to Python 2.7, converted my code but still had issues
Google Cloud Platform + Heroku - can only find tutorials covering deploying applications, I think these could do what I'm looking for but I can't figure out how.
Thanks!
I'd start by taking a look at this question/answer that I asked previously on unix.stackexchange - I went with an AWS redhat installation and it was free to use.
Once you've decided on your VM, you can add SSH onto your server using any SSH client and upload your Python script. A personal preference is this application.
If you need to update the Python version on the server, you can do this by installing the required Python RPMs. A quick google should return the yum [or whichever RPM management system you're using] repository for the required RPMs.
Once you've installed the version of Python that you need, I'd suggest looking into the 'crontab' which can be used to schedule jobs. You can set a cronjob to run every 10minutes which will call your script.
See this site for more information on how to use the crontab
This sounds like a perfect use case for AWS Lambda which supports Python. You can invoke your Lambda on a schedule using Scheduled Events.
I see that you tried Lambda and it didn't work out for you which is too bad as that seems like the easiest route. You could also launch an EC2 instance and use userdata to schedule a cron when the instance starts.
Another option would be an Elastic Beanstalk worker with a cron.yml that defines your schedule. Elastic Beanstalk supports Python 3.4.
Update: AWS does now support Python 3.6. Just select Python 3.6 from the runtime environments when configuring.
Plesk on the MediaTemple DV servers uses Python 2.4 for stuff, so the 2.4 install can't be replaced but someone recommended installing a separate python 2.7 install since my app runs on that. I'm new to the whole server thing, so this is new territory. My sense is that I can create a new directory for the source files and use SSH to download the files to said directory and then cd into it and install python 2.7. Then I have to figure out how to make sure Apache knows to use Python 2.7 to run the django app in question. Does this sound correct?
Unless you can make a system wide change to your python installation, you would have to run Apache, python and Django in the same virtual environment. If it is not possible, use gunicorn (instead of apache) to run the Django app in a virtualenv (in a port different from Apache's). If this app runs in a subdomain, you should consider hosting the Django app in a PAAS (Heroku, Google App Engine, etc.) which allow you to easily switch running environments.
Never mind, sorted it. the important thing seemed to be to edit the etc/ld.so.conf by adding usr/local/lib, then running sbin/ldconfig, which then makes ld.so.conf look like:
include ld.so.conf.d/*.conf
/usr/local/lib
Donwload and then compile Python
and use make && make altinstall.
I've used the web dashboard of Elastic Beanstalk to make an application and an environment. I know I can update that using the dashboard and uploading a zip file of my application, but I would rather use the command line to upload my application.
Apparently the correct tool for this is eb, the CLI for Elastic Beanstalk. I've installed this and attempted to use it, following the Amazon "Deploying a Flask Application to AWS Elastic Beanstalk" tutorial. However, this seems to create a completely different application to the one visible on the EB dashboard - changes made to it don't appear on the dashboard, and the application even has a different URL.
How can I use the command line to access an existing application on AWS Elastic Beanstalk?
To begin using git aws.push for your application you will have to initialize your git repository with AWS Beanstalk metadata. I'm assuming you are using git for version control (if you are not, you will have to initialize your project with git init first).
$ cd angrywhopper
$ git init #optional
$ eb init
...
$ git aws.push
Walk through wizard steps, commit your code and push the app.
Elastic Beanstalk container can be further customized by either rerunning eb init or with configuration file inside .ebextensions directory.
If eb does not support something you would like to use, have a look at AWS Elastic Beanstalk API Command Line Interface, which is more feature-rich.
More details on the configuration can be found in the following guides:
Customizing and Configuring AWS Elastic Beanstalk Environments
Customizing and Configuring a Python Container
Make sure that service region in eb wizard is the same as region you pick in dashboard dropdown.
NB: I would suggest to use temporary name in the beginning to make sure your app works as expected with the new workflow and then rename it to the original by rerunning eb init. Don't forget to terminate the temporary environment as soon as you done with the migration to avoid any unnecessary fees.
Here are the steps to use "git aws.push" with your existing ElasticBeanstalk(EB) application. (These steps would be useful, specifically, for your question and also if you had setup EB using command line from another machine and are now setting up the tools on a new machine.)
--
Before you start
You should have git installed on your system and your code should have a git repository.
Download the latest "AWS Elastic Beanstalk Command Line Tool" and get it working. Find a link to download here: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-reference-branch-environment.html
The git aws.push command won't work yet cause your .ebextensions isn't configured. (Basically the .ebextensions stores your AWS Keys and info on EB instance to deploy to etc.)
--
Steps
Run the eb --init command. (I do this from the root of my application code directory, and it automatically picks the name of the application. Maybe you can run the command from any other location as well and specify the name manually later.)
AWS-ElasticBeanstalk-CLI-2.6.0/eb/linux/python2.7/eb (on Linux) or
AWS-ElasticBeanstalk-CLI-2.6.0/eb/windows/eb.exe (on Windows)
Enter you AWS Access Key ID and Secret Access Key
Select the environment you configured your application with (The choices are AMI Linux 64bit, Ubuntu 32bit etc.). Basically select the options that you selected while creating your first EB instance.
For Create RDS instance? [y/n]: Say n (You already have a DB instance or don't need one).
Choose "Create a default instance profile".
This would be the last step under eb --init and the script will exit.
You can find more information on the above steps here: http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_Python.html
--
Environment ready to use
The above steps will result in an .ebextensions directory (in ~/ I guess.)
From now on just git commit the changes in your code and run git aws.push and the application will be deployed to AWS. It's quite cool once you have it all configured.
Hope this helps. I jotted this down quickly. Let me know if you find the steps confusing, and I'll try to write it better.
Created Application in aws.amazon.com->elastic beanstalk and trying to access the application on the eb CLI:
a. When you provide eb init on console, CLI will prompt you to choose the region.
b. Make sure to choose the same region as the one you choose on the webpage.
(Note: if you don't choose the same region, it is going to take you into creating a whole new application. This was the mistake I did)
Creating application using eb CLI first locally and then trying to access the same application on the webpage.
a. $> eb console (from the app root directory, provided you did $> eb init initially)
b. You can directly login into the website and make sure to choose the same region (eg: US - N California) where you configured the app locally and you should be able to see the application you deployed.
Im in the process of launching a Django app on ec2, but have hit a wall trying to install my code on my AMI instance. This is my situation: I have a bitnami AMI up and running that has Django, apache, Postgresql, and nearly all my dependancies pre installed, and I have my fully functional Django app running on my local machine that I have been testing thus far with the Django Dev server. After quite a bit of googling, the most common methods of installing an app to an ec2 instance seem either using ssh/sftp/scp to drop a tarball in the instance, or creating a repository and importing code from there. If anyone can tell me the method they prefer, and guide me through the process, or provide a link to a good tutorial, it would be hugely appreciated!
tar -pczf yourfile.tar.gz MyProject
scp -i /home/user/.cert/yourcert.pem yourfile.tar.gz user#serveripaddress:/home/user
tar -xvf /home/user/yourfile.tar
I usually simply scp -R my whole site directory into /home/bitnami of my AMI. I'm using Apache/NGINX/Django with mod_wsgi. So the directory (for example /home/bitnami/djangosites/) gets referred to based on my mod_wsgi path in my apache cfg file.
In other words, why not just move the whole directory recursively (scp -R) instead of making a tarball etc?
Directly copy the folder where your project resides may work. However you mention that you are using a BitNami image, so it is likely that you are using the BitNami Django Stack Amazon image. BitNami also provides a native version of the BitNami Django Stack so I would suggest that you first try to deploy your application on top of the native installer and see what exact steps you need to follow. For instance you may need to install python dependencies or if you plan to use Apache on production instead of the Django development server you will need to configure Apache to serve your project. I'm a BitNami developer and I mention this because make easier the deployment in different platforms (including ec2) is one of the goal of BitNami and as you are already using it you can take advantage of this.