I am trying to create a fabfile.py so that I can deploy on EC2. I have the following in my fabfile.py:
from __future__ import with_statement
from fabric.api import *
def ec2():
env.hosts = ['111.111.111.111']
env.user = 'ubuntu'
env.key_filename = '/path/to/my/pem/key.pem'
def run_ls():
run('ls -alt')
'111.111.111.111' is the elastic ip of my instance, and i alway login with ubuntu, not root.
when i run the following command
fab ec2 run_ls
i see the following output:
[111.111.111.111] Executing task 'run_ls'
[111.111.111.111] run: ls -alt
Fatal error: Host key for 111.111.111.111 did not match pre-existing key! Server's key was changed recently, or possible man-in-the-middle attack.
Aborting.
Not sure what is going on, but I can't find to seem any great tutorials on using fabric on ec2, and I do not know how that is possible.
Thanks
Update:
Looks like
env.hosts = ['111.111.111.111']
is not valid, you need to use the actually URL
env.hosts = ['mywebsite.com']
which fixed my issue
You can also use the '--disable-known-hosts' switch to ignore this error.
Make sure your elastic IP is attached to the instance. I think the key_filename takes a single argument but mine is working when you pass in an array instead:
env.user = "ubuntu"
env.key_filename = ["my_key.pem",]
Maybe you should try using the public hostname of your instance like:
env.roledefs.update({
'prod': ['ec2-52-14-72-225.us-west-1.compute.amazonaws.com'],
})
From a Vagrant issue on GitHub, you may need to remove the host from the known_hosts file using a command like this:
ssh-keygen -R 111.111.111.111
Related
I am trying to use the Docker Python API to login to a Docker cloud:
https://docker-py.readthedocs.io/en/stable/client.html#creating-a-client1
What is the URL? What is the Port?
I have tried to get it to work with cloud.docker.com, but I am fine with any registry server, so long as it is free to use and I can use it to upload Docker images from one computer and run them on another.
I have already got everything running using my own locally hosted registry, but I can’t seem to figure out how to connect to a server. It’s kind of ridiculous that hosting my own registry is easier than using an existing registry server.
My code looks like this, but I am unsure what the args.* parameters should be:
client = docker.DockerClient(base_url=args.docker_registry)
client.login(username=args.docker_user, password=args.docker_password)
I’m not sure what the base_url needs to be so that I can log in, and the error messages are not helpful at all.
Can you give me an example that works?
The base_url parameter is the URL of the Docker server, not the Docker Registry.
Try something like:
from docker.errors import APIError, TLSParameterError
try:
client = docker.from_env()
client.login(username=args.docker_user, password=args.docker_password, registry=args.docker_registry)
except (APIError, TLSParameterError) as err:
# ...
Here's how I have logged in to Docker using Python:
import docker
client = docker.from_env()
client.login(username='USERNAME', password='PASSWORD', email='EMAIL',
registry='https://index.docker.io/v1/')
and here's what it returns:
{'IdentityToken': '', 'Status': 'Login Succeeded'}
So, that means it has been logged in successfully.
I still haven't figured out what the registry of cloud.docker.com is called, but I got it to work by switching to quay.io as my registry server, which works with the intuitive registry='quay.io'
I am trying to establish a SSH connection between a Windows PC and a Linux server(amazon ec2).
I decided to use Fabric API implemented using python.
I have Putty installed on the Windows PC.
My fabfile script looks like this:
import sys
from fabric.api import *
def testlive():
print 'Test live ...'
run("uptime")
env.use_ssh_config = False
env.host_string = "host.something.com"
env.user = "myuser"
env.keys_filename = "./private_openssh.key"
env.port = 22
env.gateway = "proxyhost:port"
testlive()
I am running Fabric in the same directory with the private key.
I am able to login on this machine using Putty.
The problem: I am constantly asked for Login password for specified user.
Based on other posts(here and here) I already tried:
pass as a list the key file to env.keys_filename
use username#host_string
use env.host instead of env.host_string
How to properly configure Fabric to deal with proxy server and ssh private key file ?
The following should work.
env.key_filename = "./private_openssh.key"
(notice the typo in your attempt)
Fabric's API is best avoided really, way too many bugs and issues (see issue tracker).
You can do what you want in Python with the following:
from __future__ import print_function
from pssh import ParallelSSHClient
from pssh.utils import load_private_key
client = ParallelSSHClient(['host.something.com'],
pkey=load_private_key('private_openssh.key'),
proxy_host='proxyhost',
proxy_port=<proxy port number>,
user='myuser',
proxy_user='myuser')
output = client.run_command('uname')
for line in output['host.something.com'].stdout:
print(line)
ParallelSSH is available from pip as parallel-ssh.
PuTTYgen is what you will use to generate your SSH key then upload the copied SSH key to your Cloud Management portal - See Joyant
You will have to generate and authenticate a private key, to do so, you need PuTTYgen to generate the SSH access using RSA key with password, key comment and conform the key passphrase, here is a step by step guide documentation SSH Access using RSA Key Authentication
I have a fabfile
env.hosts = [
# huge list, 80 hostnames
]
def dm():
with cd('/home/project/application'):
# how to get the actual hostname here, so I can run certain commands depending on hostname?
any ideas? fabric documentation gives me nothing on this
You can access hostname using env.host.
I have a flask app hosted on Heroku that needs to run commands on an AWS EC2 instance (Amazon Linux AMI) using boto.cmdshell. A couple of questions:
Is using a key pair to access the EC2 instance the best practice? Or is using username/password better?
If using a key pair is the preferred method, what's the best practice on managing/storing private keys on Heroku? Obviously putting the private key in git is not an option.
Thanks.
Heroku lets you take advantage of config variables to manage your application. Here is an exmaple of my config.py file that lives inside my flask application:
import os
# flask
PORT = int(os.getenv("PORT", 5000))
basedir = str(os.path.abspath(os.path.dirname(__file__)))
SECRET_KEY = str(os.getenv("APP_SECRET_KEY"))
DEBUG = str(os.getenv("DEBUG"))
ALLOWED_EXTENSIONS = str(os.getenv("ALLOWED_EXTENSIONS"))
TESTING = os.getenv("TESTING", False)
# s3
AWS_ACCESS_KEY_ID = str(os.getenv("AWS_ACCESS_KEY_ID"))
AWS_SECRET_ACCESS_KEY = str(os.getenv("AWS_SECRET_ACCESS_KEY"))
S3_BUCKET = str(os.getenv("S3_BUCKET"))
S3_UPLOAD_DIRECTORY = str(os.getenv("S3_UPLOAD_DIRECTORY"))
Now i can have two different sets of results. It pulls from my Environment variables. One when my application is on my local computer and from Heroku config variables when in production. For example.
DEBUG = str(os.getenv("DEBUG"))
is "TRUE" on my local computer. But False on Heroku. In order to check your Heroku config run.
Heroku config
Also keep in mind that if you ever want to keep some files part of your project locally but not in heroku or on github you can use git ignore. Of course those files won't exist on your production application then.
What I was looking for was guidance on how to deal with private keys. Both #DrewV and #yfeldblum pointed me to the right direction. I ended up turning my private key into a string and storing it in a Heroku config variables.
If anyone is looking to do something similar, here's a sample code snippit using paramiko:
import paramiko, base64
import StringIO
import os
key = paramiko.RSAKey.from_private_key(StringIO.StringIO(str(os.environ.get("AWS_PRIVATE_KEY"))))
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(str(os.environ.get("EC2_PUBLIC_DNS")), username='ec2-user', pkey=key)
stdin, stdout, stderr = ssh.exec_command('ps')
for line in stdout:
print '... ' + line.strip('\n')
ssh.close()
Thanks to #DrewV and #yfeldblum for helping (upvote for both).
You can use config vars to store config items in an application running on Heroku.
You can use a username/password combination. You may make the username something easy; but be sure to generate a strong password, e.g., with something like openssl rand -base64 32.
I'm having an error I can't seem to get past. I have a simple fabric task that must be run as a different user on the remote system, e.g:
def update():
env.user = 'otheruser'
#~ env.password = 'otherpass' # this works, but I don't want it here.
with cd(env.sitefolder):
run('hg pull -u')
If I run this with env.password hardcoded it works. If I use fab -p otherpass update it works too. If I omit it the docs say I will get prompted. This is true, but it doesn't work. Every time after entering the password I get this error:
> fab dev update
[darkstar] Executing task 'update'
[darkstar] run: hg pull -u
[darkstar] Login password:
ERROR:paramiko.transport:Exception: Error reading SSH protocol banner
...
Fatal error: Error reading SSH protocol banner
Aborting.
Using fabric 1.2.2 on Ubuntu Natty. I also tried the env.no_keys option but it didn't change anything. Can anyone help?
Prompt for the password yourself!
def update():
env.user = 'otheruser'
env.password = getpass.getpass('Enter password: ')
with cd(env.sitefolder):
run('hg pull -u')
getpass is part of the standard library, it's just a raw_input that doesn't echo what you type
This can also happen if target is out of memory/disk space. Restarting and/or solving memory/disk space problems can solve this.