We Keep Coding

mitmproxy & python - ignore all hosts with https/ssl

I've done alot of research, and I can't find anything which actually solves my issue.
Since basically no site accepts mitmdumps certificate for https, I want to ignore those hosts. I can access a specific website with "--ignore-hosts (ip)" like normal, but I need to ignore all HTTPS/SSL hosts.
Is there any way I can do this at all?
Thanks alot!

There is a script file called tls_passthrough.py on the mitmproxy GitHub which ignores hosts which has previously failed a handshake due to the user not trusting the new certificate. Although it does not save for other sessions.
What this also means is that the first SSL connection from this perticular host the will always fail. What I suggest you do is write out all the IPs which has failed previously into a text document and ignore all hosts which are in that text file.
tls_passthrough.py
To simply start it, you just add it with the script argument "-s (tls_passthrough.py path)"
Example,
mitmproxy -s tls_passthrough.py

you need a simple addon script to ignore all tls connections.
import mitmproxy
class IgnoreAllTLS:
def __init__(self) -> None:
pass
def tls_clienthello(self, data: mitmproxy.proxy.layers.tls.ClientHelloData):
'''
ignore all tls event
'''
# LOGC("tls hello from "+str(data.context.server)+" ,ignore_connection="+str(data.ignore_connection))
data.ignore_connection = True
addons = [
IgnoreAllTLS()
]
the latest version ( 7.0.4 for now) is not support ignore_connection feature yet,so u need to install the main source version:
git clone https://github.com/mitmproxy/mitmproxy.git
cd mitmproxy
python3 -m venv venv
activate the venv before startup the proxy
source /path/to/mitmproxy/venv/bin/activate
startup mitmproxy
mitmproxy -s ignore_all_tls.py

You can ignore all https/SSL traffic by using a wildcard:
mitmproxy --ignore-hosts '.*'

Ansible specifying inventory file doesn't work

I am trying to specify inventory file in Ansible.
The help command output:
-i INVENTORY, --inventory-file=INVENTORY
specify inventory host file
(default=/usr/local/etc/ansible/hosts)
I tried to do like this:
ansible -i /Users/liu/personal/test_ansible/hosts
but it doesn't work and instead it outputs the help content once again:
➜ test_ansible ansible -i /Users/liu/personal/test_ansible/hosts
Usage: ansible <host-pattern> [options]
Options:
-a MODULE_ARGS, --args=MODULE_ARGS
module arguments
--ask-become-pass ask for privilege escalation password
-k, --ask-pass ask for SSH password
--ask-su-pass ask for su password (deprecated, use become)
-K, --ask-sudo-pass ask for sudo password (deprecated, use become)
--ask-vault-pass ask for vault password
-B SECONDS, --background=SECONDS
run asynchronously, failing after X seconds
(default=N/A)
.......
What am I missing here?

When you use the ansible command it will run ad-hoc Ansible modules rather than the more typical Ansible playbooks (which is ran by the ansible-playbook executable instead).
The ansible executable has a requirement of a "host pattern" which will match a group of remote nodes defined in the inventory.
So if we supplied an inventory file (named inventory.ini for this example) that looked like this:
[web]
web-1.example.org
web-2.example.org
[app]
app-1.example.org
app-2.example.org
app-3.example.org
[database:children]
database-master
database-slave
[database-master]
database-master.example.org
[database-child]
database-slave1.example.org
database-slave2.example.org
We could target just the web nodes by using ansible web -i /path/to/inventory.ini -m ping to get Ansible to use the ping module against the web-1.example.org and web-2.example.org.
Alternatively we could target all of the database nodes including the master and the 2 slaves by using ansible database -i /path/to/inventory.ini -m ping.
And finally, we can also target all of the servers in the inventory by using the "magic" all group that covers all of the groups in the inventory file by using ansible all -i /path/to/inventory.ini -m ping.

I found the solution:
export ANSIBLE_INVENTORY=/Users/liu/personal/test_ansible/hosts
then will be ok!

I want change directory and run command in fabric

I want to change directory and when I run command with cd('myApp') I get:
No hosts found. Please specify (single) host string for connection:
I have this code:
def example():
local('sudo apt-get install python-dev libmysqlclient-dev')
local('pip install MySQL-python')
local('sudo apt-get install apache2')
with cd('myApp'):
run('pwd')
run('python manage.py syncdb --no-initial-data')
run('python manage.py migrate')
print(green('DONE.'))

As per the official tutorial, the error specifies that you have not specified a connection in your fabfile for it to deploy. Please check here.
Other than that, in the cd method(used along side the with statement), use full path like
with cd('/path/to/directory/myApp')
rather than just the 'myApp'. Even if it is just '/myApp'. It improves readability and also makes sure that that is the path you wish to go.

Using fabric to copy file from one remote server to another remote server?

I need to make a service that can copy a file from one remote server to another remote server. I could just run this in a subshell:
scp user#host1:/path/to/file/video.mp4 user#host2:/path/to/file
But that doesn't seem nearly as good as using something like fabric's put command, which only copies a local file to a remote machine. I also need this to run entirely with ssh keys, no password prompting.

You can use get to get the file from host1 and then put to push it to host2.
For keys, if the private key isn't in the default location, you can specify it with the -i flag, e.g.:
fab -i /path/to/priv/key
or put it in the fabfile:
from fabric import env
env.key_filename = '/path/to/priv/key'

Also there is a possibility where agent forwarding can be used(as of fabric 1.10, with option -A). In which case you can just run the following command from host1
scp /path/to/file/video.mp4 user#host2:/path/to/file
or from host2:
scp user#host1:/path/to/file/video.mp4 /path/to/file

Can I prevent fabric from prompting me for a sudo password?

I am using Fabric to run commands on a remote server. The user with which I connect on that server has some sudo privileges, and does not require a password to use these privileges. When SSH'ing into the server, I can run sudo blah and the command executes without prompting for a password. When I try to run the same command via Fabric's sudo function, I get prompted for a password. This is because Fabric builds a command in the following manner when using sudo:
sudo -S -p <sudo_prompt> /bin/bash -l -c "<command>"
Obviously, my user does not have permission to execute /bin/bash without a password.
I've worked around the problem by using run("sudo blah") instead of sudo("blah"), but I wondered if there is a better solution. Is there a workaround for this issue?

Try passing shell=False to sudo. That way /bin/bash won't be added to the sudo command. sudo('some_command', shell=False)
From line 503 of fabric/operations.py:
if (not env.use_shell) or (not shell):
real_command = "%s %s" % (sudo_prefix, _shell_escape(command))
the else block looks like this:
# V-- here's where /bin/bash is added
real_command = '%s %s "%s"' % (sudo_prefix, env.shell,
_shell_escape(cwd + command))

You can use:
from fabric.api import env
# [...]
env.password = 'yourpassword'

In your /etc/sudoers file add
user ALL=NOPASSWD: some_command
where user is your sudo user and some_command the command you want to run with fabric, then on the fabric script run sudo it with shell=False:
sudo('some_command', shell=False)
this works for me

In your /etc/sudoers file, you could add
user ALL=NOPASSWD: /bin/bash
...where user is your Fabric username.
Obviously, you can only do this if you have root access, as /etc/sudoers is only writable by root.
Also obviously, this isn't terribly secure, as being able to execute /bin/bash leaves you open to essentially anything, so if you don't have root access and have to ask a sysadmin to do this for you, they probably won't.

Linux noob here but I found this question while trying to install graphite-fabric onto an EC2 AMI. Fabric kept prompting for a root password.
The evntual trick was to pass in the ssh private key file to fabric.
fab -i key.pem graphite_install -H root#servername

You can also use passwords for multiple machines:
from fabric import env
env.hosts = ['user1#host1:port1', 'user2#host2.port2']
env.passwords = {'user1#host1:port1': 'password1', 'user2#host2.port2': 'password2'}
See this answer: https://stackoverflow.com/a/5568219/552671

I recently faced this same issue, and found Crossfit_and_Beer's answer confusing.
A supported way to achieve this is via using env.sudo_prefix, as documented by this github commit (from this PR)
My example of use:
env.sudo_prefix = 'sudo '