I have a list of hostnames which i am using to populate an SCP command to transfer a file from the remote server to my local server. The problem is, some of the hosts do not connect via hostname, and i must use the IP. However this issue with this is that the file i want to transfer has the hostname in the filename, therefore i am using the list of hostnames to capture this, which becomes a problem when we go via the IP rather than the hostname
see code
hostnames = [
'HOST1',
'HOST2',
'1.1.1.1', # HOST3
'2.2.2.2', # HOST4
]
for host in hostnames:
child = pexpect.spawn('scp user#{}:/var/log/{}.backup_file0.{}.gz /home/backups/'.format(host, host))
As you can see, host1 and host 2 will connect and successfully locate the correct file as we have used the hostname, for example:
host1.backup_file0.gz
host2.backup_file0.gz
However host 3 and host 4 will fail due to the hostname in the list being an IP, however the file on these hosts is using the hostname as opposed to an IP
As a note i am not connecting to these devices via ssh so i cannot use the socket to pull the hostname
Is there any way to dynamically map these hostnames / ips and use them as needed?
I don't really see why you couldn't just match the ip with the hostname. Maybe use a dictionary:
hosts = {
'HOST1': 'HOST1',
'HOST2': 'HOST2',
'HOST3': '1.1.1.1', # HOST3
'HOST4': '2.2.2.2', # HOST4
}
for host, ip in hosts.items():
child = pexpect.spawn(f"scp user#{ip}:/var/log/{host}.backup_file0.{date}.gz /home/backups/")
Edit, if I understand correctly you added that you do not have the hostname of hosts for which you connect via IP.
Maybe just try globbing then, as each host surely only backs itself up?
for host in hostnames:
child = pexpect.spawn(f"scp user#{host}:/var/log/*.backup_file0.{date}.gz /home/backups/")
Related
I am using PAM authentication to authenticate with my linux server. I have created a view on my website through Apache2 where I can use python to manually validate each login through a web shell with facial recognition and two factor authentication. This is working, but I can't seem to recover the IP address of the incoming connection. I need a way to find the IP address of my connection to the server before SSH is connected, in the PAM module which is running Python. I would like to use bash for this.
I am trying to execute commands to recover the IP address, I tried using "who" and other commands to see incoming SSH connections to no avail. I also tried using "echo $PAM_RHOST" and "$SSH_CLIENT" and "$SSH_CONNECTION" with no success.
I ended up using the auth.log which seems to work perfectly. All I had to do was reverse the log and get the last IP. The below code also collects unique IPs in order of the last login.
`
output = run_command('sudo tail -500 /var/log/auth.log')
op = output.split('\n')
op.reverse()
output = '\n'.join(op)
def unique(thelist):
u = []
for i in thelist:
if i not in u: u.append(i)
return u
ips = unique(re.findall('Accepted publickey for user from ([\d]+\.[\d]+\.[\d]+\.[\d]+)', output))
ip = ips[0]
print(ip) # The last IP
`
A project I am working on has an Android app as a front-end and a Python program that would be used as the back-end.
I want to send data from the Android app (primarily images) to the Python program, do some processing and send the result back to the Android app.
I have found numerous tutorials that suggest using the socket module in python to create the server side, but all tutorials show the server on local network only (For testing purposes I created the client side also in Python, but it would be converted to Java later on)
The server code:
from requests import get
import socket
public_ip = get('https://api.ipify.org').text
print('My public IP address is: {}'.format(public_ip))
# getting the hostname by socket.gethostname() method
hostname = socket.gethostname()
# getting the IP address using socket.gethostbyname() method
local_ip = socket.gethostbyname(hostname)
# printing the hostname and ip_address
print(f"Hostname: {hostname}")
print(f"IP Address: {local_ip}")
#
HOST = local_ip
PORT = 80 # Port to listen on (non-privileged ports are > 1023)
with socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM, proto=0) as s:
s.bind((HOST, PORT))
s.listen()
conn, addr = s.accept()
with conn:
print('Connected by', addr)
while True:
data = conn.recv(1024).decode('utf-8')
if not data:
break
conn.sendall(data.encode('utf-8'))
The client code:
import socket
HOST = '…' # I modify it to the server's public IP address, as printed from the server code
PORT = 80 # The port used by the server
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
with socket.create_connection((HOST, PORT)) as s:
s.sendall(b'Hello, world')
data = s.recv(1024)
print('Received', repr(data))
Using the code above, if I try using any port other than 80 I get ConnectionRefusedError: [Errno 111] Connection refused. And for port 80, I get TimeoutError: [Errno 110] Connection timed out.
In both cases, I try to connect from a device on another network.
I tried to use the ping command in Windows CMD to check the connection to the server, and I get 'connection refused message'.
I understand that the Firewall is what probably blocks the connection, but I don't know how to bypass it. I added a new rule in the Inbound Rules section (as suggested on other websites) but for no avail… The results were the same.
How can I make the connection between remote devices on different networks?
Thanks in advance ☺
In order to connect to your server using a TCP socket connection, you need to make sure your server can listen on a port on a publically available IP address.
If the External IP address is assigned to your computer directly,
and if you run the server code on that computer, then the TCP port opened by the server code should be available on the internet.
However, IP addresses are often assigned to a modem/router in home networks,
instead of assigning them to any connected device directly.
To find out if your External IP address is assigned to the computer directly you can use tools that your OS support (eg. ipconfig on windows). If you can see the IP address returned by api.ipify.org, then it means your computer is connected directly. You can change your code to connect using publically exposed IP:
HOST = public_ip
If this is successful means your computer is assigned an external address directly. Which is highly unlikely.
There are several workarounds for this problem though:
1) Configure your router to forward port
Configure your router to forward all connections to it's external TCP port, to an internal host in your network which is assigned to your computer. Please find instructions how it is done for your router.
2) Setup a remote proxy
If you don't have permission to change your router settings you can set up a remote proxy listening on the TCP port. While there is a number of ways of doing this, very popular is to set up a remote SSH tunnel, for that you need to have a server with SSH access and an external IP. Run this command:
ssh -R 80:localhost:8080 root#your-ssh-server-host
You can also use a third-party service that exposes your private host on the internet like:
Ngrok (Commercial, with free plans)
Localtunnel (Open Source, can be self-hosted)
I am using server(server_name.corp.com) inside a corporate company. On the server i am running a flask server to listen on 0.0.0.0:5000.
servers are not exposed to outside world but accessible via vpns.
Now when i run host server_name.corp.com in the box i get some ip1(10.*.*.*)
When i run ifconfig in the box it gives me ip2(10.*.*.*).
Also if i run ping server_name.corp.com in same box i get ip2.
Also i can ssh into server with ip1 not ip2
I am able to access the flask server at ip1:5000 but not on ip2:5000.
I am not into networking so fully confused on why there are 2 different ips and why i can access ip1:5000 from browser not ip2:5000.
Also what is equivalent of host command in python ( how to get ip1 from python. I am using socktet.gethostbyname(server_name.corp.com) which gives me ip2)
As far as I can tell, you have some kind of routing configured that allows external connections to the server by hostname (or ip1), but it does not allow connection by ip2. And there is nothing unusual in this. Probably, the system administrator can advise why it is done just like this. Assuming that there are no assynchronous network routes, the following function can help to determine public ip of server:
import socket
def get_ip():
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.connect(("8.8.8.8", 80))
local_address = sock.getsockname()
sock.close()
local_address = local_address[0]
except OSError:
local_address = socket.gethostbyname(socket.gethostname())
return local_address
Not quite clear about the network status by your statements, I can only tell that if you want to get ip1 by python, you could use standard lib subprocess, which usually be used to execute os command. (See subprocess.Popen)
I have a script that loops over a dozen hosts and executes several functions in each host. The functions take as a parameter the SSHClient() and then execute commands on it.
I could simply set some attribute on the SSHClient(), but before I do that, is there already a way to determine from an instance of SSHClient() which host is currently being connected to?
for host in hosts:
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(host, username=USERNAME)
f1(ssh)
f2(ssh)
...
Using the ssh variable for the Client, as you've done, there is:
ssh.get_transport().getpeername()
Which will return a tuple of ('ip address', portnumber)
Will that be enough? Looking at the source for paramiko/client.py it doesn't seem to keep a record of the value of connect()'s hostname parameter, it looks up the address with socket.getaddrinfo and then passes the result on to the transport, which is what getpeername() is asking.
The following is probably what you are after...
stdin, stdout, stderr = ssh.exec_command('hostname')
hostname = stdout.read().decode("utf-8").strip('\n')
print(hostname)
How it works:
On any windows and linux, if you type hostname on any terminal, then you will get back the server name. We execute this cmd on the remote, get back the returned output and change it from byte to string format and finally take off trailing new line character.
This is how I solved the issue:
getpeername() returns an IP address, which is not very user friendly for a log message. So I wrapped it with socket.gethostbyaddr().
This returns the FQDN as a string:
socket.gethostbyaddr(ssh.get_transport().getpeername()[0])[0])
And this splits it to just the hostname itself:
socket.gethostbyaddr(ssh.get_transport().getpeername()[0])[0].split('.')[0])
Configuration
LOCAL: A local machine that will create an ssh connection and issue commands on a REMOTE box.
PROXY: An EC-2 instance with ssh access to both LOCAL and REMOTE.
REMOTE: A remote machine sitting behind a NAT Router (inaccessible by LOCAL, but will open a connection to PROXY and allow LOCAL to tunnel to it).
Port Forwarding Steps (via command line)
Create an ssh connection from REMOTE to PROXY to forward ssh traffic on port 22 on the REMOTE machine to port 8000 on the PROXY server.
# Run from the REMOTE machine
ssh -N -R 0.0.0.0:8000:localhost:22 PROXY_USER#PROXY_HOSTNAME
Create an ssh tunnel from LOCAL to PROXY and forward ssh traffic from LOCAL:1234 to PROXY:8000 (which then forwards to REMOTE:22).
# Run from LOCAL machine
ssh -L 1234:localhost:8000 PROXY_USER#PROXY_HOSTNAME
Create the forwarded ssh connection from LOCAL to REMOTE (via PROXY).
# Run from LOCAL machine in a new terminal window
ssh -p 1234 REMOTE_USER#localhost
# I have now ssh'd to the REMOTE box and can run commands
Paramiko Research
I have looked at a handful of questions related to port forwarding using Paramiko, but they don't seem to address this specific situation.
My Question
How can I use Paramiko to run steps 2 and 3 above? I essentially would like to run:
import paramiko
# Create the tunnel connection
tunnel_cli = paramiko.SSHClient()
tunnel_cli.connect(PROXY_HOSTNAME, PROXY_PORT, PROXY_USER)
# Create the forwarded connection and issue commands from LOCAL on the REMOTE box
fwd_cli = paramiko.SSHClient()
fwd_cli.connect('localhost', LOCAL_PORT, REMOTE_USER)
fwd_cli.exec_command('pwd')
A detailed explanation of what Paramiko is doing "under the hood" can be found at #bitprohet's blog here.
Assuming the configuration above, the code I have working looks something like this:
from paramiko import SSHClient
# Set up the proxy (forwarding server) credentials
proxy_hostname = 'your.proxy.hostname'
proxy_username = 'proxy-username'
proxy_port = 22
# Instantiate a client and connect to the proxy server
proxy_client = SSHClient()
proxy_client.load_host_keys('~/.ssh/known_hosts/')
proxy_client.connect(
proxy_hostname,
port=proxy_port,
username=proxy_username,
key_filename='/path/to/your/private/key/'
)
# Get the client's transport and open a `direct-tcpip` channel passing
# the destination hostname:port and the local hostname:port
transport = proxy_client.get_transport()
dest_addr = ('0.0.0.0', 8000)
local_addr = ('127.0.0.1', 1234)
channel = transport.open_channel("direct-tcpip", dest_addr, local_addr)
# Create a NEW client and pass this channel to it as the `sock` (along with
# whatever credentials you need to auth into your REMOTE box
remote_client = SSHClient()
remote_client.load_host_keys(hosts_file)
remote_client.connect('localhost', port=1234, username='remote_username', sock=channel)
# `remote_client` should now be able to issue commands to the REMOTE box
remote_client.exec_command('pwd')
Is the point solely to bounce SSH commands off PROXY or do you need to forward other, non SSH ports too?
If you just need to SSH into the REMOTE box, Paramiko supports both SSH-level gatewaying (tells the PROXY sshd to open a connection to REMOTE and forward SSH traffic on LOCAL's behalf) and ProxyCommand support (forwards all SSH traffic through a local command, which could be anything capable of talking to the remote box).
Sounds like you want the former to me, since PROXY clearly already has an sshd running. If you check out a copy of Fabric and search around for 'gateway' you will find pointers to how Fabric uses Paramiko's gateway support (I don't have time to dig up the specific spots myself right now.)