We Keep Coding

PwnTools recv() on output that expects input directly after

Hi I have a problem that I cannot seem to find any solution for.
(Maybe i'm just horrible at phrasing searches correctly in english)
I'm trying to execute a binary from python using pwntools and reading its output completely before sending some input myself.
The output from my binary is as follows:
Testmessage1
Testmessage2
Enter input: <binary expects me to input stuff here>
Where I would like to read the first line, the second line and the output part of the third line (with ':' being the last character).
The third line of the output does not contain a newline at the end and expects the user to make an input directly. However, I'm not able to read the output contents that the third line starts with, no matter what I try.
My current way of trying to achieve this:
from pwn import *
io = process("./testbin")
print io.recvline()
print io.recvline()
print io.recvuntil(":", timeout=1) # this get's stuck if I dont use a timeout
...
# maybe sending data here
# io.send(....)
io.close()
Do I missunderstand something about stdin and stdout? Is "Enter input:" of the third line not part of the output that I should be able to receive before making an input?
Thanks in advance

I finally figured it out.
I got the hint I needed from
https://github.com/zachriggle/pwntools-glibc-buffering/blob/master/demo.py
It seems that Ubuntu is doing lots of buffering on its own.
When manually making sure that pwnTools uses a pseudoterminal for stdin and stdout it works!
import * from pwn
pty = process.PTY
p = process(stdin=pty, stdout=pty)

You can use the clean function which is more reliable and which can be used for remote connections: https://docs.pwntools.com/en/dev/tubes.html#pwnlib.tubes.tube.tube.clean
For example:
def start():
p = remote("0.0.0.0", 4000)
return p
io = start()
io.send(b"YYYY")
io.clean()
io.send(b"ZZZ")

Unable to fetch and display files using snakebite

I'm trying out snakebite. I started the following client:
from snakebite.client import Client
client = Client("my.host.com", 8020, effective_user='datascientist')
First, I tried to list the users directory:
for x in client.ls(['/user/datascientist']):
print x
This worked nicely and printed couple of dictionaries; one for each item in the directory. One of the items is a file foobar.txt which I'd like to see. To that end, I believe I should use Client.cat:
for cat in client.cat(['/user/datascientist/da-foobar.txt',]):
print(cat)
for item in cat:
print(item)
However, this didn't work. I got the following error message:
ConnectionFailureException: Failure to connect to data node at (10.XXX.YYY.ZZZ:50010)
What am I doing wrongly?
BTW: using PyWebHdfsClient from pywebhdfs.webhdfs I managed to see the file by starting a client with the same address but with port 50070. I don't know whether this is relevant or not.
Edit 1: I also tried to use snakebite.client.Client.text and got the same error. I guess this is not surprising.
BTW, the file's content is my file is this\ntest file.

I found a/the solution. It seems like the listing operation can be accomplished on the name-node alone. In contrast, the printing of the text file needs to access the data-nodes! By instantiating the client as follows
client = Client("stage-gap-namenode-2.srv.glispa.com", 8020, effective_user='datascientist',
use_datanode_hostname=True)
the cat operation works as it is not using the internal IP, but the hostname. I summarized a minimal example.

Python based network sniffer (scapy not good enough?)

I am looking for the specific task:
Grab the payload/data from a packet -> Append to a file... BUT. I want specifically to follow packets according to flags/ICMP types/etc... So lets say I want specifically to take the payload of "echo" packets and not the rest.
My (ineffective) code is the following:
from scapy.all import *
f= open('filecaptured', 'a+')
def pkt_diam(pkt):
raw = pkt.getlayer(Raw).load
print raw
# pkt.show()
# fo = open("payload", "wb")
f.write(raw);
sniff (filter="icmp" , store=0, prn=pkt_diam, timeout = 120 )
The problem here is that I cannot find a way to sniff specifically for "type = echo request" and the only parameters that I can use is 'protocol' and host or 'and not host 127.0.0.1'.
Is there a way around this?
I think for this one I need to use ctypes and libpcap.so but I am not sure... (I didnt find any [other] libraries for python - sniffing )

I don't have scapy installed right now, but what if you simply check for the type echo-reply in your callback-function pkt_diam:
if not "echo-reply" in pkt.show():
return

Try filter="icmp[0]=8" for filtering during capture or
if pkt[ICMP].type==8:
in callback function.

print scapy sniff output to file

I have created a sniffer in scapy and I want the packets captured by scapy to be written onto a file for further analysis?
def sniffer(ip):
filter_str = "icmp and host " + ip
packets=sniff(filter=filter_str,count=20)
f = open('log.txt',"a")
#f.write(packets)
The last line of code does not work. Is there any way I could do this?

f.write expects a character buffer, but you supply it with a Sniffed object which is the result of calling sniff. You can, very simply, do the following:
f.write(str(packets))
This should work. But it probably won't display the information exactly as you would like it. You are going to have to do more work collecting information from packets as strings before you write to f.

Python telnetlib not reading everything

I'm trying to automate the download of Argos data using Python's telnetlib, but I can't seem to figure out how to get it to download all of the output. Part of my problem may be that I don't really understand the seemingly asynchronous nature of the commands.
Here's the code:
tn = telnetlib.Telnet(host = HOST, timeout = 60)
with open("argos_prv_{0}-1.txt".format(now_str), 'w') as of:
tn.read_until("Username: ")
tn.write(user + "\n")
tn.read_until("Password: ")
tn.write(password + "\n")
tn.read_until("/")
# Here's the command I'm trying to get the results of:
tn.write("prv,,ds,{0:d},009919,009920\n".format(start_doy))
# At this point, it's presumably dumped it all
tn.read_until("ARGOS READY")
tn.read_until("/")
# Logging out
tn.write("lo\n")
lines = tn.read_all()
of.write(lines)
of.flush()
The code seems to run just fine, but when I look at the output file, it never has everything in it, cutting out at some random point. When I type the same commands in a real telnet session, it works just fine.
I get the sense it has something to do with trying to read_all() after logging out (tn.write("lo\n")), but when I look at the example documentation for telnetlib, it pretty much looks just like this.
Anyway, my question is: can anyone see what I'm doing wrong here? I want to grab the results of the prv,,ds command, but I'm only getting some of it using this particular code.
Thanks.

# At this point, it's presumably dumped it all
tn.read_until("ARGOS READY")
tn.read_until("/")
At a guess, this bit is sucking up the data and doing nothing with it. Think of it like a pair of pipes - you send stuff one way with write, and pull stuff back with read_*. If you've already sucked the stuff up, it won't still be waiting in the pipe when you do read_all later.
EDIT:
OK, you're seeing a different problem. Try this:
lines = tn.read_until("ARGOS READY")
lines += tn.read_until("/")
tn.write("lo\n")
# Write out lines to file.