We Keep Coding

Sending png file via socket in Python

I'm using python version 2.7.9 and i try to send png file.
But something strange happens..i using sockets and sends a post request(or kind of).
I send the request to the server from the client,then i prints the length of the request received on the server, for example, the length is:1051.
Then I do a regex to take the png file data, and then prints the length, and the length is 2632, that he larger than the response?!
I think the problem is that it's actually write the content, but not the right of representation, I tried different things but they did not work, so I ask here how to solve this problem.
Server source code:
import socket
import re
server = socket.socket()
server.bind(('0.0.0.0',8080))
while True:
server.listen(2)
(client, client_addr) = server.accept()
print 'IP :',client_addr
res = client.recv(0xfffffff)
print len(res)
#get file name
file_name = res.split('&')[0]
file_name = str(file_name.split('=')[1])
print repr(res)
#get the data of the file
raw_img = str(re.findall("&photo_data=(.*)" ,res ,re.DOTALL))
print "File name:" + file_name
print "Size:" + str(len(raw_img))
with open(file_name, 'wb') as f:
f.write(raw_img)
print "Done"
Client source code:
import socket
client = socket.socket()
client.connect(('127.0.0.1',8080))
raw_data = open('test.png', 'rb').read()
save_file_name = raw_input("Enter the file name:")
print len(raw_data)
output = 'POST /upload HTTP/1.1\r\n'
output += 'Content-Length:' + str(len(raw_data)) + str(len(save_file_name)) + '\r\n\r\n'
output += 'file_name=' + save_file_name + '&'
output += 'photo_data=' + raw_data
print len(output)
client.send(output)
client.close()

First, you should use while True to receive the full data:
res = ''
while True:
data = client.recv(1024)
if not data:
break
res += data
print len(res)
Then, re.findall actually returns an array, not a string. So you should do this:
r = re.findall("&photo_data=(.*)" ,res ,re.DOTALL)
raw_img = str(r[0])
Now it works fine.
Why doesn't the code before work? Let's say we have a list:
r = ['\x45']
The data in raw_img part is basically like this. If we brutely convert this list to a str, we have:
print len(str[r])) # ['E'], 5
Actually, what we need is r[0]:
print len(str[r[0])) # 1
That's why the size of the file became larger.

Sending json data over irc (python3)

In this project I am trying to send json data to a specific irc channel. The irc bot is supposed to do two things at a time, check for a certain message, and send the json data (Sorry for repeating json data so often :).)
I created a class, for the function which, searches on a website for the data it is supposed to send over irc (search.py):
import re
import json
import requests
class search:
def run():
data = requests.get("http://boards.4chan.org/g/catalog").text
match = re.match(".*var catalog = (?P<catalog>\{.*\});.*", data)
if not match:
print("Couldn't scrape catalog")
exit(1)
catalog = json.loads(match.group('catalog'))
running = True
while running:
try:
filtertext = ("tox")
for number, thread in catalog['threads'].items():
sub, teaser = thread['sub'], thread['teaser']
if filtertext in sub.lower() or filtertext in teaser.lower():
#liste.append(teaser)
#return(teaser)
#print(liste[0])
return(teaser)
running = False
The file which is supposed to send the data to the data to the irc channel (irctest.py):
import socket,threading,time
from search import search
# Some basic variables used to configure the bot
server = b"irc.freenode.net" # Server
channel = b"#volafile" # Channel
botnick = b"Mybot" # Your bots nick
troo = True
def ping(): # This is our first function! It will respond to server Pings.
ircsock.send(b"PONG :pingis\n")
def sendmsg(chan , msg): # This is the send message function, it simply sends messages to the channel.
ircsock.send(b"PRIVMSG "+ chan +b" :"+ msg +b"\n")
def joinchan(chan): # This function is used to join channels.
ircsock.send(b"JOIN "+ chan + b"\n")
def worker():
print(threading.currentThread().getName(), 'Starting')
while True:
#liste3= search.run()
#teaser1 = str(search.teaser)
ircsock.send(b"PRIVMSG "+ channel + b" :"+ search.run() + b"\n")
print(threading.currentThread().getName(), 'Exiting')
ircsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ircsock.connect((server, 6667))
ircsock.send(b"USER "+ botnick + b" "+ botnick + b" "+ botnick + b" :This bot is a result of a tutoral covered on http://shellium.org/wiki.\n")
ircsock.send(b"NICK "+ botnick + b"\n")
joinchan(channel) # Join the channel using the functions we previo
time.sleep(10)
w = threading.Thread(name='worker', target=worker)
i=0
w.start()
while 1: # Be careful with these! it might send you to an infinite loop
ircmsg = ircsock.recv(2048) # receive data from the server
ircmsg = ircmsg.strip(b'\n\r') # removing any unnecessary linebreaks.
print(ircmsg) # Here we print what's coming from the server
if ircmsg.find(b":Hello "+ botnick) != -1: # If we can find "Hello Mybot" it will call the function hello()
hello()
if ircmsg.find(b"PING :") != -1: # if the server pings us then we've got to respond!
ping()
but I get this error message:
Exception in thread worker:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "irctest.py", line 24, in worker
ircsock.send(b"PRIVMSG "+ channel + b" :"+ bytes(search.run()) + b"\n"\
TypeError: string argument without an encoding

When you call ircsock.send() in your function worker(), your last string isn't a bytestring ("\n"). You should also cast the search.run() return value to a bytestring with bytes(search.run(), "utf-8"). Change the specific line to:
ircsock.send(b"PRIVMSG "+ channel + b" :"+ bytes(search.run(), "utf-8") + b"\n")

Python connect to IRC function - must be integer

I made a basic IRC bot that would join a certain channel and say a defined phrase, as part of learning Python. However, are there changes in the latest version of Python?
import socket
nick = 'TigerBot'
passdwd = '*****'
port = '6667'
net = 'irc.snoonet.org'
chan = '#WritingPrompts'
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Was ist da IRC Socket?
sock.connect ((net, port)) # Open server connection
irc.recv (4096) # Buffer setup
irc.send('nick ' + nick + '\r\n') # What is your name?
irc.send('USER AffixBot AffixBot AffixBot :Affix IRC\r\n') #Send User Info to the server
irc.send('JOIN ' + chan + '\r\n') # Join the pre defined channel
irc.send('PRIVMSG ' + chan + ' :Hello.\r\n') #Send a Message to the channel
while True: #While Connection is Active
data = irc.recv (4096) #Make Data the Receive Buffer
if data.find('PING') != -1: # If PING
irc.send('PONG ' + data.split()[1] + '\r\n') # Then PONG
Line 11 is the problem - apparently, what I've got down as a string needs to be an integer. How can I go about this?

Change your definition of port to be an integer.
port = 6667
not
port = '6667'
In general, if you have an existing string value that needs to be an int for a specific call, you can cast by calling int(str), which will return the value of str as an integer or raise a ValueError if it can't cast.

one recv gets two or more send in python sockets

this is a simple socket program which has a server and some clients. clients send their texts encrypted by a simple RSA cryptography then the server side decrypts the sentence and then sends the decrypted sentence back to the client.
server:
import socket
import sys
from thread import *
from math import *
from random import *
import random
HOST = ''
PORT = 8888
size=2**16
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
#Bind socket to local host and port
try:
s.bind((HOST, PORT))
except socket.error , msg:
print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
sys.exit()
print 'Socket bind complete'
#Start listening on socket
s.listen(10)
print 'Socket now listening'
def decoder(codedString,d,n):
breakcoded=[]
coded=(codedString)
#print coded;
for i in range (len(coded)):
breakcoded.append(chr(((int(coded[i])**d) % n)+48))
stri= ""
for i in range (len(breakcoded)):
stri+=breakcoded[i]
return stri
#Function for handling connections. This will be used to create threads
def clientthread(conn):
#Sending message to connected client
conn.send('Welcome to the server. Type something and hit enter\n') #send only takes string
#infinite loop so that function do not terminate and thread do not end.
while True:
#Receiving from client
data = conn.recv(1024)
## data=s.recv(size)
l = int (data)
#print l
coded=[]
i=0
data1=conn.recv(size)
print 'Recieved n: ',data1
n = int (data1)
data2=conn.recv(size)
print 'Recieved d: ',data2
d = int (data2)
for i in range (l):
data3=conn.recv(size)
#print 'Recieved: ',data3
print
coded.append(data3)
print 'coded string has been recieved....'
print ('coded string: ' , coded)
d= decoder(coded,d,n)
print d
reply = 'OK... your message decrypted as: ' + d
if not d:
break
conn.sendall(reply)
#came out of loop
conn.close()
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = s.accept()
print 'Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
s.close()
client:
import socket
from math import *
from random import *
host='localhost'
port=8888
size=2**16
def declare():
a = sample([5,3],2)
return (a[0],a[1])
def coder(input_message):
(p,q)=declare()
for i in range (1):
p=2**p-1
for i in range (1):
q=2**q-1
#print p
#print q
#print ("n= ",p*q)
#print a
def gcd(a,b):
if a%b==0:
return b
elif a%b > 0:
s=a%b
return gcd(b,s)
else:
raise ValueError
n=p*q
phi=(p-1)*(q-1)
e=2
while gcd(phi,e)!=1:
e+=1
d=1
while ((e*d)%phi)!=1:
d+=1
public_key=(n,e)
special_key=(n,d)
ListOfAsciis=[]
coded=[]
breakcoded=[]
for i in input_message:
ListOfAsciis.append(ord(i)-48)
for j in ListOfAsciis:
coded.append((j**e)%n)
#print ("e= ",e)
#print ("d= ",d)
#print ("coded= ",coded)
for i in coded:
breakcoded.append(chr(((i**d) % n)+48))
#print ('n= ' , n)
#print str(coded)
#print coded
return (d,n,str(coded[0]))
def decoder(codedString,d,n):
#input_d= input("please enter your private key d: ")
#input_n= input("please enter your private key n: ")
#d = int (input_d)
#n = int (input_n)
breakcoded=[]
coded=(codedString)
print coded;
for i in range (len(coded)):
breakcoded.append(chr(((int(coded[i])**d) % n)+48))
stri= ""
for i in range (len(breakcoded)):
stri+=breakcoded[i]
return stri
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
print 'socket created'
s.connect((host,port))
print 'connected'
data=s.recv(size)
print 'Recieved: ', data
while True:
input_message= raw_input("please enter your message: ")
message = list(input_message)
s.send(str(len(message)))
## (p,q)=declare()
#n=p*q
(d,n,c)=coder('i')
n=str(n)
print " ",
print " ",
print " ",
print " ",
s.send(n)
print " ",
d=str(d)
s.send((d))
print " ",
print " ",
print " ",
for i in range (len(message)):
(d,n,c)=coder(input_message[i])
print " ",
print " ",
print " ",
s.send((c))
print 'coded string has been sent to the server....'
data=s.recv(size)
print 'Recieved: ', data
now the problem is that the program sometimes works correctly and sometimes not! in false cases the server side gets two send items by the client by one recv. what should i

This is an inherent part of TCP. Stream sockets are byte streams, not message streams.
So, things are working exactly as they're supposed to. If you want to send a sequence of messages over a TCP stream, it's exactly the same problem as saving a sequence of objects to a file—you need some way of delimiting the messages. This could be as simple as using a stream of text, where newlines delimit the messages, or it could be a complex protocol, but it has to be something.
See this blog post for more detail.

Unindent does not match any outer indentation level

On line 51/52, I get the error:
Unindent does not match any outer indentation level.
I understand this has something to do with tabs and spaces.
Note I did not write this code, I found it online and plan to modify it.
Full code (also at http://pastebin.com/n7ML6Rpz)
import os
import re
import socket
import sys
import time
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Create socket
server_socket.bind(("", 9020)) #Bind server to this socket
server_socket.listen(4) #Max number of queued connections
# Welcome message
print ("TCP chat server now awaiting client connection on port 9020...")
chat_log = [] #Contains chat log
time = time.strftime('%l:%M %p %Z on %b %d, %Y') #Server start time formatted nicely
start_time = str(time) #Convert server start time to string
username = "ChatUser" #Default server username if user does not provide one
# Support ~2^x client connections, where x is the number of process forks
os.fork()
os.fork()
os.fork()
# This variable contains the help documentation for the "help" command
chatHelp = ("The chat server accepts the following commands:\n"
+ "adios Closes the program\n"
+ "connection Shows client connection info (IP, port)\n"
+ "get Returns complete chat log\n"
+ "getrange <#> <#> Get chat log entries from <#> to <#> (starts at 1)\n"
+ "help Lists valid commands\n"
+ "name: <text> Sets your username to <text>\n"
+ "test: <text> Echo data back to you <text>\n"
+ "time Shows time when server was initiated\n"
+ "push: <text> Add <text> to chat log\n"
+ "save Save chat log to file\n")
while 1:
# Accept connection
client_socket, address = server_socket.accept()
# Print connection info from client for server log
print ("Received connection from client at"), address
# Used in the connection command function (client request) below
connection = str(address)
# Send welcome string to client
client_socket.send("Welcome to Nigel's chat room! You are logged in as ChatUser.\n Type help for a list of valid commands.\n")
# Loop indefinitely while server running
while 1:
data = client_socket.recv(2048) #Receive client data into buffer
process_data = data.lower() #Lowercase received data for processing
print ("Data received from client>>"), process_data #Print data received from client for log reference
# Functions for the received commands (I use the find library to reduce compatibility errors with other languages)
# ---"adios" command function---
if (process_data.find("adios") == 0):
client_socket.close() #Close socket connection
print ("<Ctrl+C to exit.>>")
break;
# ---"connection:" command function---
elif(process_data.find("connection") == 0):
client_socket.send("Client connection info: " + connection + "\n")
print "User requested connection information"
# ---"getrange" command function w/ regular expression filtering (must be BEFORE "get" command function)---
elif(re.match(r'getrange\s+(\d+)\s+(\d+)',process_data)): # Regex to find correct match with dynamic numbers input
match = re.match(r'getrange\s+(\d+)\s+(\d+)',process_data)
getValue = "Chat log from range "+ match.group(1) + " and " + match.group(2) + ":\n" # Grab first and second range number provided by client
if(len(chat_log) >= int(match.group(1)) and len(chat_log) >= int(match.group(2))): # Check to see if chat log extends to given range
count = int(match.group(1)) - 1
while(count < int(match.group(2))):
getValue += chat_log[count] + "\n"
count += 1
else:
getValue += "<>\n" #No data in range provided by client
client_socket.send(getValue) #Send results to client
# ---"get" command function---
elif(process_data.find("get") == 0):
log = "Chat log: \n"
for item in chat_log:
log += item+" \n"
client_socket.send(log)
# ---"help:" command function---
elif(process_data.find("help") == 0):
client_socket.send(chatHelp + "\n")
print "User requested help"
# ---"name:" command function---
elif(process_data.find("name:") == 0):
username = data[5:].strip() #Only grab the value client set (not "name:")
client_socket.send("Username set to: " + data[5:] + "\n")
# ---"test:" command function---
elif(process_data.find("test:") == 0):
client_socket.send(data[5:]+"\n") #Echo last 5 elements to client
print data
# ---"time" command function---
elif(process_data.find("time") == 0):
client_socket.send("Chat server was started at: " + start_time + "\n")
print "User requested server start time"
# ---"save" command function---
elif(process_data.find("save") == 0):
print "(Saving chat log to file)"
client_socket.send("Saving chat log to file..." + "\n")
filename = "chat.log"
file = open(filename,"w") #Create file
for item in chat_log: #Loop through elements in chat_log
file.write("%s\n" % item) #Write elements one by one on a new line
file.close() #Close/write file
# ---"push" command function---
elif(process_data.find("push:") == 0):
print "(Pushing data to chat log)"
if(username != ""):
chat_log.append(username + ": " + data[5:].strip()) #Save actual chat text to log (not "push:")
else:
chat_log.append(data[5:].strip())
client_socket.send("OK\n")
else:
print "<<Unknown Data Received>>",data #Server log
try:
client_socket.send("Unrecognized command: " + data + "") #Advise client of invalid command
except socket.error, e:
print "<<Ctrl+C to exit>>" #Server log
break;

Python code is sensitive to the indent level you use. Your code reads:
if (process_data.find("adios") == 0):
client_socket.close() #Close socket connection
print ("<Ctrl+C to exit.>>")
break;
Inside the if block, the statements must all line up. Notice how client_socket.close() and the following print statement have different indent levels. You need to change this so that they both have the same indent, like this:
if (process_data.find("adios") == 0):
client_socket.close() #Close socket connection
print ("<Ctrl+C to exit.>>")
break;

The code presently reads:
if (process_data.find("adios") == 0):
client_socket.close() #Close socket connection
print ("<Ctrl+C to exit.>>")
break;
The first statement in the body of the if is indented 6 spaces, while the last two statements are only indented by 1 space. The indentation level ought to be same and consistent throughout the program, something like this:
if (process_data.find("adios") == 0):
client_socket.close() #Close socket connection
print ("<Ctrl+C to exit.>>")
break;
The indentation in the code doesn't seem very standard (or consistent). For instance the body of the while loop in 41/42 is indented more than other bodies of similar statements, e.g., lines 31-33, that's trouble in a language like Python where whitespace matters.
Finally, note it's not a good idea to mix tabs and spaces. PEP 8 -- Style Guide for Python Code recommends the use of spaces over tabs and an indentation of 4 spaces per level.