I am on a journey of understanding what is the proper way to send an email from Python code. I have somewhat progressed in understanding of MX lookup, though: "the larger the island of knowledge, the longer the shoreline of wonder".
Thanks to this answer, I was able to send an email (to a disposable mailbox though), with this code-snippet:
import smtplib
from email.message import EmailMessage
message = EmailMessage()
message.set_content('Content of the message here.')
message['Subject'] = 'Mail sent from code'
message['From'] = 'whoever#whatever.com'
message['To'] = 'aloun36zmzazyxd3tyop#3mail.rocks'
smtplib.SMTP('mail.3mail.rocks:2525')
smtp_server.send_message(message)
smtp_server.quit()
Here is how I come up with SMTP address and port (mail.3mail.rocks:2525):
Done MX lookup for 3mail.rocks domain:
host -t mx 3mail.rocks
3mail.rocks mail is handled by 10 mail.3mail.rocks.
Then I just started checking ports used by default, with telnet mail.3mail.rocks xxx, this gave me the following results:
telnet mail.3mail.rocks 25
Trying 89.38.99.80...
telnet: connect to address 89.38.99.80: Connection refused
telnet: Unable to connect to remote host
telnet mail.3mail.rocks 465
Trying 89.38.99.80...
telnet: connect to address 89.38.99.80: Operation timed out
telnet: Unable to connect to remote host
telnet mail.3mail.rocks 587
Trying 89.38.99.80...
telnet: connect to address 89.38.99.80: Operation timed out
telnet: Unable to connect to remote host
telnet mail.3mail.rocks 2525
Trying 89.38.99.80...
Connected to mail.3mail.rocks.
Escape character is '^]'.
220 node1 ESMTP Haraka 2.8.16 ready
So, that is how I figured out the needed port (by brute-force, essentially).
I went on to test my snippet on another disposable mail service (mailforspam.com), following the same steps — MX lookup (host -t mx mailforspam.com) returned:
mailforspam.com mail is handled by 10 mail2.mailforspam.com.
mailforspam.com mail is handled by 10 mail1.mailforspam.com.
Though I was not able to connect via telnet (I have tried both servers mail2.mailforspam.com and mail1.mailforspam.com) to any of the default ports: port 25 — Connection refused, ports 2525, 587, 465 — Operation timed out.
Questions are:
How do I figure out the proper ports for the server accepting mails on behalf of a particular domain (one that is returned by MX lookup)? My understanding here is that "default" ports are just conventions, and, in fact, servers can use any free port they choose.
I assume that when email is sent from one email provider to another, the SMTP server it is submitted to (one belonging to the user that is sending email) does something similar (i.e. MX lookup => connection to mail accepting server => submitting an email). How do such "real-world" servers figure out the proper port (or they just brute-forcing through the default ones)?
How do I figure out the proper ports for the server accepting mails on behalf of a particular domain (one that is returned by MX lookup)?
What you've shown in your question is more or less correct. You may want to try the ports in a different order. Also, port 2525 is not an official port from any standard I'm aware of but seems to be a convention for bypassing firewalls that block the submission port 587.
One thing to note is that "accepting mail" is not actually one thing. There are "mail user agents" that do "submission" and "mail transfer agents" that do "transfer". "Submission" and "transfer" often live on different ports which explains some of the diversity you've seen. Figure out whether you're doing submission or transfer and select the appropriate group of ports.
My understanding here is that "default" ports are just conventions, and, in fact, servers can use any free port they choose.
This isn't really true, at least not if the servers want anyone to be able to find them, because ...
How do such "real-world" servers figure out the proper port (or they just brute-forcing through the default ones)?
Mail servers that actually want to be able to receive mail must run on a standard port number. For MTAs this means port 25 with maybe a fallback to 465 (though this isn't standardized either). For MUAs this means port 587 with maybe a fallback to 2525 (also not standardized but apparently in common use as a workaround to MUAs being blocked).
In particular, MX records carry no port information, nor does any other DNS record type related to SMTP.
The MX you get needs to support port 25, that's part of the SMTP definition. If you are unable to connect, chances are the block is in the firewall on your own side — port 25 outbound is aggressively blocked from consumer-grade networks, in an attempt to curb direct-injection spam.
Port 25 between authorized servers is not blocked, there is no reason or need for the server to figure out a different port number.
Related
I have to test a broadcast with acknowledgement on localhost. So I have some text files that represent the nodes and inside there is a list of neighbors.
I use localhost as the IP and the port is the number of the node.
The problem is when I receive a message (that I sent) from a node like 7000, python replaces it with a random number for example 65724. So now my father is 65724 instead of 7000, so I cannot remove 7000 from the list of neighbors.
I cannot complete my algorithm and that is very frustrating.
I can send a message with the port number that I want, but it's not very elegant.
Could someone tell me how to not allow python to randomize the port?
rmunn saved me, the answer I was looking far is the bind before connect method. Befor sending a message you bind your own port and you connect to the other one.
This is not a Python problem, per se. You are confused about how ports work.
Each TCP communication (sending or receiving) has two IP addresses and two ports: the host IP and host port, and the destination IP and destination port.
If you're communicating with a computer "out there" on the network, your host and destination IPs will be different. In your test case, your host and destination IPs will both be 127.0.0.1 (localhost). But I'm going to go with the "different IPs" case for my example, because it makes it easier to see.
So your IP address is, say 10.1.2.3, and you're talking to a computer at 10.1.2.99. You tell the system that you want to talk to 10.1.2.99 at port 7000, and it opens up a connection. When that happens, it will randomly pick a source port that's not in use. So now there's a two-way communication channel open:
10.1.2.3:65274 <-> 10.1.2.99:7000
Note that you did not pick that host port. EDIT: I originally said "In fact, the system will not allow you to pick the host port; it will be assigned to you" here, but that is wrong. If s is a socket object, you can call s.bind() to set its source port, then call s.connect() to connect to a destination port.
Now, when you're listening for a message, then you pick the port you're listening on, and the computer that's connecting to you will have a random port. So if you were listening for a message on port 8912, the incoming connection (once established) will look like:
10.1.2.3:8912 <-> 10.1.2.99:38290
Note that 38290 was chosen at random by the operating system of the computer at the 10.1.2.99 IP address.
Now for the bit of Python. You mention sockets in your question title, so I'll assume you're using the socket module from Python's standard library. Once you've created a socket object s, use s.getpeername() to find out the address (host and port) that you've connected to, and s.getsockname() to find out the address (host and port) that you've connected from.
Since you talk about expecting the number 7000 and getting a random number, I think you're using the host socket when you should be using the destination socket.
I've made a server (python, twisted) for my online game. Started with TCP, then later added constant updates with UDP (saw a big speed improvement). But now, I need to connect each UDP socket client with each TCP client.
I'm doing this by having each client first connect to the TCP server, and getting a unique ID. Then the client sends this ID to the UDP server, connecting it also. I then have a main list of TCP clients (ordered by the unique ID).
My goal is to be able to send messages to the same client over both TCP and UDP.
What is the best way to link a UDP and TCP socket to the same client?
Can I just take the IP address of a new TCP client, and send them data over UDP to that IP? Or is it necessary for the client to connect twice, once for TCP and once for UDP (by sending a 'connect' message)?
Finally, if anyone with knowledge of TCP/UDP could tell me (i'm new!), will the same client have the same IP address when connecting over UDP vs TCP (from the same machine)? (I need to know this, to secure my server, but I don't want to accidentally block some fair users)
Answering your last question: no. Because:
If client is behind NAT, and the gateway (with NAT) has more than one IP, every connection can be seen by you as connection from different IP.
Another problem is when few different clients that are behind the same NAT will connect with your server, you will have more than one pair of TCP-UDP clients. And it will be impossible to join correct pairs.
Your method seems to be good solution for the problem.
1- Can I just take the IP address of a new TCP client, and send them data over UDP to that IP? NO in the general case, but ...
2- is it necessary for the client to connect twice, once for TCP and once for UDP ? NO, definitively
3- will the same client have the same IP address when connecting over UDP vs TCP (from the same machine)? YES except in special cases
You really need some basic knowledge of the TCP, UDP and IP protocol to go further, and idealy, on the OSI model.
Basics (but you should read articles on wikipedia to have a deeper understanding) :
TCP and UDP are 2 protocol over IP
IP is a routable protocol : it can pass through routers
TCP is a connected protocol : it can pass through gateways or proxies (firewalls and NATs)
UDP in a not connected protocol : it cannot pass through gateways
a single machine may have more than one network interface (hardware slot) : each will have different IP address
a single interface may have more than one IP address
in the general case, client machines have only one network interface and one IP address - anyway you can require that a client presents same address to TCP and UDP when connecting to your server
Network Address Translation is when there is a gateway between a local network and the wild internet that always presents its own IP address and keep track of TCP connections to send back packets to the correct client
In fact the most serious problem is if there is a gateway between the client and your server. While the client and the server are two (virtual) machines for which you have direct keyboard access, no problem, but corporate networks are generally protected by a firewall acting as a NAT, and many domestic ADSL routers also include a firewall and a NAT. In that case just forget UDP. It is possible to instruct a domestic router to pass all UDP traffic to a single local IP, but it is not necessarily an easy job. In addition, that means that if a user of yours has more than one machine at home, he will be allowed to use only one at a time and will have to reconfigure his router to switch to another one !
First of all when you send data with TCP or UDP you have to give the port.
If your client connect with TCP and after your server send a response with UDP the packet will be reject by the client.
Why? Because you have to register a port for connection and you can not be sure the port is correctly open on the client.
So when you begin a connection in TCP the client open a port to send data and receive the response. You have to make the same with UDP. When client begin all communication with server you can be sure all the necessary port are open.
Don't forget to send data on the port which the connection was open.
Can I just take the IP address of a new TCP client, and send them data over UDP to that IP? Or is it necessary for the client to connect twice, once for TCP and once for UDP (by sending a 'connect' message)?
Why you don't want create 2 connections?
You have to use UDP for movement for example. because if you create an FPS you can send the player's position every 50ms so it's really important to use UDP.
It's not just a question of better connection. If you want to have a really good connection between client and server you need to use Async connection and use STREAM. But if you use stream you'r TCP socket do not signal the end of a socket but you have a better transmition. So you have to write something to show the packet end (for example <EOF>).
But you have a problem with this. Every socket you receive you have to analyze the data and split over the <EOF>. It can take a lot a processor.
With UDP the packet always have a end signal. But you need to implement a security check.
If there any way to know if a client expects server to enable SSL?
I am building a small SMTP server and have implemented SSL on 465 but some clients do not expect SSL so obviously connection fails.
Is it possible to tell this in any way?
There is no clean way for a server to detect if a client expects to use SSL/TLS at the start of the connection. In fact, if the server is expected to send data first (as is the case with SMTP: the server sends a banner before the client sends any data), there is no way at all to do that.
This is the reasons why SSL/TLS is generally used in one of these two ways:
A new port number is designated for the SSL/TLS version of the protocol. For example, HTTP (port 443 instead of port 80), IMAP (port 993 instead of port 143), SMTP (port 465 instead of 25 or 587). The server knows to use SSL/TLS right away if it accepts the connection on the new port.
STARTTLS: The server and client start by talking the non-SSL/TLS version of the protocol, but the server indicates STARTTLS in its service capabilities announcement. The client accept the offer and requests it. Both server and client now restart the protocol using SSL/TLS.
STARTTLS is a bit less efficient because of the non-SSL/TLS conversation between the server and client that happens first (uses several network round trips) and it is not available for use with all protocols (HTTP doesn't support it), but it's generally preferred if available because it makes it easier for things like automatic configuration of email settings (no need to probe a bunch of possible ports and pick the best one).
Port 465 is an example of the first solution: pick a new port and run SSL/TLS on it. That means servers and clients are both supposed to use SSL/TLS right away for communications on that port.
If you are seeing clients trying to talk plaintext SMTP on port 465, those clients are BROKEN. There really isn't anything you can do to work around them. The clients have serious bugs which should be fixed...
Moreover, for SMTP, you really need to be using STARTTLS, not SMTP over SSL/TLS on port 465.
I'm trying to learn how to directly (no libraries) send DHCP request from python on multi-homed machine (multiple interfaces).
I've looked at pydhcplib, but still do not get it.
This code send DHCP packet on specific interface (eth3 in my case - no IP assigned), but it sends with eth0 IP address. How to change my src IP to 0.0.0.0?
dhcp-message is truncated in this example
LOCAL_PORT=68
SERVER_PORT=67
LOCAL_IP="0.0.0.0"
BCAST_IP="255.255.255.255"
LISTEN_DEV="eth3"
MSG_SIZE=2048
Conn=socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
Conn.settimeout(5)
Conn.setsockopt(socket.SOL_SOCKET,IN.SO_BINDTODEVICE,LISTEN_DEV+'\0')
Conn.setsockopt(socket.SOL_SOCKET,socket.SO_BROADCAST,1)
Conn.bind((LOCAL_IP, LOCAL_PORT))
# Create DHCP-Discovery
msg="010106003f7d1664......"
Conn.sendto(msg.decode("hex"),(BCAST_IP,SERVER_PORT))
received = Conn.recv(MSG_SIZE)
Conn.close()
I assume you already know about the Advanced Interprocess Communication Tutorial.
Spoiler Alert: If you want to jump straight to the bottom line, have a look at the DHCP Query recipe.
Edit:
The special value INADDR_ANY (0.0.0.0, or the empty string '' in a python socket) is not an IP address.
"When an address is specified as INADDR_ANY (a manifest constant
defined in < netinet/in.h >), the system interprets the address as 'any
valid address'."
From RFC 2131:
In the case of a client using DHCP for initial configuration
(before the client's TCP/IP software has been completely
configured), DHCP requires creative use of the client's TCP/IP
software and liberal interpretation of RFC 1122. The TCP/IP
software SHOULD accept and forward to the IP layer any IP packets
delivered to the client's hardware address before the IP address is
configured; DHCP servers and BOOTP relay agents may not be able to
deliver DHCP messages to clients that cannot accept hardware
unicast datagrams before the TCP/IP software is configured.
Presumably you're running this program on a system where the ethernet interfaces have already been configured and have valid IP addresses. I'm not sure why you'd want the source IP address to be 0.0.0.0, but perhaps you could set the interface IP to 0.0.0.0 with ifconfig to get the effect you want.
Or you could use a RAW socket and build the IP and UDP headers yourself to contain anything.
How can I get information about a user's PC connected to my socket
a socket is a "virtual" channel established between to electronic devices through a network (a bunch of wires). the only informations available about a remote host are those published on the network.
the basic informations are those provided in the TCP/IP headers, namely the remote IP address, the size of the receive buffer, and a bunch of useless flags. for any other informations, you will have to request from other services.
a reverse DNS lookup will get you a name associated with the IP address. a traceroute will tell you what is the path to the remote computer (or at least to a machine acting as a gateway/proxy to the remote host). a Geolocation request can give you an approximate location of the remote computer. if the remote host is a server itself accessible to the internet through a registered domain name, a WHOIS request can give you the name of the person in charge of the domain. on a LAN (Local Area Network: a home or enterprise network), an ARP or RARP request will get you a MAC address and many more informations (as much as the network administrator put when they configured the network), possibly the exact location of the computer.
there are many many more informations available, but only if they were published. if you know what you are looking for and where to query those informations, you can be very successful. if the remote host is quite hidden and uses some simple stealth technics (anonymous proxy) you will get nothing relevant.
Look here. See "# Echo server program" section.
conn, addr = s.accept()
print 'Connected by', addr
I am unsure if this is what you are looking for, hth.
You could try asking identd about the connection, but a lot of hosts don't run that or only put up info there that you can't use.