We Keep Coding

Connecting Client/Server program outside of LAN with Python

I am trying to create a chess game between two different computers that are not in the same LAN. I am having trouble connecting the two via a TCP connection (UDP would probably be sufficient as well if the packets are arriving, but ideally TCP).
I am new to a lot of networking and am unaware of many different tools that may be useful and I am also in university and therefore don't have control over the router to update firewall rules. What can I do to work around the router firewall to connect the two devices.
I am primarily using the Python socket library at the moment to implement the connection.
Any information about how I can send messages between the two computers outside of a LAN would be very useful. Thank you for your help!
I have ensured that the client side is using the public IP of the server and the server is using "" for its socket host. I also checked that the connection was working when utilizing a LAN without issue. I included a batch file that enables the specific port used for the game at the beginning of runtime and disables it at the end of the program. If I am not mistaken, that only impacts the computer's firewall rules not the router's. I have looked into receive the packets through port 80 and redirecting it to my specific program, but was unsuccesful in finding a solution of that type.

If the server is behind a router/firewall you'll have to use some sort of hole punching method to create the connection. STUN is one of the most common, though I've never actually used it in a Python program so I don't know what Python implementations are out there.

Socket over internet Python Embedded

I have an embedded system on which I can connect to internet. This embedded system must send sensor data to PC client.
I put a socket client using python on my PC. I put a socket server ( using C++ language on the embedded system because you can only use C++ ).
I can succesfully connect from my PC to the embedded system using the sockets and send and recieve whatever I want.
Now, the problem is I use local IP to connect to the system and both of them must be connected to the same Wifi router.
In the real application, I won't know where the embedded system is in the world. I need to get to it through internet, because it will be connectet to internet through 4g.
My question is, how can I connect to it through internet, if the embedded system is connected to internet using 4G?
Thank you

Realistically in typical situations, neither a PC nor an embedded device hanging off a 4g modem will likely have (or should be allowed) to have externally routable addresses.
What this practically means is that you need to bounce your traffic through a mutually visible relay in the cloud.
One very common way of doing that for IoT devices (which is basically to say, connected embedded devices) is to use MQTT. You'll find support in one form or another for most computing platforms with any sort of IP networking capability.
Of course there are many other schemes, too - you can do something with a RESTful API, or websockets (perhaps as an alternate mode of an MQTT broker), or various proprietary IoT solutions offered by the big cloud platforms.
It's also going to be really key that you wrap the traffic in SSL, so you'll need support for that in your embedded device, too. And you'll have to think about which CA certs you package, and what you do about time given its formal requirement as an input to SSL validation.

I think your problem is more easily solved if you reverse the roles of your embedded system and PC. If you are communicating to a device using IP protocols across cellular networks, it is much easier to have the device connect to a server on the PC rather than the other way around. Some networks/cellular modems do not allow server sockets and in any case, the IP address is usually dynamically allocated and therefore difficult to know. By having the device connect to a server, it "knows" the domain name (or IP address) and port to which it should make the connection. You just have to make sure that there is indeed a server program running at that host bound to some agreed upon port number. You can wake up the device to form the connection based on a number of criteria, e.g. time or amount of collected data, etc.

How can I connect front-end with back-end?

I have a web page (front-end) created with ASP.NET (visual studio) on one laptop and a server (back-end) for my web page on another laptop. In the first place, I have a login page and I want to take the username and the password from textBox-es (front-end) and somehow transfer the data to the other laptop to verify if it's correct and then receive if the data it's correct or not. The front-end is on Windows and the server is on Linux. My teammate is using python on back-end.
I'm working with HTML, CSS and a little bit of C# on the front-end and I don't know how to transfer that username and password to my teammate and how to receive an answer from him.
He has the database on his laptop and he's the one who will verify if the pair is correct or not and send me an answer, also he wants to code in python.
We think that we can do some magic connection between ASP.NET and Node.js.
How do I do this data transfer ?
It's my first time doing this so please be gentle because I really don't know what to do.

Your question is quite broad and while I am not that fluent in web applications I will share what advice I can. Because the question is broad, you cannot get an exact, short and concise, answer.
Some clarifications and feedback regarding your question.
I have a web page (front-end) created with ASP.NET ...
You actually have a web application here (that, in your case, includes one, or more, web pages). Give more attention to such little details, programming in itself is a form of grammar natzy-ism but it’s not grammar and you are not being corrected by a person, but by machines and life. A nice definition of web application: https://blog.stackpath.com/web-application/
We think that we can do some magic connection between ASP.NET and Node.js.
I am not specialized in web application but it appears to me that this line comes completely out of the blue, you have not indicated that you are using or intend to use Node.js in your back-end; you indicated that you were using Python, so I will ignore this statement. Oh and there is no magic, there are only race conditions.
Structurally speaking, I would split this question into several smaller ones. Some suggestions follow.
Because you are attempting a somewhat more complicated task you cannot escape de idea of a design pattern (especially because of using ASP.NET which is designed to be patterning friendly).
So your first question should be similar to
how to communicate with a web server asp.net
Two-way communication between ASP.NET Web App and C# Application
back-end is not python but read the answer, it gives you a valid indicator, that request-response pattern could help you
search for questions regarding how to implement that pattern in C#/ASP.NET
Another possible way of how to-ing would be
how to transmit data to server C#
a bit old but reading might give you an understanding of how the ecosystem works
C# server and client communication - send/receive data
maybe helps?
https://codingvision.net/networking/c-sending-data-using-get-or-post
https://www.csharp-examples.net/socket-send-receive/
From you explication I presume that you already have implemented the UI forms that will retrieve the input data so I will skip UI related questions.
Since you wish to send user SENSITIVE data over the internet then you MUST think about security. Further questions
how to store password in database securely
this question will actually lead you to examples of transmitting the password and maybe even receiving the content
https://www.meziantou.net/how-to-store-a-password-in-a-web-application.htm
How to store passwords in database securely?
There are 2 more issues I wish to discuss.
While searching the internet for tutorials you will probably see a
lot working with client and server applications on the same machine.
Those are still valid tutorials for you! Read them! I will explain
further down why it's (almost) the same for you even if your case is
a remote server.
You need to somehow be able to test your application.
Addressing #1
Why working with localhost (127.0.0.1) is for you the same as working with a real server (in your case).
Let's say you finish the application, the back-end is also made and you wish to communicate with one another. You read on the internet that you need to known the IP of the server so you ask your partner to for him to give his IP. You search the wild wild web and reach the conclusion that to find your IP you would try:
executing ipconfig.exe (or what tool you use depending on your platform) and getting the IPv4 Address. . . . . . . . . . . : 192.168.1.12 value.
you would presume that that is the IP you need to put in your application to connect to. This is FALSE.
reaching the conclusion that that does not work, you find that you can get your IP with online sites such as https://whatismyipaddress.com/
and use that IP for your server address. This (in 99% cases) is FALSE.
A bit of poetry
Consider IP as you address in the world and your local (private) IP as the exact place in your house. The IP address you see there is a local IP and it would indicate in which chair you are sitting in, let’s say the 12 from the already provided example. Anyone in the world (who has 12 chairs anyway) can sit in the 12th chair in their house. You cannot use this information to identify the address of someone.
How would you know that that IP is a local IP?
One way would be to look at the the full output of ipconfig.exe (or whatever tool you are using) where you see a Wireless LAN adapter Wi-Fi: above from where you got your IP, the LAN there (hxxps://en.wikipedia.org/wiki/Local_area_network) is an indicator of that.
The other (more straightforward) is by recognizing the IP range (or visual pattern, if IP range is not a clear concept for you just yet). If it starts with 198.168. then it's a local/private IP address. Please read here more: https://en.wikipedia.org/wiki/Private_network
IPs are a valuable commodity (I joke you not). As the world has evolved you cannot now simply plug your computer with an internet cable, and voila you have your own IP. This was (relatively) true when the internet first started decades ago but now the IPs are managed by big (huge) companies that they allocate them as they consider.
The IP address space is managed globally by the Internet Assigned Numbers Authority (IANA), and by five regional Internet registries (RIRs) responsible in their designated territories for assignment to local Internet registries, such as Internet service providers, and other end users.
Wiki quote: https://en.wikipedia.org/wiki/IP_address
I also highly (please do) recommend you read the following answer
hxxps://superuser.com/questions/323801/how-can-i-own-an-ip-address
If you ask your (real world) neighbors to look up https://whatismyipaddress.com/ (and they have the same ISP and monthly plan as you) there is a strong possibility that they will see the same IP as you see. To put it simply, the IP you see there, when accessing sites such as whatismyipaddress, is the IP that that your ISP has assigned to the network node you are in. It would be a waste of IPs to assign each user an IP (not to mention there actually aren’t mathematically that many IPs to do so).
Regarding your current predicament. I can see 3 options.
Option 1 - buy hosting
If your response is "I really really wanna have a separate IP" (or domain) then you will need to buy from a hosting service an IP (the good kind) that will point to a device the hosting company has. You usually buy domains that point to IPs, but let's not got there for now. You will connect to that machine/devices and install your code on the server there. Then, and only then, you could get the IP/domain of that machine and use it in your code and it will work (this is the 1% cases where sites like whatismyipaddress will actually help you).
Recommended searches: server hosting cheap :D (or something free for students if you are students)
Option 2 - work locally
If your project is used for academia and there is no actual need for a separate server then you and your partner will run the client/server applications while connected to the same wireless modem. Ask him to get his IP in this way (here you actually need the LAN IP), you hardcoded it in your code (preferably a separate config file, this way no extra compilation is required) and voila, it works.
Option 3 - LAN over the internet
Technically this would be hard to describe but the idea is you use programs that communicate over the internet, you each install it on your computer, somehow connect with each other (usually a name/password pair) and that software make is seem for the programs on your computer that you are actually in a LAN setup (Option 2). In my days I used hamachi so that we could play CS online :D.
hxxps://www.howtogeek.com/172762/how-to-play-old-lan-games-over-the-internet/
Regarding #2
Until your partner provides a server you will need to test your code. So you will need to somehow fake a server response so that you can actually continue development.
Quite simply, the operation of faking something, a return/a class, in your code, so that you can correctly develop and test your code is known as mocking
how to mock server response asp.net
hxxps://gingter.org/2018/07/26/how-to-mock-httpclient-in-your-net-c-unit-tests/
For beginning programmers I think this is one of the most complicated parts. The concept of mocks and tests, while they do appear simple, are usually not engraved into the mind of young, innocent, previously untainted, future programmers. They are left stuck with printfs until graduations (even a dose of hate towards tests is sprinkled in the mix, what a pity).
As a courtesy, I wish to further extend that madness. Please, for all that is good and sacred in this world, do not use this code in production, burn it after you used it to understand some of the basics, BURN IT!
I modified the code from here Python 3.x BaseHTTPServer or http.server so that you could run a local HTTP server on your machine and test your application. You simply run it (python3) and test the sending of user name and password to the server using the following pagan syntax:
127.0.0.1:4242/?user=aba&password=Aba1 which would return I'al cohol you later
#!/usr/bin/python
from http.server import BaseHTTPRequestHandler,HTTPServer
from urllib.parse import urlparse, parse_qs
HOST = '127.0.0.1' # Standard loopback interface address (localhost)
PORT_NUMBER = 4242 # Port to listen on (non-privileged ports are > 1023)
# !!!!!!!!!!!!!!! THIS IS ONLY FOR DIDACTICAL PURPOSE< NEVER EVER DO SOMETHING LIKE THIS IN PRODUCTION!!!!!!!
# NEVER EVER STORE PASSWORDS LIKE THIS !!!!!!!
# LITERRLAY NEVER!!!!!!
data_base = [
{
'user': 'ionzapada',
'password': 'neaomat',
'data': b"I like pie!"
},
{
'user': 'aba',
'password': 'Aba1',
'data': b"I'al cohol you later"
}
]
class myHandler(BaseHTTPRequestHandler):
#Handler for the GET requests
def do_GET(self):
self.send_response(200)
self.send_header('Content-type','text/html')
self.end_headers()
# Send the html message
query_components = parse_qs(urlparse(self.path).query)
# DO NOT USE THIS LOGIC ON ANY PRODUCTION PLATFORM!!!!
# NEVER SEND PASSWORDS THIS WAY!!!
# NEVER SEND PASSWORDS!!!!
# NEVER SEND HASHES WITHOUT SALT!!!!
# NEVER IMPLEMENT YOUR OWN TRICKY ALGORITHM!!!!
user_name = query_components.get("user",[None])[0]
password = query_components.get("password",[None])[0]
if user_name and password:
for data_base_entry in data_base:
if data_base_entry['user'] == user_name:
print('Found searched user: {}'.format(user_name))
if data_base_entry['password'] == password:
print('Valid password for user {} given; returning db data'.format(user_name))
self.wfile.write(data_base_entry['data'])
else:
self.wfile.write(b"Not supported!")
else:
self.wfile.write(b"Not supported!")
return
try:
server = HTTPServer((HOST, PORT_NUMBER), myHandler)
print('Started httpserver on port', PORT_NUMBER)
server.serve_forever()
except KeyboardInterrupt:
print('CTRL+C received, shutting down the web server')
server.socket.close()
I burned my finger writing that, do not ever send user names and passwords in this way!!! DO NOT EVEN SEND PASSWORDS FOR THE LOVE OF GOD!!!!!!!! search for standard and secure ways of doing this!!!!
Please also visit the links indicated by #amc they are an also good start.
P.S. I am not allowed to have more then 8 link in one post, on some links that I have posted change the hxxp to http for them to work.

How to make a port forward rule in Python 3 in windows? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Purpose:
I'm making a program that will set up a dedicated server (software made by game devs) for a game with minimal effort. One common step in making the server functional is port forwarding by making a port forward rule on a router.
Me and my friends have been port forwarding through conventional means for many years with mixed results. As such I am hoping to build a function that will forward a port on a router when given the internal ip of the router, the internal ip of the current computer,the port and the protocol. I have looked for solutions for similar problems, but I found the solutions difficult to understand since i'm not really familiar with the socket module. I would prefer not to use any programs that are not generally installed on windows since I plan to have this function work on systems other than my own.
Approaches I have explored:
Creating a bat file that issues commands by means of netsh, then running the bat.
Making additions to the settings in a router found under Network -> Network Infrastructure (I do not know how to access these settings programmaticly).
(I'm aware programs such as GameRanger do this)
Using the Socket Module.
If anyone can shed some light how I can accomplish any of the above approaches or give me some insight on how I can approach this problem another way I would greatly appreciate it.
Thank you.
Edit: Purpose

You should read first some sort of informations about UPnP (Router Port-Forwarding) and that it's normally disabled.
Dependent of your needs, you could also try a look at ssh reverse tunnels and at ssh at all, as it can solve many problems.
But you will see that working with windows and things like adavanced network things is a bad idea.
At least you should use cygwin.
And when you really interessted in network traffic at all, wireshark should be installed.

I'm not sure if that's possible, as much as I know, ports aren't actually a thing their just some abstraction convention made by protocols today and supported by your operating system that allows you to have multiple connections per one machine,
now sockets are basically some object provided to you by the operating system that implements some protocol stack and allows you to communicate with other systems, the API provides you some very nice API called the socket API which allows you use it's functionality in order to communicate with other computers, Port forwarding is not an actual thing, it just means that when the operating system of the router when receiving incoming packets that are destined to some port it will drop them if the port is not open, think of your router as some bouncer or doorman, standing in the entrance of a building, the building is your LAN, your apartment is your machine and rooms within your apartment are ports, some package or mail arrives to your doorman under the port X, a port rule means on IP Y and Port X of the router -> forward to IP Z and port A of some computer within the LAN ( provides and implements the NAT/PAT ) so what happens if we'll go back to my analogy is something such as this: doorman receives mail destined to some port, and checks if that port is open, if not it drops the mail if it is it allows it to go to some room within some apartment.. (sounds complex I know apologize) my point is, every router chooses to implement port rules or port blocking a little bit different and there is no standard protocol for doing, socket is some object that allows you program to communicate with others, you could create some server - client with sockets but that means that you'll need to create or program your router, and I'm not sure if that's possible,
what you COULD do is:
every router provides some http client ( web client ) that is used to create and forward ports, maybe if you read about your router you could get access to that client and write some python http script that forwards ports automatically
another point I've forgot is that you need to make sure you're own firewall isn't blocking ports, but there's no need for sockets / python to do so, just manually config it

Clustering TCP servers, so can send data to all clients

Important note:
I've asked this question already on ServerFault: https://serverfault.com/questions/349065/clustering-tcp-servers-so-can-send-data-to-all-clients, but I'd also like a programmers perspective on the problem.
I'm developing a real-time mobile app by setting up a TCP connection between the app and server backend. Each user can send messages to all other users.
(I'm making the TCP server in Python with Twisted, am creating my own 'protocol' for communication between the app/backend and hosting it on Amazon Web Services.)
Currently I'm trying to make the backend scalable (and reliable). As far as I can tell, the system could cope with more users by upgrading to a bigger server (which could become rather limiting), or by adding new servers in a cluster configuration - i.e. having several servers sitting behind a load balancer, probably with 1 database they all access.
I have sketched out the rough architecture of this:
However what if the Red user sends a message to all other connected users? Red's server has a TCP connection with Red, but not with Green.
I can think of a one way to deal with this problem:
Each server could have an open TCP (or SSL) connection with each other server. When one server wants to send a message to all users it simply passes this along it's connection to the other servers. A record could be kept in the database of which servers are online (and their IP address), and one of the servers could be a boss - i.e. decides if others are up and running, if not it could remove them from the database (if a server was up and lost it's connection to the boss it could check the database and see if it had been removed, and restart if it had - else it could assume the boss was down.)
Clearly this needs refinement but shows the general principle.
Alternatively I'm not sure if this is possible (- definitely seems like wishful thinking on my part):
Perhaps users could just connect to a box or router, and all servers could message all users through it?
If you know how to cluster TCP servers effectively, or a design pattern that provides a solution, or have any comments at all, then I would be very grateful. Thank you :-)

You need to decide (or if you already did this - to share these decisions with us) reliability requirements for your system: should all messages be sent to all users in any case (e.g. one or more servers crashed), can you tolerate sending the same message twice to the same user on server crash? Your system complexity depends directly on these decisions.
The simplest version is when a message is not delivered to all users on server crash. All your servers keep TCP connection to each other. One of them receives a message from a user and sends it to all other connected users (to this server) and to all other connected servers. Other servers send this message to all their users. To scale the system you just run additional server which connects to all existing servers.

Have a look how it is handled with IRC servers. They essentially can do this already. Everbody can send to everybody else, on all servers. Or just to single users, also on another server. And to groups, called "channels". It works best by routing amongst the servers.
It's not that hard, if you can make sure the servers know each other and can talk to each other.
On a side note: At 9/11, the most reliable internet news source was the IRC network. All the www sites were down because of bandwidth; it took them ages to even get a plain-text web page back up. During this time, IRC networks were able to provide near real-time, moderated news channels across the atlantic. You maybe could no longer log into a server on the other side, but at least the servers were able to keep up a server-to-server connection across.

An obvious choice is to use the DB as a clearinghouse for messages. You have to store incoming messages somewhere anyway, lest they be lost if a server suddenly crashes. Put incoming messages into the central database and have notification processes on the TCP servers grab the messages and send them to the correct users.

TCP server cannot be clustered, the snapshot you put here is a classic HTTP server example.
Since the device will send TCP connection to server, say, pure socket, there will be noway of establishing a load-balancing server.