I'm trying to run a simple FTPServer on EC2. My client connects to it fine, and I can make and remove directories. However, I get the following error whenever I try to store or retrieve a file: Failure: twisted.protocols.ftp.CommandFailed: ["425 Can't open data connection."]
I've tested the server on my localhost, and everything behaves as normal. I was paranoid and decided to turn on all ports in my security groups, but it still doesn't work. Am I missing something when setting up my EC2 machine?
Your FTP client appears to be in active mode, and is (like most client computers these days) behind a NAT or firewall which does not permit inbound connections.
There's another question on Stack Overflow about this topic that quite clearly explains the differences between active and passive mode in FTP; you should read it.
Configure your client to use passive mode, or "PASV", and it ought to work.
Related
So first of all, what I really want to achieve: I want to know when an IoT device has stopped working (i.e. lost connection, shut down, basically it's not longer talking to IoT Core). I can't seem to find an implementation for this on GCP.
I have a raspberry pi as my IoT device, I have configured it on IoT core and somewhere I read that since this is not implemented a way to solve it is to create a logging sink which activates a cloud function whenever there is a CONNECT/DISCONNECT log. This would serve my purpose and I have implemented this sink and cloud function to alert me.
I have been following this guide on connecting to MQTT. However, the way the explain it, they set it up such that whenever the expiration time on the JWT is exceeded, they disconnect the client and create a new one to re-new the JWT. This would make it such that I am going to be alerted of connection/disconnection whenever this client needs to be renewed. So I won't be able to differentiate of a real issue from renewals of the MQTT client.
In the same guide, I see that they mention MQTT long term or LTS, and they claim that this way you can set up the client once and communicate continuously through it for the supported time which it says its until 2030. This seems to be what I really want, but I have not been able to connect this way and they don't explain it other than saying the hostname should be mqtt.2030.ltsapis.goog and to use a primary and backup certificates which are different from the complete root CA from the first method.
I tried using basically the same process for setting up the client:
client = mqtt.Client(client_id=client_id)
# With Google Cloud IoT Core, the username field is ignored, and the
# password field is used to transmit a JWT to authorize the device.
client.username_pw_set(
username='unused',
password=create_jwt(project_id, private_key_file, algorithm))
# Enable SSL/TLS support.
client.tls_set(ca_certs=ca_certs, tls_version=ssl.PROTOCOL_TLSv1_2)
but changing the hostname and giving it the primary cert where I would give it the complete ca_certs, but it won't accept it and I am not sure how to do it otherwise with primary and backup certifications. I am looking at the documentation on tls_set, but I don't see where these would go or how they differ from the complete ca certs. I haven't seen any other examples outside of this guide.
I am hoping to be able to connect to this MQTT LTS so that I can maintain the connection without having to constantly renew the client.
The long term MQTT domain lets you use the LTS configuration for a long period of time, not the connection.
As you mention, for your use case the solution would be to activate and use device logs. One of the events is triggered when a device disconnects from IoT Core, and you can use that event to trigger an alert.
Keep in mind that the time limits for the connection are set for security purposes, and the client should renew the connection.
I have a bit of an open ended questions for you all. I wish to create a simple chat-room such as this example here: https://www.geeksforgeeks.org/simple-chat-room-using-python/ but I am lost as how to do it over the internet rather than just local network.
Any pointers/help would be appricated!
Thanks :)
There are multiple ways about this. You can either:
Run locally and expose your Python chat system to the internet.
Run your Python chat system in some online server provider (Heroku, AWS, etc.).
The first method requires you to do some port-forwarding on your local network, essentially mapping your 127.0.0.1:8081 local server to your public IP (so you would connect via the internet as myip:8081). This method, however, comes with its limitations; when you turn off your computer you are also effectively turning off your server to the rest of the internet. The second method will ensure the server stays on at all times, and is likely what you are looking for. Heroku is a great starting point as they provide a free tier that you can test everything out.
Hey I was wondering if anyone knew how to connect to a bitcoin wallet located on another server with bitcoinrpc
I am running a web program made in django and using a python library called bitcoinrpc to make connections.
When testing locally, I can use bitcoinrpc.connect_to_local), or even bitcoinrpc.connect_to_remote('account','password') and this works as well as long as the account and password match the values specified in my 'bitcoin.conf' file. I can then use the connection object to get values and do some tasks in my django site.
The third parameter in connect_to_local is default localhost. I was wondering:
A) What to specify for this third parameter in order to connect from my webserver to the wallet stored on my home comp (is it my IP address?)
B) Because the wallet is on my PC and not some dedicated server, does that mean that my IP will change and I won't be able to access the wallet?
C) The connection string is in the django app - which is hosted on heroku. Heroku apps are launched by pushing with git but I believe it is to a private repository. Still, if anyone could see the first few lines of my 'view' they would have all they need to take my BTC (or, more accurately, mBTC). Anyone know how bad this is - or any ways to go about doing btc payments/movements in a more secure way.
Thanks a lot.
I'm currently doing something very similar (heroku using express/nodejs instead of django/python tho) so I will try to share my thoughts.
In spite of using other library and other language, all the wallet remote libraries should be primarily a wrapper around JSON RPC (remote procedure call) API, which is actually the same for most of the coins out there (i would say all, but that would be a wild guess).
Specifically to your questions:
A)
To access the wallet from outside, use your external ip (fastest way to find it is to query google for it). Depending on your ISP you hopefully have static external address. You must provide this address to bitcoin.conf file under rpcallowip= option to allow incomming connections.
Moreover you should forward the used port in your home router (usually under NAT settings) to your local machine so the incoming connection from the server is allowed and redirected to your wallet computer.
There is one important thing to consider (https://en.bitcoin.it/wiki/Running_Bitcoin):
By default, only RPC connections from localhost are allowed. Specify
as many rpcallowip= settings as you like to allow connections from
other hosts (and you may use * as a wildcard character).
NOTE: opening up the RPC port to hosts outside your local
trusted network is NOT RECOMMENDED, because the rpcpassword
is transmitted over the network unencrypted.
I am yet to look into it further, from this comment alone it seems totally unusable for monetary transactions.
B)
As I said before, it depends on your home ISP, type of connection and the service provided to you.
C)
If I understand correctly from a django point of view, as long as the login parameters (username/password) are inside a view (views.py of your app) and the debug mode is turned off, source code of the server should not be publicly accessible. But the security concern from A still applies.
You can use SSL with RPC to hide the password.
rpcssl=1
What's the easiest way to establish an emulated TCP connection over HTTP with python 2.7.x?
Server: a python program on pythonanywhere (or some analogue) free hosting, that doesn't provide a dedicated ip. Client: a python program on a Windows PC.
Connection is established via multiprocessing.BaseManager and works fine when testing both server and client on the same machine.
Is there a way to make this work over HTTP with minimal additions to the code?
P.S. I need this for a grid computing project.
P.P.S. I'm new to python & network & web programming, started studying it several days ago.
Found this: http://code.activestate.com/recipes/577643-transparent-http-tunnel-for-python-sockets-to-be-u/. Appears to be exactly what I need, though I don't understand how to invoke setup_http_proxy() on server/client side. Tried setup_http_proxy("my.proxy", 8080) on both sides, but it didn't work.
Also found this: http://docs.python.org/2/library/httplib.html. What does the HTTPConnection.set_tunnel method actually do? Can I use it to solve the problem in question?
Usage on the client:
setup_http_proxy("THE_ADRESS", THE_PORT_NUMBER) # address of the Proxy, port the Proxy is listening on
The code wraps sockets to perform an initial HTTP CONNECT request to the proxy setup to get an HTTP Proxy to proxy the TCP connection for you but for that you'll need a compliant proxy (most won't allow you to open TCP connections unless it's for HTTPS).
HTTPConnection.set_tunnel basically does the same thing.
For your use case, a program running on free hosting, this just won't work. Your free host probably will only allow you to handle http requests, not have long running processes listen for tcp connections(which the code assumes).
You should rethink your need to tunnel and organize your communication to post data (and poll for messages from the server, unless they're answers to the stuff you post). Or you can purchase a VPS hosting that will give you more control over what you can host remotely.
Important note:
I've asked this question already on ServerFault: https://serverfault.com/questions/349065/clustering-tcp-servers-so-can-send-data-to-all-clients, but I'd also like a programmers perspective on the problem.
I'm developing a real-time mobile app by setting up a TCP connection between the app and server backend. Each user can send messages to all other users.
(I'm making the TCP server in Python with Twisted, am creating my own 'protocol' for communication between the app/backend and hosting it on Amazon Web Services.)
Currently I'm trying to make the backend scalable (and reliable). As far as I can tell, the system could cope with more users by upgrading to a bigger server (which could become rather limiting), or by adding new servers in a cluster configuration - i.e. having several servers sitting behind a load balancer, probably with 1 database they all access.
I have sketched out the rough architecture of this:
However what if the Red user sends a message to all other connected users? Red's server has a TCP connection with Red, but not with Green.
I can think of a one way to deal with this problem:
Each server could have an open TCP (or SSL) connection with each other server. When one server wants to send a message to all users it simply passes this along it's connection to the other servers. A record could be kept in the database of which servers are online (and their IP address), and one of the servers could be a boss - i.e. decides if others are up and running, if not it could remove them from the database (if a server was up and lost it's connection to the boss it could check the database and see if it had been removed, and restart if it had - else it could assume the boss was down.)
Clearly this needs refinement but shows the general principle.
Alternatively I'm not sure if this is possible (- definitely seems like wishful thinking on my part):
Perhaps users could just connect to a box or router, and all servers could message all users through it?
If you know how to cluster TCP servers effectively, or a design pattern that provides a solution, or have any comments at all, then I would be very grateful. Thank you :-)
You need to decide (or if you already did this - to share these decisions with us) reliability requirements for your system: should all messages be sent to all users in any case (e.g. one or more servers crashed), can you tolerate sending the same message twice to the same user on server crash? Your system complexity depends directly on these decisions.
The simplest version is when a message is not delivered to all users on server crash. All your servers keep TCP connection to each other. One of them receives a message from a user and sends it to all other connected users (to this server) and to all other connected servers. Other servers send this message to all their users. To scale the system you just run additional server which connects to all existing servers.
Have a look how it is handled with IRC servers. They essentially can do this already. Everbody can send to everybody else, on all servers. Or just to single users, also on another server. And to groups, called "channels". It works best by routing amongst the servers.
It's not that hard, if you can make sure the servers know each other and can talk to each other.
On a side note: At 9/11, the most reliable internet news source was the IRC network. All the www sites were down because of bandwidth; it took them ages to even get a plain-text web page back up. During this time, IRC networks were able to provide near real-time, moderated news channels across the atlantic. You maybe could no longer log into a server on the other side, but at least the servers were able to keep up a server-to-server connection across.
An obvious choice is to use the DB as a clearinghouse for messages. You have to store incoming messages somewhere anyway, lest they be lost if a server suddenly crashes. Put incoming messages into the central database and have notification processes on the TCP servers grab the messages and send them to the correct users.
TCP server cannot be clustered, the snapshot you put here is a classic HTTP server example.
Since the device will send TCP connection to server, say, pure socket, there will be noway of establishing a load-balancing server.