Over the past few weeks I've been coding a program which runs a reddit bot locally from my machine. I've perfected it such that it does not reply to the same comment which it has replied to before, it runs quite efficiently, and in my opinion is complete.
Now, I'm looking for a way to get the program to run on a schedule. I currently have the code in google colab, and I don't know how to use google colab for this functionality.
The program does not require any local storage, it's one code file, and does not require much memory, so I wanted to ask if anyone has a resource which has an detailed tutorial accessible for beginners which I could use to host this code
Note: The code requires an installation of PRAW, in google colab I simply do !pip install PRAW if that means anything differently for what I need to do, what should I do differently?
Thank you in advance.
Google Collab is not designed for this kind of things, and most likely it cannot be used to run your app on schedule.
Probably the easiest solution is some kind of Continuous Integration tool that lets you run code remotely.
Step 1 would be to host your code to some remote code repository like GitHub. Since it most likely won't have to be interactive switching from Collab notebook to simple Python script will make your config much easier later on.
Step 2 would be connecting that repo to some CI tool. One I am familiar with that lets you run pipelines on schedule is CircleCI, with this tutorial here showing very simplistic configuration for running Python scripts from a pipeline.
Related
I have a simple python bot that I want to run non-stop 24/7. I haven't been able to figure out which GCP product would be best for this and how exactly I could set it up to work in this way. Any help or a detailed explanation would be greatly appreciated.
Deploy a VM on Google Cloud, ssh into your VM and put all the relevant code in there, then run your python bot as you normally would
Although not a Google product, you should also consider repl.it:
It allows you to host for free (unlike GCP), 24/7 with uptimerobot constantly pinging it.
But, it is also a security problem. Others can easily see your code
Therefore, I think it is really about "security" and "price" (how confidential is the source code of your bot).
I wanted to share my python program to my friends but the problem is they will have to install python first then all the libraries which I used in order to run my program and it might be hard to do so as I have used too many libraries like 15-20 something.
MY Questions:-
Q1. How can I share my python program without making them install so much stuff?
Q2. Is there any other language on which it could be done?
Thank you.
Regards
Google Colab
You can write your python program in Google Colab and then share the notebook with others to run.
It's free to use and your dependencies can be imported or installed based on the information from this stackoverflow post.
Streamlit
Streamlit allows you to build custom web apps using python that are shareable. It's marketed use is for data science and machine learning python projects. You should check their website to see if it satisfies your specific needs.
repl.it
On the website repl.it you can create public Python projects which can even include PyPI dependencies. The user can then run and edit them, for example here: https://replit.com/#TedTaras/Monster-Hunter. Projects are public by default, private ones cost extra.
I have a script that uses requests library. It is a web scaper that runs for at least 2 days and I don't want to leave my laptop on for that long. So, I wanted to run it on the Cloud but after a lot of trying and reading the documentation, I could not figure out a way to do so.
I just want this: When I run python my_program.py it shows the output on my command line but runs it using Google Cloud services. I already have an account and a project with billing enabled. I already installed the GCP CLI tool and can run it successfully.
My free trial has not ended. I have read quickstart guides but as I am fully beginner regarding the cloud, I don't understand some of the terms.
Please, help me
I think you'll need to setup a Google Cloud Compute Engine instance for that. It's basically a reserved computer/machine where you can run your code. Here's some steps that you should do just to get your program running on the cloud.
Spin up a Compute Engine instance
Gain access to it (through ssh)
Throw your code up there.
Install any dependencies that you may have for you script.
Install tmux and start a tmux session.
Run the script inside that tmux session. Depends on your program, you should be able to see some output inside the session.
Detach it.
Your code is now executing inside that session.
Feel free to disconnect from the Compute Engine instance now and check back later by attaching to the session after connecting back into your instance.
I have been using GPU acceleration sevices like Google Colab for a while, but I am not satisfied. I don't like having to write all my code in Jupyter Notebooks, and I have some other issues too. I am wondering if there is a way to get something set up where i could just run a command from my terminal, something like upload train.py to upload a file train.py to a server, and then later run run train.py or something like that to run it on that server, and to have the output appear in my local terminal. Does anyone know a way to achieve something like this?
.. if there is a way to get something set up where I could just run a command from my terminal, something like upload train.py to upload a file train.py to a server, and then later run run train.py or something like that to run it on that server
If you are talking about running a code on the google colab server with GPU, no.
As I remember they updated their policy and now you can only use the GPU on google Colab via the Colab notebooks. If you have a Linux server with a GPU, you can connect to it via SSH and install Cuda and libraries like tensorflow_gpu or pytorch and run your code.
If you are looking for cheap alternatives for GPU servers, check this and this link.
Heroku is a non-GPU alternative where you can prototype your codes and then use any of the cloud providers such as AWS or Google Cloud. (As I remember AWS provides a decent number of free hours of GPU time at signup).
Then there is another alternative called FloydHub that I have heard people call heroku for deep learning. I haven't used it personally but this might also be what you are looking for.
On a personal note, even though it's not that efficient. I prototype my codes locally and then upload them to my google drive and do the final training on google Colab GPU. It's an unnecessary step, but that's the best I could find without renting a server.
I make this topic as we have a problem uploading a python archive in Azure. We did a modular code with python, which is a text generator. The code is working on python environment, besides we wanted to upload it to the cloud. When we tried it, we used the App service of Microsoft azure, we didn’t have any trouble while we were doing the process this way. Nevertheless, at the end, the program gave us a json, when we tried to put that URL into the navigator… There’s were the problem is, the generated Jason doesn’t connect with the navigator. We don’t have any clue of what’s happening, because the console doesn´t give any warning or error.
If any of you knows or have a tip for us to solve this problem, I would appreciate it,
Thanks for your attention and have a nice day.
Welcome to Stack Overflow! From what I understand, I suppose you should be going with Azure Automation Runbooks instead of Azure App Service, if you intend to execute or automate a task-like job. You can also test your runbook and implement error handling streams to ensure it works as expected. Please refer to this example for a detailed walkthrough.
Please elaborate your requirement if this is not what you were looking for, and we can explore other alternatives.