I'm new to Azure but already have an 8-core machine running there with Ubuntu 12.04 and OpenCV properly installed. Also, i have an executable (C++) that receives an image as argument and prints the result on terminal (in case, if the object was found or not in the image).
So, what i need is to call this executable from my notebook to another C++ code. Basically speaking, I need to run a program on my notebook that call another one on Azure, passing an image as argument. I do not have much experience on this area, so all help is welcome. I heard i need to do this using Python, so how would it be ? And why ?
Thanks!!
You can do it in whatever language you want. The answer to make 2 machines "talk" via Internet is always sockets, can't be done other way.
If I were you I'll do it this way:
A program in Azure listens to a port (server) and when you want your notebook software to process an image, that software will connect to that port and send the image, then the server will process it using OpenCV and return the results to the client if needed so.
Other way can be that the server software queries a database or looks for a directory if there are new images each X seconds, and your client software will upload to that directory the image vía ftp or scp (more secure) or whatever. This way will be more difficult to retrieve the results.
Now, you need to know about language programming to develop this.
Related
I have python scripts for automated trading for currency and I want to deploy them by running on Jupter Lab on a cloud instance. I have no experience with cloud computing or linux, so I have been trying weeks to get into this cloud computing mania, but I found it very difficult to participate in it.
My goal is to set up a full-fledged Python infrastructure on a cloud instance from whichever provider so that I can run my trading bot on the cloud.
I want to set up a cloud instance on whichever provider that has the latest python
installation plus the typically needed scientific packages (such as NumPy and pandas and others) in combination with a password-protected and Secure Sockets Layer (SSL)-encrypted Jupyter
Lab server installation.
So far I have gotten no where. I am currently looking at the digital ocean website for setting jupter lab up but there are so many confusing terms.
What is Ubuntu or Debian? Is it like a sub-variant of Linux operating system? Why do I have only 2 options here? I use neither of the operating system, I use the windows operating system on my laptop and it is also where I developed my python script. Do I need a window server or something?
How can I do this? I tried a lot of tutorials but I just got more confused.
Your question raises several more about what you are trying to accomplish. Are you just trying to run your script on cloud services? Or do you want to schedule a server to spin up and execute your code? Are you running a bot that trades for you? These are just some initial questions after reading your post.
Regarding your specific question regarding Ubuntu and Debian, they are indeed Linux distributions which are popular option for servers. You can set up a Windows server on AWS or another cloud provider, but Linux distributions being much more popular are going to have lots of documentation, articles, stackoverflow posts around a Linux based server.
If you just want to run a script on a cloud on demand, you would probably have a lot of success following Wayne's comment around PythonAnywhere or Google Colab.
If you want your own cloud server, I would suggest starting small and slow with a small or free tier EC2 instance by following a tutorial such as this https://dataschool.com/data-modeling-101/running-jupyter-notebook-on-an-ec2-server/ Alternatively, you could splurge for an AWS AMI which will have much more compute power and be configured.
I have similar problem and the most suiteble solution to me is using docker container for jupyter notebooks. The instructions on how to install Docker can be found at https://docs.docker.com/engine/install/ubuntu/ There is ready to use Docker image docker pull jupyter/datascience-notebook for jupyter notebook python stack. The docker compose files und sone addional insruction you will fid at https://github.com/stefanproell/jupyter-notebook-docker-compose/blob/master/README.md.
I want to upload my a csv file daily on Ambari Apache. I've tried manuplating multiple solutions avaliable online to upload files of Google and other equivalent platforms. I have also tried methods like sftp to help me achieve it, but still have not found a solution. Please recommend any tips, ideas or methods on how should I achieve it.
There is an Ambari method to do this. You can create a custom service in ambari that would run. This would enable you to have ambari self contain the code and execute it. Out of the box Ambari wouldn't technically be running the script, you'd have to run it on a Master/Slave but you might be able to work around that by running an agent on ambari and making it a slave. If it's acceptable you could just have this service installed on one slave and have it push/pull the appropriate file to Ambari.
There are others that have implemented this on just one machine and you can google for how they make sure it's run on just one machine.
I am programming a roboter, and it needs to send data between the EV3 and my laptop (Windows)
I run pybricks on the EV3, which enables me to code in python.
I already did research, but the only things that are remaining are some blogs from 2014 that don't help either.. the only thing that helped a little was the official documentation of pybrick.
I thought that running the example code on the EV3 and laptop would work, but the code only worked on the EV3. The EV3 waits until getting a message, but the laptop instantly says connected, even tough it isn't.
I thought it is maybe possible to get the laptop to act like an EV3 to connect them (because the original message function for EV3 is only made for interaction between different bricks), but my knowledge kinda ends here, even tough I tried a few things, like a virtual box.. maybe I did something wrong, but I hadn't had good results
If I understand correctly, you want to,
send a value to a remote machine
have the remote machine do some stuff
get a response from the remote machine
If that is correct, this is normally achieved by API. The remote machine serves an API, which the client calls using the requests library. For example, TheCatAPI allows you to call for pictures of cats. Your laptop and EV3 are still logically distinct, even though they are physically nearby.
You can choose to host the API on your laptop or make it available to others using services like AWS. Avoiding copypasta, Real Python has some fantastic docs on how to build an API.
You will want to connect your EV3 to WiFi then tell it to call your new API. If you set this up in AWS lambda, others will be able to do the same and it will probably be free for you.
If you want to do it purely locally, you will need to connect your EV3 and laptop via Bluetooth and establish a serial link.
Personal opinion
I would suggest that showcasing your ability to create an API and get it working on AWS would be more beneficial than going down the Bluetooth route.
I have an application running on linux server and I need to create a local backup of it's data.
However, new data is being added to the application after every hour and I want to sync my local backup data with server's data.
I want to write a script (shell or python) that can automatically download new added data from the linux server to my local machine backup. But I am newbie to the linux envoirnment and don't know how to write shell script to achieve this.
What is the better way of achieving this ? And what would be the script to do so ?
rsync -r fits in your use case and it's a single line command.
rsync -r source destination
or the options you need according to your specific case.
So, you don't need a python script for that, but you can still write it and let it use the command above.
Moreover, if you want the Python script to do it in an automatic way, you may check the event scheduler module.
This depends on where and how your data is stored on the Linux server, but you could write a network application which pushes the data to a client and the client saves the data on the local machine. You can use sockets for that.
If the data is available via aan http server and you know how to write RESTful APIs, you could use that as well and make a task run on your local machine every hour which calls the REST API and handles its (JSON) data. Keep in mind that you need to secure the API if the server is running online and not in the same LAN.
You could also write a small application which downloads the files every hour from the server over FTP (if you want to backup files stored on the system). You will need to know the exact path of the file(s) to do this though.
All solutions above are meant for Python programming. Using a shell script is possible, but a little more complicated. I would use Python for this kind of tasks, as you have a lot of network related libraries available (ftp, socket, http clients, simple http servers, WSGI libraries, etc.)
I'm working on an embedded device that runs Linux on ARM7 with 64MB RAM and 64MB storage (12MB free). The device should be configured via web therefore it needs to run an embedded web server. Currently it's using Lighttpd and LUA, but I'm thinking about replacing LUA (or maybe even Lighttpd) with Python. The server will occasionally be accessed by one or two users for making changes to internal settings of the C program that is running in Linux. So the server load isn't really a lot. I also need it to be Open Source Software. Web.py seems to be small enough but I still need to compile Python which I haven't done before. So I'm wondering what are the system requirements of Python? LUA seems to do quite well for small embedded systems but I don't like its syntax for C-binding.
However, I couldn't find updated information about system requirements for embedding Python in such settings. This page from Michael Lauer seems to be old.
Any ideas? Suggestions? hints? links?
I'm working on this device using OpenWRT + Python:
http://wiki.openwrt.org/oldwiki/OpenWrtDocs/Hardware/Meraki/Mini
The first python run is veeeeeery slow but it are metacompiling all .pyc files, next it work well.