i'm trying to connect pycharm to docker remote interpreter, but the container is part of a cluster (specifically using AWS ECS cluster)
i can not access the container directly i have to go through the bastion machine (the instance that is running the container do not have public IP)
i.e to access the container i need to access it via bastion machine
i had an idea of using ssh tunneling but i could not figure out how to do so using pycharm docker utility
is there any solution that pycharm suggests for that?
Related
Is it possible, using the docker SDK for Python, to launch a container in a remote machine?
import docker
client = docker.from_env()
client.containers.run("bfirsh/reticulate-splines", detach=True)
# I'd like to run this container ^^^ in a machine that I have ssh access to.
Going through the documentation it seems like this type of management is out of scope for said SDK, so searching online I got hints that the kubernetes client for Python could be of help, but don't know where to begin.
It's not clearly documented by Docker SDK for Python, but you can use SSH to connect to Docker daemon by specifying host with ssh://[user]#[host] format, for example:
import docker
# Create a client connecting to Docker daemon via SSH
client = docker.DockerClient(base_url="ssh://username#your_host")
It's also possible to set environment variable DOCKER_HOST=ssh://username#your_host and use the example your provided which use current environment to create client, for example:
import docker
import os
os.environ["DOCKER_HOST"] = "ssh://username#your_host"
# Or use `export DOCKER_HOST=ssh://username#your_host` before running Python
client = docker.from_env()
Note: as specified in the question, this is considering you have SSH access to target host. You can test with
ssh username#your_host
It's possible, simply do this:
client = docker.DockerClient(base_url=your_remote_docker_url)
Here's the document I found related to this:
https://docker-py.readthedocs.io/en/stable/client.html#client-reference
If you only have SSH access to it, there is an use_ssh_client option
If you have a k8s cluster, you do not need the Python sdk. You only need the cmd line tool kubectl.
Once you have it installed, you can create a deployment that will deploy your image.
I need to run a Python program on a Microsoft Azure virtual machine (VM), because it uses a significant amount of memory (over 12 GB), so I need to run it in a virtual machine in the cloud, which would provide enough memory.
However, I didn't seem to find an option in the virtual machine's dashboard which lets me run a Python program on it?
You can RDP or SSH into your Azure virtual machine, then you can install the application or execute the scripts as on the traditional VM as you would do.
You'll use the Connect button in the Azure portal to start a Remote Desktop (RDP) session from a Windows desktop. Read https://learn.microsoft.com/en-us/azure/virtual-machines/windows/connect-logon
Alternatively, you will create and use an SSH RSA public-private key file pair for SSH client connections. Read https://learn.microsoft.com/en-us/azure/virtual-machines/linux/create-ssh-keys-detailed
Furthermore, you can use an Azure network security group to filter network traffic to and from Azure virtual machines in an Azure virtual network. Read security rules and how to open ports to a virtual machine with the Azure portal.
As mentioned in the comments, for VMs you will not find any option to install software from Azure Portal. This is something you would need to do by connecting to your Virtual Machine.
You can connect to your VMs using multiple ways: Remote Desktop Connection, SSH, and Bastion etc. You will see all of these options in Azure Portal in the overview blade for your VM. Please see this link as an example of how to connect to your Windows VM using RDP: https://learn.microsoft.com/en-us/azure/virtual-machines/windows/connect-logon.
Once you're connected to your VM, you can install software (python in your case). Installation mechanism would depend on the VM Operating System. Next you will need to copy your application code/binaries in the VM and run it there while still connected to your VM.
I'm new to docker. I have deployed a python server in a docker container. And I'm able to access using my python application from my machine using virtual machine IP(192.168.99.100).
Ex: http://192.168.99.100:5000
How do I access my the application from the other machine which is in the same network?
I tried giving my machine IP but didn't work.
I run the application using "docker run -p 5000:80 myPythonApp"
An easy way out would be to forward port from your host machine to the virtual machine.
Solution may differ w.r.t the VM providers & Host OS that you use. Like for vagrant you can do something as below -
https://www.vagrantup.com/docs/networking/forwarded_ports.html
I want to use a Python Docker image running on a powerful remote machine (cloud or local network - but different from the one I sit in front of and installed PyCharm on) as PyCharm Python remote interpreter.
How do I set up the PyCharm access?
Do I need to run an ssh server on the docker image and treat it as a classic ssh remote interpreter while setting up port mapping on the cloud host machine?
The tutorials (e.g. https://blog.jetbrains.com/pycharm/2017/03/docker-compose-getting-flask-up-and-running/) apparently only apply to Docker and the Docker image running locally on the same machine as PyCharm, "localhost" or "127.0.0.1". I would like "some_machine_with_many_gpus.cloudapp.net:port" or similar. Is this possible?
So I have a RedHat System in AWS running Spark on top of HDFS. Now I want to access PySpark from my local machine i.e. Interactive Python.
So, I installed Spyder-Py2 to connect to the remote AWS machine so I can access Spyder Python.
Route:
Using Cookbook: Connecting to a remote kernel via ssh, I started Kernel on AWS machine i.e. Server and copied the json file over to local machine.
Changed the ip in kernel1234.json to point to the public ip.
Went to Consoles in Spyder-Py2 and select "Connect to existing kernel".
Passed the kernel1234.json file, added username and host in the hostname section.
Passed my AWS pem key in SSH Key section and entered the password
But it fails with the below error:
Unable to connect to IPython kernel-1234.json
Can anybody please tell me what am I missing here ?
Note: The server on the AWS VM is still running.