We Keep Coding

How can I use ssh with python and linux to connect to windows and run commands?

I wrote a generic script on Python that supports Windows and Unix ssh connection from Unix.
When I try Unix command from Unix to create dir in Windows, I get exit 53
/usr/bin/ssh2 --password pass -l admin ip_address mkdir "C:\Temp\ALEX_TEST_EX" &
When I write only /usr/bin/ssh2 --password pass -l admin ip_address, it is Ok. I login to Windows
When I try C:\Temp\ALEX_TEST_EX on this machine manually it is also Ok.
What is problem?
I also try to use with Python ssh2 command like
import paramiko
ssh = paramiko.SSHClient()
ssh.connect("ip_address", username="admin", password="pass")
but I get exceptions
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/python3.5/site-packages/paramiko/client.py", line 402, in connect
self, server_hostkey_name, server_key
File "/python3.5/site-packages/paramiko/client.py", line 768, in missing_host_key
'Server {!r} not found in known_hosts'.format(hostname)
paramiko.ssh_exception.SSHException: Server 'X.Y.Z.W' not found in known_hosts

You have not configued the client machine to allow you to know the server you are trying to connect to. You can either configure the client, or as a work around you can set the MissingHostKeyPolicy on paramiko like:
Code:
To warn when host not in known hosts:
ssh.set_missing_host_key_policy(paramiko.WarningPolicy)
To auto add host to know hosts:
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy)
Full Code:
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.WarningPolicy)
ssh.connect("localhost", username="admin", password="pass")

flask mod_wsgi address already in use [duplicate]

This question already has answers here:
How to kill a process running on particular port in Linux?
(34 answers)
Closed 6 years ago.
I have tried everything from last 2 days. But nothing really helped me out.
Background : I followed this to install flask app on production but I need to change serverName in vhost file so I forgot to kill an already running flask app/process and edited same vhost file to point to other location with few changes.
Problem : Now after apache restart, I continue getting following error when I access modified serverName.
mod_wsgi (pid=1685): Target WSGI script '/var/www/html/machine/machine.wsgi' cannot be loaded as Python module., referer: http://dev.badiyajobs.com/
mod_wsgi (pid=1685): Exception occurred processing WSGI script '/var/www/html/machine/machine.wsgi'., referer: http://dev.badiyajobs.com/
Traceback (most recent call last):
File "/var/www/html/machine/machine.wsgi", line$
from run import app as application
File "/var/www/html/machine/assessment/run.py",$
app.run()
File "/usr/local/lib/python2.7/dist-packages/fl$
run_simple(host, port, self, **options)
File "/usr/local/lib/python2.7/dist-packages/we$
inner()
File "/usr/local/lib/python2.7/dist-packages/we$
fd=fd)
File "/usr/local/lib/python2.7/dist-packages/we$
passthrough_errors, ssl_context, fd=fd)
File "/usr/local/lib/python2.7/dist-packages/we$
HTTPServer.__init__(self, (host, int(port)), $
File "/usr/lib/python2.7/SocketServer.py", line$
self.server_bind()
File "/usr/lib/python2.7/BaseHTTPServer.py", li$
SocketServer.TCPServer.server_bind(self)
File "/usr/lib/python2.7/SocketServer.py", line$
[self.socket.bind(self.server_address)
File "/usr/lib/python2.7/socket.py", line 224, $
return getattr(self._sock,name)(*args)
error: [Errno 98] Address already in use
Anybody please suggest how to get rid of already running app. I have tried lots of things regarding killing processes, but noting actually worked.

The easiest way to kill all processes that are listening on that port would be to use the fuser(1) command. For example, to see all of the processes listening for http requests on port 80 (run as root or use sudo):
fuser 80/tcp
If you want to kill them, then just add '-k' option
Source

Celery doesn't work on docker

I have the problem to use celery in docker.
I configured two docker container, web_server and celery_worker. celery_worker includes rabbitmq-server. The web_server calls the task from celery worker.
I configured same thing in VM by vagrant. And it works. But docker speak out the error message like below.
Traceback (most recent call last):
File "/web_server/test/test_v1_data_description.py", line 58, in test_create_description
headers=self.get_basic_header()
.........
.........
File "../task_runner/__init__.py", line 31, in run_describe_task
kwargs={})
File "/usr/local/lib/python3.4/dist-packages/celery/app/base.py", line 349, in send_task
self.backend.on_task_call(P, task_id)
File "/usr/local/lib/python3.4/dist-packages/celery/backends/rpc.py", line 32, in on_task_call
maybe_declare(self.binding(producer.channel), retry=True)
File "/usr/local/lib/python3.4/dist-packages/kombu/messaging.py", line 194, in _get_channel
channel = self._channel = channel()
File "/usr/local/lib/python3.4/dist-packages/kombu/utils/__init__.py", line 425, in __call__
value = self.__value__ = self.__contract__()
File "/usr/local/lib/python3.4/dist-packages/kombu/messaging.py", line 209, in <lambda>
channel = ChannelPromise(lambda: connection.default_channel)
File "/usr/local/lib/python3.4/dist-packages/kombu/connection.py", line 756, in default_channel
self.connection
File "/usr/local/lib/python3.4/dist-packages/kombu/connection.py", line 741, in connection
self._connection = self._establish_connection()
File "/usr/local/lib/python3.4/dist-packages/kombu/connection.py", line 696, in _establish_connection
conn = self.transport.establish_connection()
File "/usr/local/lib/python3.4/dist-packages/kombu/transport/pyamqp.py", line 116, in establish_connection
conn = self.Connection(**opts)
File "/usr/local/lib/python3.4/dist-packages/amqp/connection.py", line 165, in __init__
self.transport = self.Transport(host, connect_timeout, ssl)
File "/usr/local/lib/python3.4/dist-packages/amqp/connection.py", line 186, in Transport
return create_transport(host, connect_timeout, ssl)
File "/usr/local/lib/python3.4/dist-packages/amqp/transport.py", line 299, in create_transport
return TCPTransport(host, connect_timeout)
File "/usr/local/lib/python3.4/dist-packages/amqp/transport.py", line 95, in __init__
raise socket.error(last_err)
nose.proxy.OSError: [Errno 111] Connection refused
These are Dockerfiles for two containers.
Dockerfile for web_server.
FROM ubuntu:14.04
MAINTAINER Jinho Yoo
# Update packages.
RUN apt-get clean
RUN apt-get update
# Create work folder.
RUN mkdir /web_server
WORKDIR /web_server
# Setup web server and celery.
ADD ./install_web_server_conf.sh ./install_web_server_conf.sh
RUN chmod +x ./install_web_server_conf.sh
RUN ./install_web_server_conf.sh
#Reduce docker size.
RUN rm -rf /var/lib/apt/lists/*
# Run web server.
CMD ["python3","web_server.py"]
# Expose port.
EXPOSE 5000
Dockerfile for celery_worker.
FROM ubuntu:14.04
MAINTAINER Jinho Yoo
# Update packages.
RUN apt-get clean
RUN apt-get update
RUN apt-get install -y wget build-essential ca-certificates-java
# Setup python environment.
ADD ./bootstrap/install_python_env.sh ./install_python_env.sh
RUN chmod +x ./install_python_env.sh
RUN ./install_python_env.sh
# Install Python libraries including celery.
RUN pip3 install -r ./core/requirements.txt
# Add mlcore user for Celery worker
RUN useradd --uid 1234 -M mlcore
RUN usermod -L mlcore
# Celery configuration for supervisor
ADD celeryd.conf /etc/supervisor/conf.d/celeryd.conf
RUN mkdir -p /var/log/celery
# Reduce docker size.
RUN rm -rf /var/lib/apt/lists/*
# Run celery server by supervisor.
CMD ["supervisord", "-c", "/ml_core/supervisord.conf"]
# Expose port.
EXPOSE 8080
EXPOSE 8081
EXPOSE 4040
EXPOSE 7070
EXPOSE 5672
EXPOSE 5671
EXPOSE 15672

Docker containers can't talk to each other normally. My guess is that your connection string is something like localhost:<port>?
There's a couple ways to have your containers be able to communicate.
1: linking
http://rominirani.com/2015/07/31/docker-tutorial-series-part-8-linking-containers/
essentially, at runtime, docker adds an entry into your hosts file that points to the internal IP address of the docker container within the same private docker network stack.
2: docker run --net=host:
this binds your containers to your host network stack, thus, all containers will appear to be running from localhost, and can be accessed as such. You may run into port conflict issues if you are running multiple containers that bind to the same external port, just be aware of that.
3: external HAProxy:
you can bind a DNS entry to an HAProxy, and config the proxy to redirect traffic with a hostheader that matches the DNS entry to the host:port your container is running on, and any calls from other containers will "leave" the private docker network stack, hit the DNS server, and circle back to the HAProxy, which will direct to the proper container.

I found the reason. Docker container in celery_worker doesn't run rabbitmq-server. So I added two lines in Dockerfile of celery_worker like below.
# Run rabbitmq server and celery.
ENTRYPOINT service rabbitmq-server start && supervisord -c /ml_core/supervisord.conf

django.db.utils.OperationalError Could not connect to server

I am not sure how to fix this issue
I have no idea why I am getting this error when I try to runserver:
Performing system checks...
System check identified no issues (0 silenced).
Unhandled exception in thread started by <function wrapper at 0x1085589b0>
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/django/utils/autoreload.py", line 222, in wrapper
fn(*args, **kwargs)
File "/Library/Python/2.7/site-packages/django/core/management/commands/runserver.py", line 107, in inner_run
self.check_migrations()
File "/Library/Python/2.7/site-packages/django/core/management/commands/runserver.py", line 159, in check_migrations
executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
File "/Library/Python/2.7/site-packages/django/db/migrations/executor.py", line 17, in __init__
self.loader = MigrationLoader(self.connection)
File "/Library/Python/2.7/site-packages/django/db/migrations/loader.py", line 49, in __init__
self.build_graph()
File "/Library/Python/2.7/site-packages/django/db/migrations/loader.py", line 184, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/Library/Python/2.7/site-packages/django/db/migrations/recorder.py", line 59, in applied_migrations
self.ensure_schema()
File "/Library/Python/2.7/site-packages/django/db/migrations/recorder.py", line 49, in ensure_schema
if self.Migration._meta.db_table in self.connection.introspection.get_table_list(self.connection.cursor()):
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 165, in cursor
cursor = self.make_debug_cursor(self._cursor())
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 138, in _cursor
self.ensure_connection()
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 133, in ensure_connection
self.connect()
File "/Library/Python/2.7/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 133, in ensure_connection
self.connect()
File "/Library/Python/2.7/site-packages/django/db/backends/__init__.py", line 122, in connect
self.connection = self.get_new_connection(conn_params)
File "/Library/Python/2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 134, in get_new_connection
return Database.connect(**conn_params)
File "/Library/Python/2.7/site-packages/psycopg2/__init__.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
When I try to connect to postgres:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'beerad',
'USER': 'bli1',
'PASSWORD': '',
'HOST': '127.0.0.1',
'PORT': '5432',
}
}

It can be some issues:
PostgreSQL is not running. Check it with sudo service postgresql status
Your PostgresSQl is not running on port 5432. You can check it typing sudo netstat -nl | grep postgres
You have something wrong trying to connect to your db like the username, the password or the databasename. Check that they are what postgres ask for you to connect it and that is the db_name that you want to access to.
Problems with postmaster.pid in postgres. It can happen because of a shutdown unproperly done. It makes to remind a pid alive that doesn't allow your server start. To fix it you have to:
* rm /usr/local/var/postgres/postmaster.pid
* pg_resetxlog -f /usr/local/var/postgres
After this it should run properly if you make the runserver of postgres
Help in Mac OSX: How to start PostgreSQL server on Mac OS X?

Try killing all postgres processes. Im on a MAC and this solution that I've found on ubuntus forum really works.
https://askubuntu.com/questions/547434/how-to-nicely-stop-all-postgres-processes

For Windows
Go to search bar and just write "Open psql" and hit Enter.
Once screen is opened, rerun django project.

In my case, all was set up well and Postgres had the right port, PostgreSQL was running normally, but the 5432 port was being shared with phppgadmin, I could access the phppgadmin that gives me web access to Postgres database server, but my Django application was not working it would return Connection refused error. so I changed the port number on the phppgadmin config file (/etc/phppgadmin/config.inc.php) to 5433 from 5432 and all worked fine.

In MacOS I stopped and restarted postgresql according to the advice given in this StackExchange answer:
https://dba.stackexchange.com/a/171580/182403
brew services stop postgresql
rm /usr/local/var/postgres/postmaster.pid # adjust path accordingly to your install
brew services start postgresql

In project_folder/settings.py under DATABASE - HOST settings you should use local IP (127.0.0.1) not your public IP.
Correct
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'yourdb',
'USER': 'youruser',
'PASSWORD': 'yourpass',
'HOST': '127.0.0.1',
'PORT': '5432'
}
}
Incorrect
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'yourdb',
'USER': 'youruser',
'PASSWORD': 'yourpass',
'HOST': '188.252.196.234 ',
'PORT': '5432'
}
}

I had the same issue. This error occurs when an improper system shutdown has been done. I'm on an M1, and I've installed PostgreSQL via Homebrew.
I tried to start PostgreSQL after receiving an error identical to yours using brew services start postgresql, but I got this error:
Bootstrap failed: 5: Input/output error
Try re-running the command as root for richer errors.
Error: Failure while executing; `/bin/launchctl bootstrap gui/501 /Users/adithraghav/Library/LaunchAgents/homebrew.mxcl.postgresql.plist` exited with 5.
So, I tried to stop the running instance and start it again with brew services restart postgresql.
This line gave me the following output:
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)
I checked the port on which PostgreSQL was running with sudo netstat -nl | grep postgres, but received no output.
So I deleted the existing postmaster.pid file with this:
rm /opt/homebrew/var/postgres/postmaster.pid
NOTE: If you are not on an M1 Mac, you have to run rm /usr/local/var/postgres/postmaster.pid.
Then, I ran pg_resetwal -f /opt/homebrew/var/postgres to reset the write-ahead log. (NOTE: From Postgres 10 and onwards, pg_resetxlog has been renamed to pg_resetwal).
Now, python3 manage.py runserver works with no issues :)

The following command works for me (Windows)-
pg_ctl -D "C:\Program Files\PostgreSQL\11\data" restart
Then run server again-
python manage.py runserver

Restarting the server solved the problem on my side.

In case you are seeing this error while using PostgresSQL from Google Cloud follow all configurations as mentioned in
https://cloud.google.com/python/django/flexible-environment#macos-64-bit
Also separate the HOST configuration as below
DATABASES['default']['HOST'] = '/cloudsql/'
if os.getenv('GAE_INSTANCE'):
pass
else:
DATABASES['default']['HOST'] = '127.0.0.1'
This helps in resolving this error.

Go to aws instance -> security groups -> source -> inbound -> ::0

You might not connecting to the right database. If you are using Docker be sure you are using 5432 as port at the outside of the image.
eg:
db:
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- ./postgres/.env
ports:
- 5432:5432

One problem that I found common with Django/Postgres (especially with Docker) is that your Django Web App maybe starting up before your Postgres server starts up. If the other solutions don't work, try to restart your Django App after start up. With Docker, the command would be docker restart <web-app container name>

You have to enable listen addresses if you are using remote database.
cd /etc/postgresql/12.x/main/
open file named postgresql.conf
sudo nano postgresql.conf
add this line to that file
listen_addresses = '*'
then an open a file named pg_hba.conf
sudo nano pg_hba.conf
and add this line to that file
host all all 0.0.0.0/0 md5
restart your server
sudo /etc/init.d/postgresql restart

Python - Socket Error, Address In Use

I'm currently attempting to setup a SiriServer (that's beside the point) on Xubuntu 12.10 x64, when I run the server python returns error
socket.error: [Errno 98] Address already in use.
The server by default is attempting to run on port 443, which unfortunetly is required in order for this application to work.
To double check if anything is running on port 443, I execute the following:
lsof -i :443
There's no results, unless I have something like Chrome or Firefox open, which I end up closing. Here's the full return from attempting to run the server application.
dustin#dustin-xubuntu:~/Applications/SiriServer$ sudo python siriServer.py
CRITICAL load_plugins Failed loading plugin due to missing module: 'Wordnik library not found. Please install wordnik library! e.g. sudo easy_install wordnik'
INFO <module> Starting Server
Traceback (most recent call last):
File "siriServer.py", line 493, in <module>
server = SiriServer('', options.port)
File "siriServer.py", line 425, in __init__
self.bind((host, port))
File "/usr/lib/python2.7/asyncore.py", line 342, in bind
return self.socket.bind(addr)
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address already in use
I'm stuck on what to do, as this is the last part of setting up this application. Any help is appreciated.

You're not root -- that's your problem. To bind to ports under 1024 on Unix, you must be the superuser. So, hit su and try the python code again. Alternatively, bind to a port from 1024 to 65535.

This often happens when a python program doesn't exit properly when pressing ^C or ^Z. You could try reseting the terminal or exiting the terminal. You can also do killall -9 server.py
Another effective way to help prevent this even if you have root privs this can happen if a socket is not closed properly, here is a fix:
s=socket.socket( )
s.bind(("0.0.0.0", 8080))
while 1:
try:
c, addr = s.accept()
except KeyBoardInterrupt:
s.close()
exit(0)

I got that error even if port number is more than 1024
You can use
pkill -9 python
run command twice, it will list all python files which are killed

List all processes that you have running with
ps -a
Take the PID corresponding to python and pipe it into the kill command with (Example PID 2770)
kill -9 2770