We Keep Coding

Python - how to write a window shared folder path (ex. \\192.168.1.19\cert\testcert.pfx) using pywinrm

I access to one remote machine to automate some ssl task. To accomplish this from the remote I need import a file from a shared network folder in windows. I´m using pywinrm then after a successful session started I run:
s.run_ps('''
$Password = ConvertTo-SecureString -AsPlainText -Force {0}
Import-PfxCertificate -FilePath {1} -CertStoreLocation Cert:\LocalMachine\WebHosting -
Password $Password
'''.format(secret, path)
using similar script with local folder paths I don't have any problem but typing kind of this \\192.168.1.19\cert\testcert.pfx I have the following error
std_error b'Import-PfxCertificate : The system cannot find the path specified. 0x80070003 \n(WIN32: 3 ERROR_PATH_NOT_FOUND)\nAt line:3 char:9\n+ Import-PfxCertificate -FilePath \\192.168.1.19\\Certificates1\\test ...\n+
I built this path like string "host"+"\\"+cert+"\\"+file where host = "\\192.168.1.19", cert = "Certificates1", and file = "testcert.pfx"
I have seen that in powershell I should type exactly PS C:\>\\192.168.1.19\cert\testcert.pfx and it works but pointing to the error the string that is passing python to powershell is \ and not escaping and passing only one backslash.
Any help or advice?
Thank you in advance.

the error the string that is passing python to powershell is \ and not escaping and passing only one backslash.
\\ = \ so you need to do host = "\\\\192.168.1.19"

The problem deals with the second hop issue, so the script can't access to another network shared file while remoting.
The solution was change authentication to CredSSP and allowing the second hop delegate the credentials.
After this, no errors referring to any path appears.

Cant connect to superset using athena database

I am new to superset.
Going to Sources > Databases for a new connection to my athena.
I have downloaded JDBC driver and writing following connection line:
awsathena+jdbc://AKIAJ2PKWTZYAPBYKRMQ:xxxxxxxxxxxxxxx#athena.us-east-1.amazonaws.com:443/default?s3_staging_dir='s3://aws-athena-query-results-831083831535-us-east-1/' as SQLAlchemy URI. First parameter being access key and 2nd being secret key(Modified a bit for privacy)
I am getting the error:
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:awsathena.jdbc"}
I really wish to explore the open source visualisation using superset on my databases.

As per Superset documentation, you need to escape/encode at least the s3_staging_dir, i.e.,
s3://... -> s3%3A//...
Have you followed that step?

If you are sure you have done pip install "PyAthenaJDBC>1.0.9" in the same python environment as you start your superset. Try restarting Superset in the same environment.

In my case problem was with the special characters in the aws_secret_key and s3_staging_dir. I solved it by putting the output of quote_plus methods into the URI. No quotes were required.
from urllib.parse import quote_plus
secretkey = quote_plus(aws_secret_access_key)
loc = quote_plus(s3_staging_dir)
Further, make sure the schema_name (i.e. database name) already exists in the s3 path. Hope it helps!

Loading data from .gz file from remote server to postgresql instance of redshift without using S3 bucket?

Is there a way to load my .gz files that I have in a remote server, into a redshift database table (postgresql) without loading it first in s3 bucket?
I saw two options:-
"copy <tablename> from program "gzip -d <file location in remote server> with delimeter tab '\t' " command after connecting to the database. This gave me an error "syntax error at or near program".
Could not figure the way out using the psycopg2 library in python. Here is the code that was written for python.
import psycopg2
import gzip
conn_string="dbname='<>' port='5439' user='<>' password='<>' host='<>'";
print "Connecting to database\n ->%s"%(conn_string)
conn=psycopg2.connect(conn_string);
cursor=conn.cursor();
with gzip.open('<gzip filelocation>','r') as l:
for line in l:
cursor.execute('copy <table_name from %s with delimiter '\t'", (line,))
Thanks.

You can COPY directly from a remote host using SSH. For practical purposes that means the remote host needs to be a *nix machine, not Windows.
You still need to use S3 however to store a manifest file that specifies the connection details and the command to run, e.g., your gzip command above.
Docs here: COPY from Remote Host (SSH)

You can use the "INSERT" command. You can read each line from the gzip file and insert it into your table. This way you don't need a S3 bucket because you aren't using the "COPY" command.For example:
with gzip.open('<gzipfile>','r') as l:
for line in l:
b=line.split('\t') # tab delimited
# Handling null values
for k in range(len(b)):
if b[k]=='\N':
b[k]=None
b=tuple(b)
sql= "INSERT INTO"+ " " + <tablename> + " VALUES (" + ",".join(["?"]*(len(b))) + ")"
cursor.execute(sql,b)

Is this python code safe against injections?

I have a server/client socket pair in Python. The server receives specific commands, then prepares the response and send it to the client.
In this question, my concern is just about possible injections in the code: if it could be possible to ask the server doing something weird with the 2nd parameter -- if the control on the command contents is not sufficient to avoid undesired behaviour.
EDIT:
according to advices received
added parameter shell=True when calling check_output on windows. Should not be dangerous since the command is a plain 'dir'.
.
self.client, address = self.sock.accept()
...
cmd = bytes.decode(self.client.recv(4096))
ls: executes a system command but only reads the content of a directory.
if cmd == 'ls':
if self.linux:
output = subprocess.check_output(['ls', '-l'])
else:
output = subprocess.check_output('dir', shell=True)
self.client.send(output)
cd: just calls os.chdir.
elif cmd.startswith('cd '):
path = cmd.split(' ')[1].strip()
if not os.path.isdir(path):
self.client.send(b'is not path')
else:
os.chdir(path)
self.client.send( os.getcwd().encode() )
get: send the content of a file to the client.
elif cmd.startswith('get '):
file = cmd.split(' ')[1].strip()
if not os.path.isfile(file):
self.client.send(b'ERR: is not a file')
else:
try:
with open(file) as f: contents = f.read()
except IOError as er:
res = "ERR: " + er.strerror
self.client.send(res.encode())
continue
... (send the file contents)

Except in implementation details, I cannot see any possibilities of direct injection of arbitrary code because you do not use received parameters in the only commands you use (ls -l and dir).
But you may still have some security problems :
you locate commands through the path instead of using absolute locations. If somebody could change the path environment variable what could happen ... => I advice you to use directly os.listdir('.') which is portable and has less risks.
you seem to have no control on allowed files. If I correctly remember reading CON: or other special files on older Windows version gave weird results. And you should never give any access to sensible files, configuration, ...
you could have control on length of asked files to avoid users to try to break the server with abnormally long file names.

Typical issues in a client-server scenario are:
Tricking the server into running a command that is determined by the client. In the most obvious form this happens if the server allows the client to run commands (yes, stupid). However, this can also happen if the client can supply only command parameters but shell=True is used. E.g. using subprocess.check_output('dir %s' % dir, shell=True) with a client-supplied dir variable would be a security issue, dir could have a value like c:\ && deltree c:\windows (a second command has been added thanks to the flexibility of the shell's command line interpreter). A relatively rare variation of this attack is the client being able to influence environment variables like PATH to trick the server into running a different command than intended.
Using unexpected functionality of built-in programming language functions. For example, fopen() in PHP won't just open files but fetch URLs as well. This allows passing URLs to functionality expecting file names and playing all kinds of tricks with the server software. Fortunately, Python is a sane language - open() works on files and nothing else. Still, database commands for example can be problematic if the SQL query is generated dynamically using client-supplied information (SQL Injection).
Reading data outside the allowed area. Typical scenario is a server that is supposed to allow only reading files from a particular directory, yet by passing in ../../../etc/passwd as parameter you can read any file. Another typical scenario is a server that allows reading only files with a particular file extension (e.g. .png) but passing in something like passwords.txt\0harmless.png still allows reading files of other types.
Out of these issues only the last one seems present in your code. In fact, your server doesn't check at all which directories and files the client should be allowed to read - this is a potential issue, a client might be able to read confidential files.

.cgi problem with web server

The code
#!/usr/bin/env python
import MySQLdb
print "Content-Type: text/html"
print
print "<html><head><title>Books</title></head>"
print "<body>" print "<h1>Books</h1>"
print "<ul>"
connection = MySQLdb.connect(user='me', passwd='letmein', db='my_db') cursor = connection.cursor() cursor.execute(“SELECT name FROM books ORDER BY pub_date DESC LIMIT 10”)
for row in cursor.fetchall():
print "<li>%s</li>" % row[0]
print "</ul>"
print "</body></html>"
connection.close()
I saved it as test.cgi to my web server. I run it by www.mysite.com/test.cgi unsuccessfully
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
How can you solve the problem?
[edit] after the first answer
test.cgi is executable (I run $ chmod +x test.cgi)
I use Apache.
I have this in .bashrc export PATH=${PATH}:~/bin
Python module MySQLdb is installed.
The code does not have smart quotes.
[edit] after the second answer
you're getting that error because you
haven't installed the MySQLdb module
that Python needs to talk to a MySQL
database
I installed MySQLdb to my system. The module works, since I can import them.
However, I still get the same error whet I go to the www.[mysite].com/test.cgi.
[edit]
I am not sure about the questions
Are the connect() parameters correct? Is MySQL running on localhost
at the default port?
I run MySQL on my server. Is the question about the connect() parameters relevant here?
Is the SELECT statement correct?
You mean whether I have my SQL statements such as SELECT statement correct?
I have not used any SQL queries yet.
Do I need them here?

Any number of issues can cause the error you are seeing:
Is test.cgi executable (chmod 755) on the server?
Is the directory in which you placed test.cgi designated as a ScriptAlias location or have the ExecCGI option enabled (or equivalent if you're not using Apache)?
Is python in the system PATH or in the PATH in the Web server's startup environment?
Is the MySQLdb Python library installed?
Are the connect() parameters correct? Is MySQL running on localhost at the default port?
Is the SELECT statement correct?
If you're sure that python is found (test using the simplest possible script or by logging into the Web server if you can and typing which python) then you can get much better debug output by adding the following to the top of your script just below the shebang:
import cgitb
cgitb.enable()
More details: http://docs.python.org/library/cgitb.html
Additionally, if you have shell access to the Web server, try running python and just typing:
>>> import MySQLdb
If the command returns with no error, you have your answer for #4 above. If an error is printed, you will need to get MySQLdb installed into the Web server's Python installation.
EDIT: Looking more closely at the top of your question, I see that the code was scraped from an illustrative example at the very beginning of the Django Book. As such, I might expand #5 above to include the caveat that, of course, the requisite database, tables, user, and permissions need to be set up on the MySQL installation available to the Web server.

I've tidied up the code a bit by inserting linebreaks where necessary and replacing smart quotes with " and '. Do you have any more luck with the following? Can you run it from a terminal just by typing python test.cgi?
#!/usr/bin/env python
import MySQLdb
print "Content-Type: text/html"
print
print "<html><head><title>Books</title></head>"
print "<body>"
print "<h1>Books</h1>"
print "<ul>"
connection = MySQLdb.connect(user='me', passwd='letmein', db='my_db')
cursor = connection.cursor()
cursor.execute("SELECT name FROM books ORDER BY pub_date DESC LIMIT 10")
for row in cursor.fetchall():
print "<li>%s</li>" % row[0]
print "</ul>"
print "</body></html>"
connection.close()

The error in http://dpaste.com/8866/ is occurring because you are using "curly quotes" instead of standard ASCII quotation marks.
You'll want to replace the “ and ” with ". Just use find and replace in your text editor.

Make sure you're saving the file with correct line endings. i.e. LF only on unix, or CR/LF for Windows. I just recently had the exact same problem...