We Keep Coding

Environment Variables in Fabric2

I’m using Python 3.6 and Fabric 2.4. I’m using Fabric to SSH into a server and run some commands. I need to set an environment variable for the commands being run on the remote server. The documentation indicates that something like this should work:
from fabric import task
#task(hosts=["servername"])
def do_things(c):
c.run("command_to_execute", env={"KEY": "VALUE"})
But that doesn’t work. Something like this should also be possible:
from fabric import task
#task(hosts=["servername"])
def do_things(c):
c.config.run.env = {"KEY": "VALUE"}
c.run("command_to_execute")
But that doesn’t work either. I feel like I’m missing something. Can anyone help?

I was able to do it by setting inline_ssh_env=True, and then explicitly setting the env variable, ex:
with Connection(host=hostname, user=username, inline_ssh_env=True) as c:
c.config.run.env = {"MY_VAR": "this worked"}
c.run('echo $MY_VAR')

As stated on the site of Fabric:
The root cause of this is typically because the SSH server runs non-interactive commands via a very limited shell call: /path/to/shell -c "command" (for example, OpenSSH). Most shells, when run this way, are not considered to be either interactive or login shells; and this then impacts which startup files get loaded.
You read more on this page link
So what you try to do won't work, and the solution is to pass the environment variable you want to set explicitly:
from fabric import task
#task(hosts=["servername"])
def do_things(c):
c.config.run.env = {"KEY": "VALUE"}
c.run('echo export %s >> ~/.bashrc ' % 'ENV_VAR=VALUE' )
c.run('source ~/.bashrc' )
c.run('echo $ENV_VAR') # to verify if it's set or not!
c.run("command_to_execute")

You can try that:
#task
def qa(ctx):
ctx.config.run.env['counter'] = 22
ctx.config.run.env['conn'] = Connection('qa_host')
#task
def sign(ctx):
print(ctx.config.run.env['counter'])
conn = ctx.config.run.env['conn']
conn.run('touch mike_was_here.txt')
And run:
fab2 qa sign

When creating the Connection object, try adding inline_ssh_env=True.
Quoting the documentation:
Whether to send environment variables “inline” as prefixes in front of command strings (export VARNAME=value && mycommand here), instead of trying to submit them through the SSH protocol itself (which is the default behavior). This is necessary if the remote server has a restricted AcceptEnv setting (which is the common default).

According to that part of the official doc, the connect_kwargs attribute of the Connection object is intended to replace the env dict. I use it, and it works as expected.

linking mongoDB to python code?

still very new to programming so in my quest to learn git i am working on getting a chat bot for telegram up and running that i found on a github. iv never used mongoDB or any database and i see i need
DB_NAME=os.environ['DB_NAME']
DB_USERNAME=os.environ['DB_USERNAME']
DB_HOST=os.environ['DB_HOST']
DB_PASSWORD=os.environ['DB_PASSWORD']
DB_URL="dbname='"+DB_NAME+"' user='"+DB_USERNAME+"' host='"+DB_HOST+"' password='"+DB_PASSWORD+"'"
conn=psycopg2.connect(DB_URL)
cur=conn.cursor()
cur.execute("SELECT chat_id FROM users")
users=cur.fetchall()
id_list=[chat_id[0] for chat_id in users]
cur.close()
conn.close()
the DB_NAME=os.environ['DB_NAME']
where does DB_NAME come from and why does os.environ throw an error. im using python 3. the github is here https://github.com/trinhvv/trading-analysis-bot

os.environ["DB_NAME"] means that DB_NAME is stored in the system environment variables.
You can connect to the mongodb by:
Option 1:
Declare variables in system environment.
Option 2:
declaring enviornment variables in the code itself.
Environment variables must be strings, so use,
os.environ["DB_NAME"] = "your_db_name"
Option 3:
Don't use variables from environment, write it directly,
DB_NAME = "your_db_name"

Python LDAP write attribute to Active Directory

I am able to bind and query Active Directory via python-ldap without any issues except when it comes to adding or modifying attributes on AD. I can add the attribute but the encoding seems to be way off as all the text is garbled.
I've tried encoding my string with utf8 and a few others with no luck.
I've also tried binding with a Domain Admin account along with binding with the user account to which I will be changing an attribute, same result regardless.
Here is the method I use to update an attribute:
class LdapHelpers:
def __init__(self):
import ldap
# set globals
self.server = 'LDAP://dc.mycompany.com'
self.admin_dn = 'CN=Administrator,CN=users,DC=mycompany,DC=com'
self.admin_pass = 'coolpassword'
# init LDAP connection
#ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, 0)
ldap.set_option(ldap.OPT_REFERRALS, 0)
ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)
ldap.protocol_version = ldap.VERSION3
self.ldap = ldap.initialize(self.server)
def update_attribute(self, attrib, value):
try:
import ldap
conn = self.ldap
conn.simple_bind_s(self.admin_dn, self.admin_pass)
mod_attrs = [( ldap.MOD_REPLACE, "mobile", "6306564123")]
# I have tried other variations of the above
# mod_attrs = [( ldap.MOD_REPLACE, "mobile", "6306564123".encode('utf-8)]
conn.modify_s('CN=Mike Smith,OU=GoogleApps,DC=company,DC=com', mod_attrs)
print 'record updated'
except ldap.LDAPError as e:
return e.message
Doing a ldapsearch via terminal this is what the attribute looks like:
mobile:: MC8sAQAAAAAQNA==
This is what 'Hello World' looks like when I set mobile to it:
mobile:: 77+9ehsCAAAAABDvv70V
I've checked MSDN and it says that ldap attribute is just a Unicode string.
System: Ubuntu 15.10 64bit
Python: 2.7.10
python-ldap==2.4.21
As a side note I can search AD without any issues and parse/display returned user attributes, the issue only seems to be with creating or modifying attributes that this encoding issue comes in to play.

The '=' at the end is often an indicator that it is Base64 encoding. Python has a standard library for encoding/decoding base64 (The link is for Python 3, but Python 2 also has the library). LDAP does indeed use Base64 for something. See The LDAP Data Interchange Format (LDIF).

Take a look at the code from pyad to clarify what to do: https://pypi.python.org/pypi/pyad
It's Python-based.
Another example at already answered question: Use Python script to manage remote LDAP server

Ok I found out what was going on, I was using PyPy 4.0.1 as the interpreter and for some reason this was either causing issues with the python-ldap library and/or encoding for strings.
I switched back to Python 2.7.10 for the interpreter and now the very same modify commands up above work as expected using the python-ldap library. So definitely a word of caution if using PyPy and this particular library....

django 1.5 + pymysql error: ImportError: cannot import name Thing2Literal

I try to use django1.5 and pymysql as MySQLdb as here How to make Django work with unsupported MySQL drivers such as gevent-mysql or Concurrence's MySQL driver?
In the top of my management command:
+try:
+ import pymysql
+ pymysql.install_as_MySQLdb()
+except ImportError:
+ pass
but get error:
/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 30, in <module>
from MySQLdb.converters import conversions, Thing2Literal
ImportError: cannot import name Thing2Literal
how to fix it?

Just ran into same issue on Django 1.5.1 and PyMySQL 0.5
Fixed by using CMGS fork (https://github.com/CMGS/PyMySQL) of PyMySQL. Hopefully this will make it's way into mainline PyMySQL. See CMGS's pull request here: https://github.com/petehunt/PyMySQL/pull/106
Judging from author's comments and the feedback in the pull request, I think it's pretty solid for production use.
Example requirements.txt line:
-e git://github.com/CMGS/PyMySQL.git#egg=PyMySQL-dev

In MySQLdb, the Thing2Literal method is not used if you're using a recent enough version of MySQL, in which case the connection's string_literal method is used instead when a connection is available.
You'll need to patch pymysql so that it does the same thing and lets you use the connection's method.
Context
This method is used for escaping SQL statements. PLaying around with it consequently has security issues that you must consider.
The reason why you want to use the connection's method is the charset, which plays a role in escaping.
Fix the ImportError
This is a pretty easy one, you just need to implement a dummy Thing2Literal method in pymysql.converters. We'll never call it anway, so we don't care about it:
def _Thing2Literal(o,d):
"""
Implemented for compatibility with Django.
This function is overriden by the connection's escape method when one is available.
"""
raise NotImplementedError('Thing2Literal is only implemented through the Connection object.')
Thing2Literal = _Thing2Literal
Monkey-patching Thing2Literal at runtime when an connection is available
In pymysql.connections.Connection, add: import pymysql.converters
At the end of pymysql.connections.Connection.__init__, add the following:
pymysql.converters.Thing2Literal = lambda o, d: self.escape(o)
And at the end of pymysql.connections.Connection.__del__, add the reverse:
pymysql.converters.Thing2Literal = pymysql.converters._Thing2Literal
We can discard the d argument because it's a dictionary of existing conversions, which are already available to the Connection.escape method.
Caveats
There's a good chance this will break, and expose security issues.
Additionally, I'm pretty sure it will break badly if you have several active connections that use different charsets.
You might need to play around a bit with Django too to make sure it uses your monkey-patched version when available - namely replace from MySQLdb.converters import Thing2Literal with something that still binds the Thing2Literal name to the module.
You can of course achieve the same effect without patching django and making the _Thing2Literal function smarter.

How to connect to a mongoDB

I'm using mongoLabs to host my database and I want to connect to it from my app.
I'm also using the Motor module in pyMongo. I'm unsure where to instantiate the connection.
For instance I know that if the database was on same local machine as the app I would do this:
database = motor.MotorClient().open_sync().myDatabase
The mongoLab site says to include the following uri in the driver:
mongodb://<dbuser>:<dbpassword>#ds047057.mongolab.com:47057/myDatabase
But I cannot see how to create the connection to this database.
Thanks

It looks like MotorClient takes the same arguments as MongoClient:
https://github.com/ajdavis/mongo-python-driver/blob/motor/motor/init.py#L782
http://api.mongodb.org/python/current/api/pymongo/mongo_client.html
Given that, you should be able to use the URI like so:
database = motor.MotorClient("mongodb://<dbuser>:<dbpassword>#ds047057.mongolab.com:47057/myDatabase").open_sync().myDatabase

You should to specify connection settings for MotorClient following these manuals:
MotorClient takes the same constructor arguments as MongoClient, as well as, http://emptysquare.net/motor/pymongo/api/motor/motor_client.html#motor.MotorClient,
http://emptysquare.net/motor/pymongo/api/pymongo/mongo_client.html#pymongo.mongo_client.MongoClient
"The host parameter can be a full mongodb URI, in addition to a simple
hostname. It can also be a list of hostnames or URIs. Any port
specified in the host string(s) will override the port parameter. If
multiple mongodb URIs containing database or auth information are
passed, the last database, username, and password present will be
used. For username and passwords reserved characters like ‘:’, ‘/’,
‘+’ and ‘#’ must be escaped following RFC 2396."
db = database = motor.MotorClient('mongodb://<dbuser>:<dbpassword>#ds047057.mongolab.com:47057/myDatabase
').open_sync().myDatabase

Previous answers have got a bit outdated, so the correct way according to the docs and as worked for me:
import motor.motor_asyncio
import asyncio
from asyncio import coroutine
db = motor.motor_asyncio.AsyncIOMotorClient().database_name
https://motor.readthedocs.io/en/stable/tutorial-asyncio.html
https://github.com/mongodb/motor/blob/master/doc/tutorial-asyncio.rst