I am running into a problem when running the follow python script on the server looking for commit information for the push making sure it follows a particular syntax, I am unable to get input from the user which is why the username and password are hard coded. I am now also unable to get the list of commit message that occurred before this particular push.
#!/usr/bin/python
import SOAPpy
import getpass
import datetime
import sys
import re
import logging
import os
def login(x,y):
try:
auth = soap.login(x, y)
return auth
except:
sys.exit( "Invalid username or password")
def getIssue(auth,issue):
try:
issue = soap.getIssue(auth, issue)
except:
sys.exit("No issue of that type found : Make sure all PRs are vaild jira PRs")
def git_get_commit_msg(commit_id):
return get_shell_cmd_output("git rev-list --pretty --max-count=1 " + commit_id)
def git_get_last_commit_id():
return get_shell_cmd_output("git log --pretty=format:%H -1")
def getCommitText():
commit_msg_filename = sys.argv[1]
try:
commit_msg_text = open(commit_msg_filename).read()
return commit_msg_text
except:
sys.exit("Could not read commit message")
def git_get_array_of_commit_ids(start_id, end_id):
output = get_shell_cmd_output("git rev-list " + start_id + ".." + end_id)
if output == "":
return None
commit_id_array = string.split(output, '\n')
return commit_id_array
def get_shell_cmd_output(cmd):
try:
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
return proc.stdout.read().rstrip('\n')
except KeyboardInterrupt:
logging.info("... interrupted")
except Exception, e:
logging.error("Failed trying to execute '%s'", cmd)
def findpattern(commit_msg):
pattern = re.compile("\w\w*-\d\d*")
group = pattern.findall(commit_msg)
print group
found = len(group)
found =0
issues = 0
for match in group:
auth = soap.login(jirauser,passwd)
getIssue(auth,match)
issues = issues + 1
found+=1
if found ==0:
sys.exit("No issue patterns found.")
print "Retrieved issues: " + str(issues)
def update():
print sys.argv[2]
print sys.argv[3]
old_commit_id = sys.argv[2]
new_commit_id = sys.argv[3]
commit_id_array = git_get_array_of_commit_ids(old_commit_id, new_commit_id)
for commit_id in commit_id_array:
commit_text = git_get_commit_msg(commit_id)
findpattern(commit_text)
soap = SOAPpy.WSDL.Proxy('some url')
# this line if for repointing the input from dev/null
#sys.stdin = open('/dev/tty', 'r') # this fails horribly.
#ask user for input
#jirauser = raw_inp
#("Username for jira: ")
jirauser = "username"
passwd = "987654321"
#passwd = getpass.getpass("Password for %s: " % jirauser)
login(jirauser,passwd)
#commit_msg = getCommitText()
#findpattern(commit_msg)
update()
The intended goal of this code is to check the commits made locally, and to parse through them for the intended pattern, as well as checking the in jira if that PR exists. it is a server side hook that get activated on a push to the repository.
Any tips on writing python hooks would be appreciated. Please and thank you.
I suggest that you have a look at gitorious (http://gitorious.org/gitorious).
They use ssh to handle authentication and rights management (getting the username given by ssh).
They also have some hooks on git repositories. I guess it could help to see how they are processing git hooks using ruby.
By the time your update hook fires, the server has the new commits: the question is whether your hook will allow the ref in question to move. What information from the local (sending) repository do you want?
For the credentials issue, funnel everyone through a single user. For example, GitHub does it with the git user, which is why their SSH URLs begin with git#github.com:.... Then in ~git/.ssh/authorized_keys, associate a username with each key. Note that the following should be on a single line but is wrapped for presentation purposes.
no-agent-forwarding,no-port-forwarding,no-pty,no-X11-forwarding,
command="env myuser=gbgcoll /usr/bin/git-shell -c \"${SSH_ORIGINAL_COMMAND:-}\""
ssh-rsa AAAAB...
Now to see who's trying to do the update, your hook examines the $myuser environment variable.
This doesn't give you each user's Jira credentials. To solve that issue, create a dummy Jira account that has read-only access to everything, and hardcode that Jira account's credentials in your hook. This allows you to verify that a given PR exists.
Related
I'm using AWS Cloudformation to create a stack and would like to get the value of the 'PublicIP' field out of the dictionary returned from describe_stacks().
The following schematic code does the work, but it is not resilient to changes in the dictionary structure:
#!/usr/bin/python
import sys
import boto3
import rest_client
if len(sys.argv) < 2:
print "Bad usage: missing stack name"
exit(1)
session = boto3.Session(profile_name='profile name')
client = session.client('cloudformation')
response = client.describe_stacks(StackName=sys.argv[1])
try:
ip = response['Stacks'][0]['Outputs'][1]['OutputValue']
print "Extracted instance IP address ({0})".format(ip)
except IndexError:
print "IP address not found"
exit(1)
Is there a more specific API I can use to get this field directly?
Unfortunately, AWS doesn't support filtering outputs by name. But it's pretty easy to do a filter that will:
#!/usr/bin/python
import sys
import boto3
import rest_client
OUTPUT_KEY = 'InstanceIp' # <-- Use the proper output name here
if len(sys.argv) < 2:
print "Bad usage: missing stack name"
exit(1)
stack_name = sys.argv[1]
session = boto3.Session(profile_name='profile name')
cf_resource = session.resource('cloudformation')
stack = cf_resource.Stack(stack_name)
try:
ip = filter(lambda x: x['OutputKey'] == OUTPUT_KEY, stack.outputs)[0]['OutputValue']
print "Extracted instance IP address ({0})".format(ip)
except IndexError:
print OUTPUT_KEY + " not found in " + stack_name
exit(1)
Also, I can assure you that it is future-proof as they never (to my knowledge) update the syntax of their response payloads once the API is officially released.
I was playing around with oauth2 to get a better understanding of it. For this reason, I've installed offlineimap which should act as a third-party app. I've found a nice way to read encrypted credentials here on stackexchange.
Based on the linked post I've modified/copied the following python script:
import subprocess
import os
import json
def passwd(file_name):
acct = os.path.basename(file_name)
path = "/PATHTOFILE/%s" % file_name
args = ["gpg", "--use-agent", "--quiet", "--batch", "-d", path]
try:
return subprocess.check_output(args).strip()
except subprocess.CalledProcessError:
return ""
def oauthpasswd(acct, key):
acct = os.path.basename(acct)
path = "/PATHTOFILE/%s_oauth2.gpg" % acct
args = ["gpg", "--use-agent", "--quiet", "--batch", "-d", path]
try:
return str(json.loads(subprocess.check_output(args).strip())['installed'][key])
except subprocess.CalledProcessError:
return ""
def prime_gpg_agent():
ret = False
i = 1
while not ret:
ret = (passwd("prime.gpg") == "prime")
if i > 2:
from offlineimap.ui import getglobalui
sys.stderr.write("Error reading in passwords. Terminating.\n")
getglobalui().terminate()
i += 1
return ret
prime_gpg_agent()
In the corresponding offlineimaprc file I call the function with the correct arguments:
oauth2_client_id = oauthpasswd('gmail', 'client_id')
oauth2_client_secret = oauthpasswd('gmail', 'client_secret')
oauth2_request_url = https://accounts.google.com/o/oauth2/token
oauth2_refresh_token = passwd('gmail_rf_token.gpg')
Please note in the local file the PATHTOFILE is set correctly. What I've done was downloaded the JSON file from Google including the oauth2 credentials and encrypted it. I've stored the refresh token in a separate file.
However, if I run offlineimap I get an authentication error:
ERROR: While attempting to sync account 'gmail'
('http error', 401, 'Unauthorized', <httplib.HTTPMessage instance at 0x7f488c214320>) (configuration is: {'client_secret': "oauthpasswd('gmail', 'client_secret')", 'grant_type': 'refresh_token', 'refresh_token': "passwd('gmail_rf_token.gpg')", 'client_id': "oauthpasswd('gmail', 'client_id')"})
I've tried then to check the outputs of the two python functions passwd and oauthpasswd in a python interpreter. I get the desired outputs. Even more, I've copied the output from the functions within the python interpreter to the offlineimaprc config file and I was able to sync to Gmail. This implies that there must be a mistake when offlineimap executes the file but I can't see what's wrong.
If I only encrypt my Gmail password everything is working. This means there is something going wrong from the details downloaded from Google (client_id, client_secret and refresh token). As pointed out above, the values itself are correct. I've really copied the output of
oauthpasswd('gmail', 'client_id')
oauthpasswd('gmail', 'client_secret')
passwd('gmail_rf_token.gpg')
from a python console to the offlineimaprc file and it worked.
The problem which happens is the following. According to this answer offlineimap does not allow for encryption of all keys within the offlinemaprc file. That's why the python function never gets evaluated and the wrong strings are handed over.
I'm trying to access variables that are contained in functions after an operation is performed in another Python script, but I don't want to run through the operations of the function, I just need access to the returned value of the variables gh and user.
I've tried different ways to do this such as setting the initial value of the variable to None in the main script, but the problem I run into is when I do an import of script1 into script2, it runs through the script and resets the variable in question to None again. In addition, I've tried to wrap them in a class with no luck.
Also, I've tried using if __name__ = '__main__': to run the functions, but I can't seem to figure out how to get the values out of the functions into script2 to use as global variables.
I've seen some answers here that may work such as returning the values of the function to another function for use??, but I I can't quite nail the syntax as the function doesn't seem to hold the value of the variable.
If I have asked this question incorrectly, please let me know how to improve it as I'm trying to ask "good" questions so I don't get banned. I'm still learning and I do ask a lot of questions here, but I've learned a lot by doing so.
script1.py:
#! /usr/bin/python
import github3
from github3 import login, GitHub, authorize
from getpass import getuser, getpass
import requests
import configparser
def getCreds():
try:
user = input('GitHub username: ')
except KeyboardInterrupt:
user = getuser()
password = getpass('GitHub token for {0}: '.format(user))
gh = login(user, password)
if __name__ == '__main__':
getCreds()
exec(open("next_file.py").read())
script2.py
import os
import github3
from github3 import login, GitHub, authorize
from getpass import getuser, getpass
import requests
import csv
import configparser
import sys
import script1
import codecs
gh = script1.gh
user = script1.user
def one_commit_one_file_change_pr():
#open csv file and create header rows
with open('c:\\commit_filechange.csv', 'w+') as f:
csv_writer = csv.writer(f)
csv_writer.writerow(['Login', 'Title', 'Commits', 'Changed Files','Deletions', 'Additions'])
for pr in result:
data = pr.as_dict()
changes = (gh.repository(user, repo).pull_request(data['number'])).as_dict()
if changes['commits'] == 1 and changes['changed_files'] == 1:
#keep print to console statement for testing purposes
#print changes['user']['login']
with open('c:\\commit_filechange.csv', 'a+') as f:
csv_writer = csv.writer(f)
csv_writer.writerow([changes['user']['login'], changes['title'], changes['commits'], changes['changed_files']])
one_commit_one_file_change_pr()
Here is a solution which I believe is common practice. Set up a file called global_var.py (or something like it) and store global variables there. (See this post by #systemizer) Here is a simplified version of what will work with your code:
script1.py
if __name__=='__main__':
user = raw_input('GitHub username: ')
with open('global_var.py','w') as f:
f.write("user = '%s'" % user)
password = 'blah'
gh = '<login(user, password)>'
script2.py
from script1 import gh
from global_var import user
print user
print gh
This answer is purely based on my assumption that you want get the user credentials only once and want to re-use it "n" number of times later in other scripts.
Here is how I would do it ,
Update1
Your problem also has to do with how you want to organize and run your scripts, the below examples work if you have bundled your scripts in to python package and run them
For. eg
but if you are planning to run individual scripts separately , then you have no other option but to invoke the login prompt for each script unless you plan to completely remove asking for user credentials and use the pre-configured data from a file.
script1
from github3 import login
USER_CREDS = None # Store raw input credentials here ,not recomended
GIT_USERS = {} # global dict to store multiple users with username as key
def getCredentials():
global USER_CREDS
if USER_CREDS:
print "returning user credentials"
return USER_CREDS
print "gettting user credentials"
user = raw_input("Enter Username")
pwd = raw_input("Enter Password")
USER_CREDS = (user, pwd)
return USER_CREDS
def do_login():
global GIT_USERS
user, pwd = getCredentials()
if not GIT_USERS.get(user, None):
gh = login(user, password=pwd)
GIT_USERS[user] = gh
return GIT_USERS[user]
in other scripts
from script1 import do_login
# the first time do_login() is called it asks for user credentials
print do_login()
# the next times it just returns the previously collected one
print do_login()
Example 2
script 3
from script1 import do_login
creds = do_login()
script 4
from script3 import creds
from script1 import do_login
print creds
print do_login() # No input prompt would be provided
I've searched in Internet for a while, but I haven't found out anything useful...
I want to do something as simple as redirect the page to the listing page (folder) after save/create an AT content type.
I already know I have to use validate_integrity.cpy and write my redirect's logic there, but the file isn't run...
So far this is an example of my validate_integrity.cpy:
## Script (Python) "validate_integrity"
##title=Validate Integrity
##bind container=container
##bind context=context
##bind namespace=
##bind script=script
##bind state=state
##bind subpath=traverse_subpath
##parameters=
##
from Products.Archetypes import PloneMessageFactory as _
from Products.Archetypes.utils import addStatusMessage
request = context.REQUEST
errors = {}
errors = context.validate(REQUEST=request, errors=errors, data=1, metadata=0)
import pdb; pdb.set_trace()
if errors:
message = _(u'Please correct the indicated errors.')
addStatusMessage(request, message, type='error')
return state.set(status='failure', errors=errors)
else:
message = _(u'Changes saved.')
stat = 'created'
# Redirection after saving edition forms
redirects = {'Multifile': context.aq_parent.absolute_url_path() + '/multifile'}
import pdb; pdb.set_trace()
if context.portal_type in redirects:
redirect = 'redirect_to:string:${portal_url}' + redirects[context.portal_type]
state.setNextAction(redirect)
else:
stat = 'success'
addStatusMessage(request, message)
return state.set(status=stat)
RESOLUTION
I just needed to write the following upgrade step:
from Acquisition import aq_inner, aq_parent
from Products.CMFCore.utils import getToolByName
def upgrade(tool):
portal = aq_parent(aq_inner(tool))
setup = portal.portal_setup
setup.runImportStepFromProfile('profile-my.addon:default', 'skins')
Useful info about upgrade steps here
Your *.metadata file might be missing or have an action that routes to a different location than you expect: http://docs.plone.org/old-reference-manuals/forms/using_cmfformcontroller.html
The default metadata for content_edit lives in Products/Archetypes/skins/archetypes/content_edit.cpy.metadata:
...
[actions]
action.success = traverse_to:string:validate_integrity
action.success_add_reference = redirect_to:python:object.REQUEST['last_referer']
action.failure = traverse_to_action:string:edit
action.next_schemata = traverse_to_action:string:edit
Is your action button value "success"?
I've got a program I would like to use to input a password and one or multiple strings from a web page. The program takes the strings and outputs them to a time-datestamped text file, but only if the password matches the set MD5 hash.
The problems I'm having here are that
I don't know how to get this code on the web. I have a server, but is it as easy as throwing pytext.py onto my server?
I don't know how to write a form for the input to this script and how to get the HTML to work with this program. If possible, it would be nice to make it a multi-line input box... but it's not necessary.
I want to return a value to a web page to let the user know if the password authenticated successfully or failed.
dtest
import sys
import time
import getopt
import hashlib
h = hashlib.new('md5')
var = sys.argv[1]
print "Password: ", var
h.update(var)
print h.hexdigest()
trial = h.hexdigest()
check = "86fe2288ac154c500983a8b89dbcf288"
if trial == check:
print "Password success"
time_stamp = time.strftime('%Y-%m-%d_%H-%M-%S', (time.localtime(time.time())))
strFile = "txt_" + str(time_stamp) + ".txt"
print "File created: txt_" + str(time_stamp) + ".txt"
#print 'The command line arguments are:'
#for i in sys.argv:
#print i
text_file = open(strFile, "w")
text_file.write(str(time_stamp) + "\n")
for i in range(2, len(sys.argv)):
text_file.write(sys.argv[i] + "\n")
#print 'Debug to file:', sys.argv[i]
text_file.close()
else:
print "Password failure"
You'll need to read up on mod_python (if you're using Apache) and the Python CGI module.
Take a look at django. It's an excellent web framework that can accomplish exactly what you are asking. It also has an authentication module that handles password hashing and logins for you.