I am having a bit of an issue. First off, I know that this code is able to stand alone and not be in a class but I would prefer that it is in a class. Second, when I run the code, I get this error TypeError: set_options() takes exactly 2 arguments (1 given) .
Here is my code. If anyone could point me in the right direction, I would appreciate it. I'm assuming that the set_options method isn't getting my jobj instance. Am I correct in assuming that and how would one go about fixing this? ps. I do have the correct imports and here is my py command at terminal python test.py radar 127.0.0.1 hashNumber testplan:speed
class TransferStuff(object):
tool = sys.argv[1]
target = sys.argv[2]
hash = sys.argv[3]
options = sys.argv[4]
def set_options(self, test_options):
option_arr = test_options.split(',')
new_arr = [i + ':{}'.format(i) for i in option_arr if ':' not in i]
for i in option_arr:
if ':' in i:
new_arr.append(i)
d = {}
for i in new_arr:
temp = i.split(':')
d[temp[0]] = temp[1]
return d
data = {'target': target, 'test': tool, 'HASH': hash,
'options': set_options(options)}
def write_to_json(self):
"""Serialize cli args and tool options in json format.
Write stream to json file.
"""
with open('envs.json', 'w') as fi:
json.dump(TransferStuff.data, fi)
if __name__ == "__main__":
try:
jobj = TransferStuff()
jobj.write_to_json()
Your method is inside a class, you need to create a instance of the class:
transfer_stuff_instance = TransferStuff()
And call the method with this instance:
transfer_stuff_instance.ser_options(options)
Related
I have the following code.
class ResourceManagerSuspension is inheriting from class TestCase. TestCase executes tests alphabetically methods like testA will be executed before testB.
so testLinkData method is executed before testSuspension and self.link is getting it's value in testLinkData.
I am initializing a variable "self.link" and when method testSuspension gets executed it calls getQueryValues.
My question is why
"self.link" is not being passed in getQueryValues method ?
Can anyone explain how the self mechanism is working here, maybe I am doing something wrong?
class ResourceManagerSuspension(TestCase):
#classmethod
def setUpClass(self):
logger.info("=== Starting setup ===")
# self.rm_obj = ResourceManager(agg='mapper-prefix1-aggs.A.m2-test.akamai.com')
self.rm_obj = ResourceManager()
self.rm_leader = self.rm_obj.get_rm_leader()
logger.info("RM lead target is %s" % (self.rm_leader))
self.found = ""
self.link = ""
logger.info("self.link is : {}".format(self.link))
logger.info("self in setUpClass is : {}".format(self.__dict__))
#the name should be get link number to get started etc
def testLinkData(self):
linkValues = {}
#get a random link
sqlquery = "select * from rm_links_debugonly where adjuster_reason not like '\%suspend\%' and ip=" + self.rm_leader + " and link!=0 limit 1"
link_obj = self.rm_obj.get_link_info(query=sqlquery)
for row in link_obj:
self.link = row.link
self.getDynamicConfig()
logger.info("self.link is : {}".format(self.link))
logger.info("self in testLinkData is : {}".format(self.__dict__))
def testSuspension(self):
if not ResourceManagerSuspension.found:
#get the 'control_reason' from "rm_link_load_control_debugonly" and 'adjuster_cap' from "rm_links_debugonly" before submitting the dynamic config
self.control_reason_without_config, self.adjuster_cap_without_config = self.getQueryValues()
logger.info("param not present in the file, submitting with the param")
self.rm_obj.dyamic_config_submit(fromLocation = self.rm_obj.dynamic_config_modified, to = self.rm_obj.dynamic_config_incoming)
else:
logger.info("param is already present, removing it and submitting the config")
self.rm_obj.dyamic_config_submit(fromLocation = self.rm_obj.dynamic_config_modified, to = self.rm_obj.dynamic_config_incoming)
logger.info("self.link is : {}".format(self.link))
logger.info("self in testSuspension is : {}".format(self.__dict__))
def getQueryValues(self):
logger.info("self in getQueryValues is : {}".format(self.__dict__))
logger.info("self.link is : {}".format(self.link))
The output of last line in code is
[05:55:39.709 test_suspension_2: 61 I] self.link is :
Unit tests are supposed to be able to run independently from one another, entailing that getQueryValues should be able to run before or after testLinkData, but in your implementation, getQueryValues must run after testLinkData in order for the output you expect to be given.
To remedy this, you must write set-up utility methods that your test methods can call as they run, i.e. to give self.link a value independent of another test.
The purpose of this form is to let users enter a lot of places (comma separated) and it'll retrieve the phone, name, website. Have it working in a python IDE, no problem, but having issues putting it into my webapp.
I'm getting the error Exception Value: Can't pickle local object 'GetNums.<locals>.get_data' at the line where a is assigned. I checked the type of inputText and verified that it is indeed a list. So, I'm not sure why it won't pickle.
def GetNums(request):
form = GetNumsForm(request.POST or None)
if form.is_valid():
inputText = form.cleaned_data.get('getnums')
# all experimental
inputText = inputText.split(',')
def get_data(i):
#DON'T FORGET TO MOVE THE PRIMARY KEY LATER TO SETTINGS
r1 = requests.get('https://maps.googleapis.com/maps/api/place/textsearch/json?query=' + i + '&key=GET_YOUR_OWN')
a = r1.json()
pid = a['results'][0]['place_id']
r2 = requests.get('https://maps.googleapis.com/maps/api/place/details/json?placeid=' + pid + '&key=GET_YOUR_OWN')
b = r2.json()
phone = b['result']['formatted_phone_number']
name = b['result']['name']
try:
website = b['result']['website']
except:
website ='No website found'
return ' '.join((phone, name, website))
v = str(type(inputText))
with Pool(5) as p:
a = (p.map(get_data, inputText))
# for line in p.map(get_data, inputText):
# print(line)
#code assist by http://stackoverflow.com/a/34512870/5037442
#end experimental
return render(request, 'about.html', {'v': a})
It's actually barfing when trying to pickle get_data, which is a nested function/closure.
Move get_data out of GetNums (and agh rename it to snake_case please) and it should work.
I'm using SQLObject and I've got the following:
# update foo if it exists, otherwise create a new one
if self.foo_exists:
Foo.get(foo_id).set(name = foo['name'], ip = foo['ip'], port = foo['port'], mode = foo['mode'], max_conn = foo['max_conn'])
else:
Foo(name = foo['name'], ip = foo['ip_address'], port = foo['port'], mode = foo['mode'], max_conn = foo['max_conn'])
It works well, but I'd really like to remove the duplication. I'm passing the exact same arguments to Foo() as I am to .set().
I tried passing a dict but SQLObject doesn't support that.
try:
obj = Foo.get(foo_id)
obj.set(**d)
except SQLObjectNotFound:
Foo(**d)
It supports dict, use **dict. It is also a duplicate, but not so loud.
What I am trying to do is get a report using Boto and Get_Report(). I have a valid report id, and seem to be having problems getting the python call correct. The call requires the variable ReportId and I have been trying various methods of getting it to accept the id.
from boto.mws.connection import MWSConnection
import sys, getopt
def main(argv):
MarketPlaceID = 'a'
Merchant = 'a'
AccessKeyID = 'a'
SecretKey = 'a'
program_name = sys.argv[0]
MarketPlaceID = sys.argv[1]
Merchant = sys.argv[2]
AccessKeyID = sys.argv[3]
SecretKey = sys.argv[4]
print 'MarketplaceID is ', MarketPlaceID
print 'Merchant is ', Merchant
print 'AccessKey is ', AccessKeyID
print 'Secret key is ', SecretKey
conn = MWSConnection(AccessKeyID,SecretKey)
conn.SellerId = Merchant
conn.Merchant = Merchant
conn.MarketplaceId = MarketPlaceID
myId = '1432456045'
# sample one
conn.get_report(ReportId=myId)
# sample two
conn.get_report(myId)
# sample three
conn.get_report(myId,)
# sample four
conn.get_report(1432456045)
# sample five
conn.get_report('1432456045')
if __name__ == "__main__":
main(sys.argv1:)
With each of these five variations, I get the same key error: KeyError: 'GetReport requires R+e+p+o+r+t+I+d argument(s)'. This example will take the four arguments (MarketPlaceID, Merchant, AccessKeyID, and SecretKey) and tries to call get_report. The id number is a valid report, I have successfully used the scratchpad and downloaded the csv file.
The object is to call the get_report() call with a value of an inventory file. The call within boto takes an argument of an integer. I have tried various versions from an integer, integer encased within a string, and a list of one integer. Each of these failed with the same access key error.
I know that the five samples that I have above don't work as I have them. Has anyone successfully got the Get_report call to work? And could you point me to what I am doing incorrect?
Thank you
I ran into the same issue. The problem is actually in connection.py.
#requires('ReportId')
Should read
#requires(['ReportId'])
Hope this helps!
http://boto.readthedocs.org/en/latest/ref/mws.html#boto.mws.connection.MWSConnection.get_report
get_report() worked for me when I used something similar to this:
from boto.mws.connection import MWSConnection
MarketPlaceID = 'a'
MerchantID = 'a'
AccessKeyID = 'a'
SecretKey = 'a'
mws = MWSConnection(AccessKeyID,SecretKey)
mws.SellerId = MerchantID
mws.Merchant = MerchantID
mws.MarketplaceId = MarketPlaceID
report = mws.get_report(ReportId='1234567890')
I'm a bit new to Python dev -- I'm creating a larger project for some web scraping. I want to approach this as "Pythonically" as possible, and would appreciate some help with the project structure. Here's how I'm doing it now:
Basically, I have a base class for an object whose purpose is to go to a website and parse some specific data on it into its own array, jobs[]
minion.py
class minion:
# Empty getJobs() function to be defined by object pre-instantiation
def getJobs(self):
pass
# Constructor for a minion that requires site authorization
# Ex: minCity1 = minion('http://portal.com/somewhere', 'user', 'password')
# or minCity2 = minion('http://portal.com/somewhere')
def __init__(self, title, URL, user='', password=''):
self.title = title
self.URL = URL
self.user = user
self.password = password
self.jobs = []
if (user == '' and password == ''):
self.reqAuth = 0
else:
self.reqAuth = 1
def displayjobs(self):
for j in self.jobs:
j.display()
I'm going to have about 100 different data sources. The way I'm doing it now is to just create a separate module for each "Minion", which defines (and binds) a more tailored getJobs() function for that object
Example: minCity1.py
from minion import minion
from BeautifulSoup import BeautifulSoup
import urllib2
from job import job
# MINION CONFIG
minTitle = 'Some city'
minURL = 'http://www.somewebpage.gov/'
# Here we define a function that will be bound to this object's getJobs function
def getJobs(self):
page = urllib2.urlopen(self.URL)
soup = BeautifulSoup(page)
# For each row
for tr in soup.findAll('tr'):
tJob = job()
span = tr.findAll(['span', 'class="content"'])
# If row has 5 spans, pull data from span 2 and 3 ( [1] and [2] )
if len(span) == 5:
tJob.title = span[1].a.renderContents()
tJob.client = 'Some City'
tJob.source = minURL
tJob.due = span[2].div.renderContents().replace('<br />', '')
self.jobs.append(tJob)
# Don't forget to bind the function to the object!
minion.getJobs = getJobs
# Instantiate the object
mCity1 = minion(minTitle, minURL)
I also have a separate module which simply contains a list of all the instantiated minion objects (which I have to update each time I add one):
minions.py
from minion_City1 import mCity1
from minion_City2 import mCity2
from minion_City3 import mCity3
from minion_City4 import mCity4
minionList = [mCity1,
mCity2,
mCity3,
mCity4]
main.py references minionList for all of its activities for manipulating the aggregated data.
This seems a bit chaotic to me, and was hoping someone might be able to outline a more Pythonic approach.
Thank you, and sorry for the long post!
Instead of creating functions and assigning them to objects (or whatever minion is, I'm not really sure), you should definitely use classes instead. Then you'll have one class for each of your data sources.
If you want, you can even have these classes inherit from a common base class, but that isn't absolutely necessary.