Can not get data with dictCursor on Amazon - python

I have the following code that works fine when I run it from my localhost, but it fails when I run it from Amazon. There is no differences in the code, its using the same database connection.
I use pymysql and I use dictCursor to fetch the data by column name. This is how I create the link:
self.link = self.db.cursor(pymysql.cursors.DictCursor)
My problem is when I try to fetch the data. The following simple code snippet fails:
try:
self.link.execute('SELECT * FROM crawler_data WHERE id="%d" LIMIT 1' % id_crawl);
row = self.link.fetchone()
except pymysql.err.Error as ex:
print "Can not fetch crawler data from database: %s" % ex
print "Before"
try:
if row["address"]:
print "Filter Address: %s" % row["address"]
except Exception as ex:
print "Could not filter the data: %s" % ex.message
print "After"
It does not even print "Filter Address". I guess that means that it fails on if row["address"]:. It does print both "Before" and "After".
I have also tried row[u"address"]: without any success.
When I print the row. I get the following data printed:
{u'address': u'Ekhammargr\\xe4nd 7'}
So questions:
My except is done, and it prints "Could not filter the data:". It does not print ex.message. How do I get the exception or error that is causing the try to fail?
I'm using the same code and the same database connection, fetching the same data. It works fine on my localhost Windows machine, but not on my Amazon Web Services Elastic Beanstalk Linux App. How could this be?
EDIT:
I tried changing the if statement to:
if True:
print "Filter Address"
It still failed on Amazon. If it was indentation error it should'nt work on localhost either... Any ideas?
Here is the complete function: http://pastebin.com/haJGFuj3

For some unknown reason, when I rewrote the code exactly the same, it started working. I was only using tabs as indentations and no spaces, I tripple checked and I didnt get any indentation error. So I'm not sure what could be causing these issues.
But when I rewrote the code from scratch, with same code. It worked.
EDIT: I was using Sublime Text 2 as editor and utf8 as encoding.

Related

Is there a proper way to encode a search from Google Cloud vs Locally Running a Python Script?

Airtable Python Wrapper is able to find a field using Airtable.MAtch when executed from the local machine, but It cannot make the match on Google Cloud.
Declared:
-- coding: UTF-8 --
search = d['TITLE'] + d['OFFICE']
try: #THIS WONT WORK?? WHAT?
record = airtable.match('mysearch', search)
airtable.update(record['id'], d, typecast=True)
print (' UPDATED ')
except:
try: #NO PROBLEM INSERTING - TOO BAD ITS NOT CATCHING DUPLICATES
airtable.insert(d,typecast=True)
print (' INSERTED ')
Expected to find the search var in {mysearch} field.
Works great on my machine.
Will NOT find the match in GCM.
The answer is that this is we have Fiverr, clearly.

Using After The Deadline spellchecker seems to return empty list

I am trying to use the AfterTheDeadline API to read some output of my program. This is my tester code:
ATD.setDefaultKey('qg2324242424')
errors = list(ATD.checkDocument("Looking too the wter. Fixing your writing typoss."))
print(errors)
for error in errors:
print("%s error for: %s **%s**" % (error.type, error.precontext, error.string))
print("some suggestions: %s" % (", ".join(error.suggestions),))
I put list() before ATD.checkDocument because otherwise it just returns a generator object. errors is just an empty list, so the for loop returns nothing. I am using Miguel Ventura's python 3+ version to run my code. Any help on how to get the expected output would be appreciated.

Error swallowed

Problem
I am connecting with JayDeBeApi to SQL Server 2017 and running a script like:
SELECT ... INTO #a-temp-table
DELETE FROM a-table
INSERT INTO a-table SELECT FROM #a-temp-table
DELETE #a-temp-table
During step 3 i get the following error:
Cannot insert duplicate key row in object 'dbo.a-table' with unique index 'UQ_a-table'. The duplicate key value is (11, 0001, 3751191, T70206CAT, 0000).
Instead of ~360k records, only ~180k get inserted. So step 3 aborts.
The temp table however gets deleted. So step 4 completes.
I am able to fix the error. But with JayDeBeApi, I am not seeing the error.
It seems like everything went fine from the Python point of view.
My goal is to capture those errors to handle them appropriately.
Any idea how to achieve that?
What I've tried
My Python code looks like.
try:
localCursor = dbConnection.cursor()
x = localCursor.execute(query)
logInfo("Run script %s... done" % (scriptNameAndPath), "run script", diagnosticLog)
except Exception as e:
logError("Error running sql statement " + scriptNameAndPath + ". Skipping rest of row.",
"run script", e, diagnosticLog)
myrow = skipRowAndLogRecord(startRowTime, cursor, recordLog)
continue
x = localCursor.execute(myqrystm) completes successfully, so no exception is thrown. x is None and while inspecting localCursor, I see no sign of any error message(s)/code(s)
Step 3 should be all-or-none so the a-table should be empty following the duplicate key error unless your actual code has a WHERE clause.
Regarding the undetected exception, add SET NOCOUNT ON as the first statement in the script. That will suppress DONE_IN_PROC messages that will interfere with script execution unless your code handles multiple result sets.
https://learn.microsoft.com/en-us/sql/t-sql/language-elements/try-catch-transact-sql?view=sql-server-2017
-- Create procedure to retrieve error information.
CREATE PROCEDURE usp_GetErrorInfo
AS
SELECT
ERROR_NUMBER() AS ErrorNumber
,ERROR_SEVERITY() AS ErrorSeverity
,ERROR_STATE() AS ErrorState
,ERROR_PROCEDURE() AS ErrorProcedure
,ERROR_LINE() AS ErrorLine
,ERROR_MESSAGE() AS ErrorMessage;
GO
BEGIN TRY
-- Generate divide-by-zero error.
SELECT 1/0;
END TRY
BEGIN CATCH
-- Execute error retrieval routine.
EXECUTE usp_GetErrorInfo;
END CATCH;

IndexError: list index out of range from client side

Happy Friday. This puzzle is really a pain in my neck, any advice will be appreciated. Thanks a ton.
I am on Python 2.7 with Windows 7.
I send a piece of command to device via model-client model (im at client side) and wait for a response (charging status) from device. Then I would like to extract some value from the list of return. And BANG! Error pops out! "IndexError: list index out of range".
My Python code is in below:
def chrg_Test():
try:
self.s.send(getchar)
time.sleep(2)
except:
print 'Chrg cmd sent failed...'
print 'Start Chrg Test...'
result = self.s.recv(1024)
time.sleep(3)
ExtPwrPresent = (result.split('\n'))[11]
print 'ExtPwrPresent is: ', ExtPwrPresent
chrg_Test()
And the Error is:
ExtPwrPresent = (result.split('\n'))[11]
IndexError: list index out of range
Some other details: what is wired is , the error pops out like 50% or more. And rest time it works good. So it got some temper and on and off randomly which is like an annoying babe~ The correct output should be like:
Result list is:
getchar
Label,Value
FuelPercent,31
BatteryOverTemp,0
ChargingActive,0
ChargingEnabled,0
ConfidentOnFuel,0
OnReservedFuel,0
EmptyFuel,0
BatteryFailure,0
ExtPwrPresent,0
ThermistorPresent,1
BattTempCAvg,23
VBattV,14.42
VExtV,0.00
Charger_mAH,0
Discharge_mAH,200
ExtPwrPresent is: ExtPwrPresent,0
So far, the solution I tried is - turn off the server socket and turn it on again, then run this client code. It works most time, but I think it is just a temporary way, I bet there is some better way to solve it.

extracting ip address from raw_input

I have compiled a script in python 2.6.5, The script is simply to automate my most used functions for nmap and log the output to a txt file on the desktop.
I haven't written this all myself as i am still only learning python. I found an update script for updating backtrack and used that as a template for indentation and commands and modified it and added some of my own stuff to give me what i needed.
The problem i'm having is with this block
def nmap_target():
try: ip = raw_input(" [>] Enter ip to scan: ")
except KeyboardInterrupt:
print "\n [>] Exiting!\n"
sleep(1)
sys.exit()
print " [>] Attempting targeted scan.\n"
#print ip
if subprocess.Popen("nmap ip >> //root/Desktop/Target.txt && tail /root/Desktop/Target.txt",shell=True).wait() == 0:
print "\n"
print " [>] Targeted scan completed successfully!\n"
menu_main()
else:
print "\n"
print " [>] Nmap scan failed.\n"
The idea behind it is that it asks the user to input an ip address into raw_input() and call it ip, I'm trying to then pass it to nmap using subprocess.Popen("nmap ip as can be seen above.
The problem is its just writing nmap ip rather than printing the ip address and then returning errors, It even tries to scan out of my network, every time i test it i make sure the internet cable is unplugged from my router, This causes a bug with my internet test though, so if you try running the code bellow you may need to hash out the internet_check() option in the menu_main() section if you have your internet unplugged from the router
I have spent 4 days on this now and its driving me mad, At first i thought i needed to convert the number to floating point so i tried that and still the same, I've tried all sorts of things and spent hours trawling the internet looking for an answer to no avail.
I am now convinced its not the command i'm trying that is to blame i think it is the exit statement, I have tried putting "print ip" right after the line where it says "print " [>] Attempting targeted scan.\n" and sure enough the ip address that was entered is displayed on the screen, That proved to me that raw_input() is working. As soon as i move it anywhere bellow that it fails, This suggests to me that it must be either a problem with the exit statement or maybe indentation, I'm not sure though.
I have also tried hashing out the keyboard interrupt as well as the couple of lines bellow, i tried moving the try: and if statements around and even tried other commands instead but it just wont work aaaarrrrrgggghhhhhh
Would i be right in thinking that the ip is being entered into raw_input() and then the file ip that was created that holds the ip address is being destroyed before i can pass it to subprocess.Popen("nmap ip.
Like i mentioned i didn't write this script from scratch and this is my first project like this so i've got a lot to learn, I've been all through the python man pages and looked through all sorts of tutorials but just can't figure this out.
Any help will be much appreciated
i will post the full script if anyone is interested,just as soon as i can figure out how to post code properly
You need to seperate the variable from the string! Try this :D
if subprocess.Popen('nmap '+ip+' >> //root/Desktop/Target.txt && tail /root/Desktop/Target.txt',shell=True).wait() == 0:
Hope it helps!
EDIT - If for some reason python takes the raw input as an integer, convert it to string like so:
if subprocess.Popen('nmap '+str(ip)+' >> //root/Desktop/Target.txt && tail /root/Desktop/Target.txt',shell=True).wait() == 0:
Python doesn't like to concatenate str and int types, or so it tells when my script fails :P I am pretty sure your ip variable will be str type though so the first example should work.
Cheers!
You need to format the string properly or the string ip won't be interpreted at all, i.e. it won't get replaced wth the actual IP. Try something like:
cmd = "nmap ${0} >> [....] root/Desktop/Target.txt".format(ip)
if subprocess.Popen(cmd):
You could also use the % operator:
cmd = "nmap %s >> [....] root/Desktop/Target.txt" % ip

Categories