We Keep Coding

flask sqlalchemy update sql raw command does not provide any response

I'm trying to perform an update using flask-sqlalchemy but when it gets to the update script it does not return anything. it seems the script is hanging or it is not doing anything.
I tried to wrap a try catch on the code that does not complete but there are no errors.
I gave it 10 minutes to complete the update statement which only updates 1 record and still, it will not do anything for some reason.
When I cancel the script, it provides an error Communication link failure (0) (SQLEndTran) but I don't think this is the root cause of the error because on the same script, I have other sql scripts that works ok so the connection to db is good
what my script does is get some list of filenames that I need to process (I have no issues with this). then using the retrieved list of filenames, I will look into the directory to check if the file exists. if it does not exists, I will update the database to tag the file as it is not found. this is where I get the issue, it does not perform the update nor provide an error message of some sort.
I even tried to create a new engine just for the update script, but still I get the same behavior.
I also tried to print out the sql script first in python before executing. I ran the printed sql command on my sql browser and it worked ok.
The code is very simple, I'm not really sure why it's having the issue.
#!/usr/bin/env python3
from flask_sqlalchemy import sqlalchemy
import glob
files_directory = "/files_dir/"
sql_string = """
select *
from table
where status is null
"""
# ommited conn_string
engine1 = sqlalchemy.create_engine(conn_string)
result = engine1.execute(sql_string)
for r in result:
engine2 = sqlalchemy.create_engine(conn_string)
filename = r[11]
match = glob.glob(f"{files_directory}/**/{filename}.wav")
if not match:
print('no match')
script = "update table set status = 'not_found' where filename = '" + filename + "' "
engine2.execute(script)
engine2.dispose()
continue
engine1.dispose()

it appears that if I try to loop through 26k records, the script doesn't work. but when I try to do by batches of 2k records per run, then the script will work. so my sql string will become (added top 2000 on the query)
sql_string = """
select top 2000 *
from table
where status is null
"""
it's manual yeah, but it works for me since I just need to run this script once. (I mean 13 times)

Python OS Commands Not Work Due To New App Execution

I need to check GoldenGate processes' lag. In order to this, I execute Goldengate than I try to run GoldenGate's own commands "info all".
import subprocess as sub
import re
import os
location = str(sub.check_output(['ps -ef | grep mgr'], shell = True)).split()
pattern = re.compile(r'mgr\.prm$')
print(type(location))
for index in location:
if pattern.search(index)!=None:
gg_location = index[:-14] + "ggsci"
exec_ggate = sub.call(str(gg_location))
os.system('info all')
Yet, when I execute the GoldenGate it opens a new GoldenGate's own shell. So, I think because of that, Python cannot be able to do run "info all" command. How can I solve this problem? If there is missing information, please inform me.
Thank you in advance,

For command automation on Golden Gate you have the following information in the Oracle docs: https://docs.oracle.com/goldengate/1212/gg-winux/GWUAD/wu_gettingstarted.htm#GWUAD1096
To input a script
Use the following syntax from the command line of the operating system.
ggsci < input_file
Where:
The angle bracket (<) character pipes the file into the GGSCI program.
input_file is a text file, known as an OBEY file, containing the commands that you want to issue, in the order, they are to be issued.
Taking your script (keep into mind I don't know to code into python) you can simply execute a shell command in python in the following way:
import os
os.system("command")
So try doing this:
import os
os.system("ggsci < input_file")
Changing the input_file as indicated by the docs.
I think you will have an easier time doing it this way.

Unable to open/include a YARA file

I created a script that analyzes files based on yara rules ( the yara are the ones from this repository https://github.com/Yara-Rules/rules). My script import a yara file that include all other rules.When i try to compile it, i receive a syntax error: "can't open include file: rules_for_files\Antidebug_AntiVM_index.yar", pointing me to one of the rules. I tried to exclude it but it continue points to others.
I tried to use different versions of python: 1.i used python2.7 and i received the mentioned error in both case when i use a binary string/raw string. About python 3.5 when i mentioned a binary string like the one from my code sample, the interpreter broke/reset(in case i use a GUI). How can i resolve this? Thank you.
rules = yara.compile(filepaths={
"malware_set1 rules": b'C:/Users/g_bondrila/Desktop/phishme/functionalitati/yararules/importyara.yar'})
def yara_match(file_path, rules=rules):
try:
matches = rules.match(file_path, timeout=60)
return matches
#except TimeoutError:
# print("the time is running out")
except:
print("something")

Try giving the directory path as below:
"C:\\Users\\g_bondrila\\Desktop\\phishme\\functionalitati\\yararules\\importyara.yar"
Since Python doesn't reads single slash for a path in windows.

Why do I get a SyntaxError <unicode error> on my import statement? Should this not be simple?

Here's my first simple test program in Python. Without importing the os library the program runs fine... Leading me to believe there's something wrong with my import statement, however this is the only way i ever see them written. Why am I still getting a syntax error?
import os # <-- why does this line give me a syntax error?!?!?! <unicode error> -->
CalibrationData = r'C:/Users/user/Desktop/blah Redesign/Data/attempts at data gathering/CalibrationData.txt'
File = open(CalibrationData, 'w')
File.write('Test')
File.close()
My end goal is to write a simple program that will look through a directory and tabularize data from relevant .ini files within it.
Well, as MDurant pointed out... I pasted in some unprintable character - probably when i entered the URL.

.cgi problem with web server

The code
#!/usr/bin/env python
import MySQLdb
print "Content-Type: text/html"
print
print "<html><head><title>Books</title></head>"
print "<body>" print "<h1>Books</h1>"
print "<ul>"
connection = MySQLdb.connect(user='me', passwd='letmein', db='my_db') cursor = connection.cursor() cursor.execute(“SELECT name FROM books ORDER BY pub_date DESC LIMIT 10”)
for row in cursor.fetchall():
print "<li>%s</li>" % row[0]
print "</ul>"
print "</body></html>"
connection.close()
I saved it as test.cgi to my web server. I run it by www.mysite.com/test.cgi unsuccessfully
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
How can you solve the problem?
[edit] after the first answer
test.cgi is executable (I run $ chmod +x test.cgi)
I use Apache.
I have this in .bashrc export PATH=${PATH}:~/bin
Python module MySQLdb is installed.
The code does not have smart quotes.
[edit] after the second answer
you're getting that error because you
haven't installed the MySQLdb module
that Python needs to talk to a MySQL
database
I installed MySQLdb to my system. The module works, since I can import them.
However, I still get the same error whet I go to the www.[mysite].com/test.cgi.
[edit]
I am not sure about the questions
Are the connect() parameters correct? Is MySQL running on localhost
at the default port?
I run MySQL on my server. Is the question about the connect() parameters relevant here?
Is the SELECT statement correct?
You mean whether I have my SQL statements such as SELECT statement correct?
I have not used any SQL queries yet.
Do I need them here?

Any number of issues can cause the error you are seeing:
Is test.cgi executable (chmod 755) on the server?
Is the directory in which you placed test.cgi designated as a ScriptAlias location or have the ExecCGI option enabled (or equivalent if you're not using Apache)?
Is python in the system PATH or in the PATH in the Web server's startup environment?
Is the MySQLdb Python library installed?
Are the connect() parameters correct? Is MySQL running on localhost at the default port?
Is the SELECT statement correct?
If you're sure that python is found (test using the simplest possible script or by logging into the Web server if you can and typing which python) then you can get much better debug output by adding the following to the top of your script just below the shebang:
import cgitb
cgitb.enable()
More details: http://docs.python.org/library/cgitb.html
Additionally, if you have shell access to the Web server, try running python and just typing:
>>> import MySQLdb
If the command returns with no error, you have your answer for #4 above. If an error is printed, you will need to get MySQLdb installed into the Web server's Python installation.
EDIT: Looking more closely at the top of your question, I see that the code was scraped from an illustrative example at the very beginning of the Django Book. As such, I might expand #5 above to include the caveat that, of course, the requisite database, tables, user, and permissions need to be set up on the MySQL installation available to the Web server.

I've tidied up the code a bit by inserting linebreaks where necessary and replacing smart quotes with " and '. Do you have any more luck with the following? Can you run it from a terminal just by typing python test.cgi?
#!/usr/bin/env python
import MySQLdb
print "Content-Type: text/html"
print
print "<html><head><title>Books</title></head>"
print "<body>"
print "<h1>Books</h1>"
print "<ul>"
connection = MySQLdb.connect(user='me', passwd='letmein', db='my_db')
cursor = connection.cursor()
cursor.execute("SELECT name FROM books ORDER BY pub_date DESC LIMIT 10")
for row in cursor.fetchall():
print "<li>%s</li>" % row[0]
print "</ul>"
print "</body></html>"
connection.close()

The error in http://dpaste.com/8866/ is occurring because you are using "curly quotes" instead of standard ASCII quotation marks.
You'll want to replace the “ and ” with ". Just use find and replace in your text editor.

Make sure you're saving the file with correct line endings. i.e. LF only on unix, or CR/LF for Windows. I just recently had the exact same problem...