The code is running and inserting but I get errors in command prompt saying 'tab' is not recognised as an internal or external command,operable program or batch file.
What is the mistake i have done and How can i fix it ?
Here is the python code :
updatedb.py
import sqlite3 as db
import urllib
import re
import sys
url=sys.argv[1]
htmltext=urllib.urlopen(url).read()
regex='<title>(.+?)</title>'
pattern=re.compile(regex)
title= re.findall(pattern,htmltext)
print title[0]
id="1"
conn=db.connect('insertlinks.db')
cursor=conn.cursor()
with conn:
cursor.execute('insert into records (id,keyword) values(?,?)',(id,title[0]))
#print "inserted"
#conn.close()
The above code is called as follows:
import urlparse
import os
import urllib
from bs4 import BeautifulSoup
url="http://www.google.com"
urls=[url]
visited=[url]
try:
while len(urls)>0:
htmltext=urllib.urlopen(urls[0]).read()
soup=BeautifulSoup(htmltext)
urls.pop(0)
for tag in soup.findAll('a',href=True):
tag['href']=urlparse.urljoin(url,tag['href'])
if tag['href'] not in urls and tag['href'] not in visited:
os.system("python scraper/insertlinks.py %s" % (tag['href']))
os.system("python scraper/updatedb.py %s" % (tag['href']))
urls.append(tag['href'])
visited.append(tag['href'])
except:
print 'error in 1'
EDIT:
The problem is in tag['href']. Its value is http://maps.google.co.in/maps?hl=en&tab=il. The tab in the url is creating a problem. How do i solve it?
Use the subprocess.call() method instead of os.system()
The & in the url is what is causing the problem.
On Windows:
Command1 & Command2
Means run Command1 then run Command2
The error you are getting is a Windows error, not a Python error. Somehow one or both of your os.system calls are passing "tab" as a command to the Windows command line.
I suspect this is because many of the urls on the google.com page have ?tab=Wx or &tab=wT or other similar arguments tacked on to the url. The ? shouldn't hurt anything, but the & may be interpreted as the start of another command. (If that is the case, I would expect you to receive errors about a lot more than just 'tab' though.)
Related
I'm trying to use Applescript to fire a python script. When I fire it from terminal, everything works fine, but when I fire from Applescript, it appears to run but nothing happens.
I've tried all manner of combinations for everything I can find in searches and other posts for using "python file.py" or "/usr/bin/python file.py" with "#!/usr/bin/env python" and "#!/usr/bin/python".
If I enter "which python" in terminal, I get "/usr/bin/python"
Right now I have both scripts broken down to their base components. I'll eventually be using Applescript to pass a file path into python using sys.argv[1] (which is why I'm using Applescript to fire the python script) but I'm not even that far along yet as the below doesn't work yet.
Applescript
do shell script "/usr/bin/python $HOME/Desktop/test.py"
Python
#!/usr/bin/python
import sys
import os
# The notifier function
def notify(title, subtitle, message):
t = '-title {!r}'.format(title)
s = '-subtitle {!r}'.format(subtitle)
m = '-message {!r}'.format(message)
os.system('terminal-notifier {}'.format(' '.join([m, t, s])))
# Calling the function
notify(title = 'Message Test',
subtitle = 'Test1:',
message = 'Test2')
sys.exit(0)
The python script sends a notifier message. Every time I run in terminal, I receive the message without issue. Every time I run the applescript to do as shell script it runs without error-ing in AS, but no message comes from Python.
Anyone have thoughts on where I've gone wrong?
Does it work for you if you use a full path to the binary? It worked for me in both BBEdit and Smile (script editor). My path is:
/Applications/terminal-notifier-2.0.0/terminal-notifier.app/Contents/MacOS/terminal-notifier
So I used:
#!/usr/bin/python
import sys
import os
# The notifier function
def notify(title, subtitle, message):
t = '-title {!r}'.format(title)
s = '-subtitle {!r}'.format(subtitle)
m = '-message {!r}'.format(message)
os.system('/Applications/terminal-notifier-2.0.0/terminal-notifier.app/Contents/MacOS/terminal-notifier {}'.format(' '.join([m, t, s])))
# Calling the function
notify(title = 'Message Test',
subtitle = 'Test1:',
message = 'Test2')
sys.exit(0)
I have problem with my python cmd script.
I don't know why it does not work. Maybe something wrong with my code.
Im trying to run the program in cmdline through my python script.
And Im getting error in bash "sh: 1: Syntax error: redirection unexpected"
pls help Im just biologist :)
Im using spyder (anaconda)/Ubuntu
#!/usr/bin/python
import sys
import os
input_ = sys.argv[1]
output_file = open(sys.argv[2],'a+')
names = input_.rsplit('.')
for name in names:
os.system("esearch -db pubmed -query %s | efetch -format xml | xtract -pattern PubmedArticle -element AbstractText >> %s" % (name, output_file))
print("------------------------------------------")
output_file is a file object. When you do "%s" % output_file, the resulting string is something like "<open file 'filename', mode 'a+' at 0x7f1234567890>". This means that the os.system call is running a command like
command... >> <open file 'filename', mode 'a+' at 0x7f1234567890>
The < after the >> causes the "Syntax error: redirection unexpected" error message.
To fix that, don't open the output file in your Python script, just use the filename:
output_file = sys.argv[2]
I got similar error on following line:
os.system('logger Status changed on %s' %s repr(datetime.now())
Indeed, as nomadictype stated the problem is in running plain OS command. The command may include special characters. In my case this was <.
So instead of changing OS command significantly, I just added quotes and this works:
os.system('logger "Status changed on %s"' %s repr(datetime.now())
Quotes make content of passed parameter invisible for shell.
I am using the following code to call two different scripts:
soup=BeautifulSoup(htmltext)
title=soup.find('title')
os.system("python scraper/updatedb.py %s" % (title))
os.system("python scraper/insertlinks.py %s" % (tag['href']))
The second one is running but the first one is not running. Please help.
updatedb.py and insertlinks.py work fine when run individually.
updatedb.py is as follows:
import sqlite3 as db
import sys
print "inserted"
key=sys.argv[1]
id="1"
conn=db.connect('store.db')
cursor=conn.cursor()
with conn:
cursor.execute('insert into records (id,keyword) values(?,?)',(id,key))
conn.close()
Os.system always returns the code, error code if not executed the command and 0 if it is executed successfully.
result_code = os.system("your command")
you can check the error message on google by using python error code
i have a python script on the server
#!/usr/bin/env python
import cgi
import cgitb; #cgitb.enable()
import sys, os
from subprocess import call
import time
import subprocess
form = cgi.FieldStorage()
component = form.getvalue('component')
command = form.getvalue('command')
success = True
print """Content-Type: text/html\n"""
if component=="Engine" and command=="Start":
try:
process = subprocess.Popen(['/usr/sbin/telepath','engine','start'], shell=False, stdout=subprocess.PIPE)
print "{ans:12}"
except Exception, e:
success = False
print "{ans:0}"
When I run this script and add the component and command parameters to be "Engine" and "Start" respectively - it starts the process and prints to the shell
"""Content-Type: text/html\n"""
{ans:12}
but most importantly - it starts the process!
however, when I run the script by POSTing to it, it returns {ans:12} but does not run the process which was the whole intention in the first place. Any logical explanation?
I suspect it's one of two things, firstly your process is probably running but your python code doesn't handle the output so do:
process = subprocess.Popen(['/usr/sbin/telepath','engine','start'], shell=False, stdout=subprocess.PIPE)
print process.stdout.read()
This is the most likely and explains why you see the output from the command line and not the browser, or secondly because the script is run through the browsers as the user apache and not with your userid check the permission for /usr/sbin/telepath.
I am trying to run a Python program to see if the screen program is running. If it is, then the program should not run the rest of the code. This is what I have and it's not working:
#!/usr/bin/python
import os
var1 = os.system ('screen -r > /root/screenlog/screen.log')
fd = open("/root/screenlog/screen.log")
content = fd.readline()
while content:
if content == "There is no screen to be resumed.":
os.system ('/etc/init.d/tunnel.sh')
print "The tunnel is now active."
else:
print "The tunnel is running."
fd.close()
I know there are probably several things here that don't need to be and quite a few that I'm missing. I will be running this program in cron.
from subprocess import Popen, PIPE
def screen_is_running():
out = Popen("screen -list",shell=True,stdout=PIPE).communicate()[0]
return not out.startswith("This room is empty")
Maybe the error message that you redirect on the first os.system call is written on the standard error instead of the standard output. You should try replacing this line with:
var1 = os.system ('screen -r 2> /root/screenlog/screen.log')
Note the 2> to redirect standard error to your file.