This is my script which is supposed to parse a list of domains (each seperated by returns) in a .txt file, separate them into individual domain names, send a request to a whois site with the domain name, check the response to see if it is available, and if it is, write it to a new file. so i get a list of only available names.
The problem? It's pretty simple i just dont know the language well enough, I dont know how to get the domain name in a string format so that the request to the whois site is like this :
http://whois.domaintools.com/google.com
Apparently the %s thing is not working.
Code:
#!/usr/bin/python
import urllib2, urllib
print "Domain Name Availability Scanner."
print "Enter the Location of the txt file containing your list of domains:"
path = raw_input("-->")
wordfile = open(path, "r")
words = wordfile.read().split("n")
words = map(lambda x: x.rstrip(), words)
wordfile.close()
for word in words:
req = urllib2.Request("http://whois.domaintools.com/%s") % (word)
source = urllib2.urlopen(req).read()
if "This domain name is not registered" in source:
f = open("success.txt", "a")
f.write("%s\n") % (word)
f.close()
break
error in terminal:
python domain.py
Domain Name Availability Scanner.
Enter the Location of the txt file containing your list of domains:
-->a.txt
Traceback (most recent call last):
File "domain.py", line 13, in <module>
req = urllib2.Request("http://whois.domaintools.com/%s") % (word)
TypeError: unsupported operand type(s) for %: 'instance' and 'str'
Fix the parentheses:
req = urllib2.Request("http://whois.domaintools.com/%s" % (word))
As well as:
f.write("%s\n" % word)
:)
You need to use:
req = urllib2.Request("http://whois.domaintools.com/%s" % word)
# ...
f.write("%s\n" % word)
Use:
f.write("%s\n" % word)
Check out this link, it should explain how this formatting works: http://docs.python.org/release/2.5.2/lib/typesseq-strings.html
Related
The code saves a list of URLs. I want to take the lines of text and covert them to links within an HTML file by adding the A tags and place those links within properly formatted HTML code.
#!/usr/bin/env python
import sys
import os
import shutil
try:
from googlesearch import search
except ImportError:
print("No module named 'google' found")
#keyword query user input
query = raw_input('Enter keyword or keywords to search: ')
#print results from search into a file called keyword.txt
with open("keyword.txt","w+") as f:
for j in search(query, tld='co.in', lang='en', num=10, start=0, stop=200, pause=3):
f.write("%s\n" % j)
f.close()
#add keyword to list of keywords file
sys.stdout=open("keywords","a+")
print (query)
sys.stdout.close()
#rename file to reflect query input
os.rename('keyword.txt',query + ".txt")
#move created data file to proper directory and cleanup mess
source = os.listdir("/home/user/search/")
destination = "/home/user/search/index/"
for files in source:
if files.endswith(".txt"):
shutil.copy(files,destination)
os.remove(query + ".txt")
Expected results would be an HTML file with clickable links
Based on your comment, it appears that you are struggling to write the url string obtained from the search function into a file along with the required HTML tags. Try:
with open("keyword.txt","w+") as f:
for j in search(query, tld='co.in', lang='en', num=10, start=0, stop=200, pause=3):
f.write('{1} <br>\n'.format(j,j))
Which will write each url and add hyperlinks to the url. You might want to print <html> ... </html> and <body> ... </body> tags as well to keyword.txt. This can be done like
with open("keyword.txt","w+") as f:
f.write('<html> \n <body> \n')
for j in search(query, tld='co.in', lang='en', num=10, start=0, stop=200, pause=3):
f.write('{1} <br>\n'.format(j,j))
f.write('\n</body> \n </html>')
And, you don't have to close the file using f.close() if you use with open see: https://stackoverflow.com/a/8011836/937153
Personally, I prefer format over %. I will be careful about presenting a comparison between the two here. You can see Python string formatting: % vs. .format for a detailed discussion on this topic.
Here is the error message I get:
Traceback (most recent call last):
File "datalister.py", line 10, in <module>
wordlist.write(words)
TypeError: expected a string or other character buffer object
My code:
import random, sys
from urllib import urlopen
word_url = "http://scrapmaker.com/data/wordlists/dictionaries/rockyou.txt"
words = []
for word in urlopen(word_url).readlines():
print "Doing it now..."
wordlist = open('wordlist.txt', 'w')
wordlist.write(words)
wordlist.close()
print "File written successfully!"
The error is exactly what it says: you have to give write a string, not a list. You're one letter off: you need to write word, not words. That simple change fixes the program ... I think. It's running nicely for me.
wordlist = open('wordlist.txt', 'w')
for word in urlopen(word_url):
print "Doing it now..."
wordlist.write(word)
print "File written successfully!"
wordlist.close()
Since it's one word per line, you don't really need to read in the entire file before you start writing things; just use the default generator to grab the lines as you need them.
Also, note that you never put anything in to the list words. If all you want is the word list, you may as well remove that variable.
This is the code i got to work have a try:
import random, sys
from urllib import urlopen
word_url = "http://scrapmaker.com/data/wordlists/dictionaries/rockyou.txt"
wordlist = open('wordlist.txt', 'w')
for word in urlopen(word_url):
print "Added: ", word
wordlist.write(word)
print "Successfully written all words to file!"
wordlist.close()
I am trying to write a simple Python SMTP enumeration script, which reads usernames from a text file (filename supplied as the second argument - sys.argv[2]), and checks them against an SMTP server (hostname or ip supplied as the first argument - sys.argv[1]. I found something that is kind of close, and tweaked it a bit, like so:
#!/usr/bin/python
import socket
import sys
users = sys.argv[2]
for line in users:
line = line.strip()
if line!='':
users.append(line)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((sys.argv[1], 25))
fn = s.makefile('rwb')
fn.readline()
fn.write('HELO testing.com \r\n')
fn.flush()
fn.readline()
for user in users:
fn.write('VRFY %s\r\n' % user)
fn.flush()
print '%s: %s' % (user, fn.readline().strip())
fn.write('QUIT\r\n')
fn.flush()
s.close()
However, when I run the script (for example):
./smtp-vrfy.py 192.168.1.9 users.txt
It results in an error:
File "./smtp-vrfy.py", line 10, in
users.append(line)
AttributeError: 'str' object has no attribute 'append'
What am I doing wrong? How can I fix it? Perhaps there is an easier way to accomplish what I'm trying to do?
users is a file name, but you're not reading it. Instead, see what happens:
>>> users = "users.txt"
>>> for line in users:
... print(line)
...
u
s
e
r
s
.
t
x
t
You probably want:
with open(users) as f:
for line in f:
# ...
Even better:
filename = sys.argv[2]
with open(filename) as f:
users = [line.strip() for line in f.readlines() if line]
I am getting the error when trying to open a url I obtained from reading data from a .txt file in python using match.group(). This is my code below for where the error comes up. Any help as too how this can be corrected would be very much appreciated.
with open('output.txt') as f:
for line in f:
match = re.search("(?P<url>https?://docs.google.com/file[^\s]+)", line)
if match is not None:
urltest = match.group()
print urltest
print "[*] Opening Map in the web browser..."
kml_url = "urltest"
try:
webbrowser.get().open_new_tab(kml_url)
Since you have not provided what you are trying to parse I can only guess but this should pretty much work for your url:
>>> import re
>>> match = re.search('(?P<url>https:\/\/docs.google.com\/file[a-zA-z0-9-]*)', 'https://docs.google.com/fileCharWithnumbers123')
>>> match.group("url")
'https://docs.google.com/fileCharWithnumbers123'
So I'm new in python and I desperately need help.
I have a file which has a bunch of ids (integer values) written in 'em. Its a text file.
Now I need to pass each id inside the file into a url.
For example "https://example.com/[id]"
It will be done in this way
A = json.load(urllib.urlopen("https://example.com/(the first id present in the text file)"))
print A
What this will essentially do is that it will read certain information about the id present in the above url and display it. I want this to work in a loop format where in it will read all the ids inside the text file and pass it to the url mentioned in 'A' and display the values continuously..is there a way to do this?
I'd be very grateful if someone could help me out!
Old style string concatenation can be used
>>> id = "3333333"
>>> url = "https://example.com/%s" % id
>>> print url
https://example.com/3333333
>>>
The new style string formatting:
>>> url = "https://example.com/{0}".format(id)
>>> print url
https://example.com/3333333
>>>
The reading for file as mentioned by avasal with a small change:
f = open('file.txt', 'r')
for line in f.readlines():
id = line.strip('\n')
url = "https://example.com/{0}".format(id)
urlobj = urllib.urlopen(url)
try:
json_data = json.loads(urlobj)
print json_data
except:
print urlobj.readlines()
lazy style:
url = "https://example.com/" + first_id
A = json.load(urllib.urlopen(url))
print A
old style:
url = "https://example.com/%s" % first_id
A = json.load(urllib.urlopen(url))
print A
new style 2.6+:
url = "https://example.com/{0}".format( first_id )
A = json.load(urllib.urlopen(url))
print A
new style 2.7+:
url = "https://example.com/{}".format( first_id )
A = json.load(urllib.urlopen(url))
print A
Python 3+
New String formatting is supported in Python 3 which is a more readable and better way to format a string.
Here's the good article to read about the same: Python 3's f-Strings
In this case, it can be formatted as
url = f"https://example.com/{id}"
Detailed example
When you want to pass multiple params to the URL it can be done as below.
name = "test_api_4"
owner = "jainik#test.com"
url = f"http://localhost:5001/files/create" \
f"?name={name}" \
f"&owner={owner}" \
We are using multiple f-string here and they can be appended by ''. This will keep them in the same line without inserting any new line character between them.
For values which have space
For such values you should import from urllib.parse import quote in your python file and then quote the string like: quote("firstname lastname")
This will replace space character with %20.
The first thing you need to do is know how to read each line from a file. First, you have to open the file; you can do this with a with statement:
with open('my-file-name.txt') as intfile:
This opens a file and stores a reference to that file in intfile, and it will automatically close the file at the end of your with block. You then need to read each line from the file; you can do that with a regular old for loop:
for line in intfile:
This will loop through each line in the file, reading them one at a time. In your loop, you can access each line as line. All that's left is to make the request to your website using the code you gave. The one bit your missing is what's called "string interpolation", which allows you to format a string with other strings, numbers, or anything else. In your case, you'd like to put a string (the line from your file) inside another string (the URL). To do that, you use the %s flag along with the string interpolation operator, %:
url = 'http://example.com/?id=%s' % line
A = json.load(urllib.urlopen(url))
print A
Putting it all together, you get:
with open('my-file-name.txt') as intfile:
for line in intfile:
url = 'http://example.com/?id=%s' % line
A = json.load(urllib.urlopen(url))
print A