Do you know repl.it?
I am coding python on this site.
And my goal is creating Web Scraper.
I think this code is clean.
But I'm getting an error:
AttributeError: 'function' object has no attribute 'text'
My code:
import requests
indeed_result = requests.get
("https://kr.indeed.com/jobs?q=python&l=%EC%9D%B8%EC%B2%9C")
print(indeed_result.text)
Surely, I have requests package installed.
Please give me some advice
You just need to remove the back to new line after get like this:
import requests
indeed_result = requests.get("https://kr.indeed.com/jobs?q=python&l=%EC%9D%B8%EC%B2%9C")
print(indeed_result.text)
if you want to continue typping in the next line just add a backslash \ as follows:
indeed_result = requests.get\
("https://kr.indeed.com/jobs?q=python&l=%EC%9D%B8%EC%B2%9C")
Removing back to new line after get
try this
import requests
res = requests.get("https://kr.indeed.com/jobs?q=python&l=%EC%9D%B8%EC%B2%9C")
print(res.text)
# result if success 200
Related
I am trying to make a simple web scraper with python to get stock data. My code was working not long ago and I don't believe I changed anything but now I'm getting the following error:
File "tradingProgram.py", line 69, in
dataArr.append(i.find('div',{'class':'tv-screener-table__symbol-right-part'}).find('a').text)
AttributeError: 'NoneType' object has no attribute 'find'
This is the part of the code that handles beautifulsoup:
content = requests.get("https://www.tradingview.com/markets/stocks-usa/market-movers-gainers/")
soup = BeautifulSoup(content.text,'html.parser')
stockData = soup.find_all('tr',{'class':'tv-data-table__row tv-data-table__stroke tv-screener-table__result-row'})
print(len(stockData))
for i in stockData:
print(i)
dataArr.append(i.find('div',{'class':'tv-screener-table__symbol-right-part'}).find('a').text)
Ok, so I have an error and I don't know why. I want to request a bunch of links from a file
while True:
code = file.readline().replace("\n", "")
r = request.get(code)
source = r.post(code)
print(source.text)
input()
Can someone please help me?
import requests at the beggining of the file
I am learning how to scrape web information. Below is a snippet of the actual code solution + output from datacamp.
On datacamp, this works perfectly fine, but when I try to run it on Spyder (my own macbook), it doesn't work...
This is because on datacamp, the URL has already been pre-loaded into a variable named 'response'.. however on Spyder, the URL needs to be defined again.
So, I first defined the response variable as response = requests.get('https://www.datacamp.com/courses/all') so that the code will point to datacamp's website..
My code looks like:
from scrapy.selector import Selector
import requests
response = requests.get('https://www.datacamp.com/courses/all')
this_url = response.url
this_title = response.xpath('/html/head/title/text()').extract_first()
print_url_title( this_url, this_title )
When I run this on Spyder, I got an error message
Traceback (most recent call last):
File "<ipython-input-30-6a8340fd3a71>", line 11, in <module>
this_title = response.xpath('/html/head/title/text()').extract_first()
AttributeError: 'Response' object has no attribute 'xpath'
Could someone please guide me? I would really like to know how to get this code working on Spyder.. thank you very much.
The value returned by requests.get('https://www.datacamp.com/courses/all') is a Response object, and this object has no attribute xpath, hence the error: AttributeError: 'Response' object has no attribute 'xpath'
I assume response from your tutorial source, probably has been assigned to another object (most likely the object returned by etree.HTML) and not the value returned by requests.get(url).
You can however do this:
from lxml import etree #import etree
response = requests.get('https://www.datacamp.com/courses/all') #get the Response object
tree = etree.HTML(response.text) #pass the page's source using the Response object
result = tree.xpath('/html/head/title/text()') #extract the value
print(response.url) #url
print(result) #findings
I am getting an Attribute error on line 8:browser.visit("https://www.facebook.com").
"AttributeError: 'NoneType' object has no attribute 'visit'"
This is my code:
browser = Browser('firefox')
browser.visit('https://www.facebook.com')
browser.find_by_id('email').fill("")
browser.find_by_id('pass').fill("")
browser.find_by_id('loginbutton').first.click()
browser.visit('')
while x != 0:
browser.find_by_css('textarea').fill('')
browser.find_by_id('u_0_1c').click()
It pulled up with a bunch of errors at first but I fixed those, I don't understand this error? If someone could maybe explain why or what is happening it would be much appreciated.
I think you need to have import the Browser package from splinter like this
from splinter import Browser
browser = Browser('firefox)
browser.visit('https://www.facebook.com')
import requests
import json
s = requests.Session()
s.params["key"] = "MY_KEY"
s.params["cx"] = "MY_CX"
s.params["num"] = 1
result = s.get('https://www.googleapis.com/customsearch/v1', params={"q": "Search query"})
Everything is working fine and I do get a result back but it's JSON. What I want is one thing. Since my result only gives 1 search result, I want the link of that search result. From I've understood that link value is under the keys "item" and then "link". I've tried lots of things but I keep getting either one of these errors below.
TypeError: 'Response' object is not subscriptable
NameError: name 'json' is not defined
AttributeError: 'Response' object has no attribute 'read'
TypeError: the JSON object must be str, not 'Response'
What am i doing wrong and what's the solution?