Using Python to webscrape a list - python

Looking to Pass Python a list then using a combinate of Beautiful soup and requests, pull the corresponding peice of information for each web page.
So i have a list of around 7000 barcodes that i want to pass to this site 'https://www.barcodelookup.com/' (you just add the barcode after the backslash), then pull back the manufacturer of that product which is in the span "product-text".I'm currently trying to get it to run with the below;
from bs4 import BeautifulSoup
import requests
source = requests.get('https://www.barcodelookup.com/194398882321')
soup = BeautifulSoup(source, 'lxml')
#print(soup.prettify())
price = soup.find('span', {'class' : 'product-text'})
print(price.text)
This gives an error as below;
TypeError: object of type 'Response' has no len()
Any help would be greatly appreciated, thanks


If you inspect the source, you will see that the response status is 403 and the overall source.text reveals that the website is protected by Cloudflare. This means that using requests is not really helpful for you. You need the means to overcome the 'antibot' protection from Cloudflare. Here are two options:
1. Use a third party service
I am an engineer at WebScrapingAPI and I can recommend you our web scraping API. We're preventing detection by using various proxies, IP rotations, captcha solvers and other advanced features. A basic example of using our API for your scenarios is:
import requests
API_KEY = '<YOUR_API_KEY>'
SCRAPER_URL = 'https://api.webscrapingapi.com/v1'
TARGET_URL = 'https://www.barcodelookup.com/194398882321'
PARAMS = {
"api_key":API_KEY,
"url": TARGET_URL,
"render_js":1,
"timeout":"20000",
"proxy_type": "residential",
"extract_rules":'{"elements":{"selector":"span.product-text","output":"text"}}',
}
response = requests.get(SCRAPER_URL, params=PARAMS )
print(response.text)
Response:
{"elements":["\nUPC-A 194398882321, EAN-13 0194398882321\n","Media ","Sony Uk ","\n1-2-3: The 80s CD.\n"]}
2. Build an undetectable web scraper
You can also try building a more 'undetectable' web scraper on your end. For example, try using a real browser for your scraper, instead of requests. Selenium would be a good place to start. Here is an implementation example:
from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
BASE_URL = 'https://www.barcodelookup.com/194398882321'
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 5000)
driver.get(BASE_URL)
html = driver.page_source
soup = BeautifulSoup(html, 'html.parser')
price = soup.find('span', {'class' : 'product-text'})
print(price)
driver.quit()
In time though, Cloudflare might flag your 'fingerprint' and block your requests. Some more things you could add to your project are:
Residential proxies
Advanced Selenium Evasions

Related

How to scrape an API response that is requested by the target website?

I'd like to scrape content of a website that is requested asynchronous and not visible in the source code.
How can I await the website's request? I need to sniff its traffic somehow, but couldn't find anything yet.
I'm looking something like that (pseudo code):
import requests
from bs4 import BeautifulSoup
page = requests.get("http://target.tld")
traffic = page.sniff_traffic(seconds=10)
for req in traffic:
print(req) # http://api.target.tld
soup = BeautifulSoup(page.content, "html.parser")
Any ideas?

You can't do that with BeautifulSoup, you need to use something which mimics a web browser, such as Selenium with Geckodriver.

BeautifulSoup scraping the wrong page

This didnt work for me. I am scraping the website slider.kz. But when i scrape the website it pulls the html of the main page instead of the specified url.
My code
import fake_useragent
from bs4 import BeautifulSoup
from requests import get
headers = {'user-agent': fake_useragent.UserAgent().random}
soup = BeautifulSoup(get('https://slider.kz/#unravel', headers=headers).text, 'html.parser')
for i in soup.find_all('a'):
print(i.get('href'))
The actual output returned:
https://twitter.com/x_slider
https://paypal.me/xslider
/
What it should return extracted using selenium :
Selenium code used to extract the desired output
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get('https://slider.kz/#unravel')
for i in driver.find_elements(By.TAG_NAME, 'a'):
if i.text != '':
print(i.get_attribute('href'))
output
https://slider.kz/download/371745468_456554892/238/cs3-3v4/s/v1/acmp/cOE3Il1BHU9ESEFr9Hz4pL7WfYMASX1jY3IxqYGfVbDA7njmXutAbM3kHNfpwOzSBQWa2IZQ7huupcnHjAdwPgptzXqreGX0tmqzvcVCEsQn7_VVdO2neH6DlOB61uNtIBCcnHAMlApXwBfbMccaKZDif1_ZPf1e-4rkVEyHP9VQBiCGZQ/TK%20from%20Ling%20tosite%20sigure%20-%20Unravel.mp3?extra=null
https://slider.kz/download/474499200_456477539/242/cs3-2v4/s/v1/acmp/68YuXV09VzsZx8eNd_eKHNOjhjlx7Ae93T9h6eA4wjSJruGLjWdJqvGRkKfvEla9GYPHVopebj2BMYIsTf9R2a5-ztCHImKELMaUZQBuM1_OtyVbt2_AxQqxfs9ziB-1F1_vSJ2wJQO8EfhsuQi8Ouz-qH5nyqJPMAaPhuKuqeJ7XDdIew/Anata%20No%20Senin%20-%20Unravel.mp3?extra=null
https://slider.kz/download/-2001620886_66620886/237/cs3-1v4/s/v1/acmp/9F7u-H32JUQGkAjlBPuZTbwaM2ZkjLchrhJxmqDZjIODWGIF6ClRnA49X4_rMHf5E8b9SPQUJqKtEwS_4cq1EkwDf97msqqps6gnKzAwJRNU7lhLkaJ6hqd0yZg27uBAvK6wMpizrt3HpGUIuURFmYDMWX7ggp802aWW51gkn_rQOAE0Gg/FalKKonE%20-%20unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/474499141_456425057/270/cs3-1v4/s/v1/acmp/sD64r4KNC9Ye4CfOMggOFmSA-ric7pUSKLaN_bfGIArMK7MSwjhu4lHpICSxWKCDniNKGTOOiGcg_6b6VUfLRuvSzn-Cakp4YU2BFw-5DJ2KAlFf4Duz6txtUAen3mslUB-vOSV8mT48PYf4VuoxFkltIb-kYqzkXgvsr56GGrXFzYo54w/Dima%20Lancaster%20-%20unravel%20(Tokyo%20Ghoul%20OP).mp3?extra=null
https://slider.kz/download/474499133_456468907/240/psv4/s/v1/amp/I4gkKu5n8-ePABgx_5blRV5izpZRPdm6o3_5C619b-f0tDciBsojh2EkLvtQ3xref_TYPVL7vBktamf1mdxDmxi3xZz1XC_HKAwsjUqU03dJpvytOBXsiCZ6kgP2FQ/Yung%20Anime%20-%20Unravel.mp3?extra=null
https://slider.kz/download/-2001476389_94476389/229/cs3-6v4/s/v1/acmp/A6FBXBqjCCmhlytjFG4rAKFNfD78JTsqzgNAZToGubFqSZ_t1OQ_4gXhqO9qxFs3nPX2V6u745hB34NMrpGSvAEEKhfV0u5jYjHKeAdqBTbp05hvkdg2aJ6lJMFO3gjA9DZjanfjDL1fjhDC02e50HYr-K-ILhQkjdrixmzi8Ed5PQJfXA/Nika%20Lenina%20-%20Unravel.mp3?extra=null
https://slider.kz/download/-2001355458_97355458/244/cs3-1v4/s/v1/acmp/ZRd1eY7d7KVAi23V-pkv91B0rBxyH6TBawtgTp0u_agc_1WbMS1061fhyFRFye9GHrUcaKX2KI1HIEzV0SXmJbSgy3q86HHZq08hTiwHdE27YVaxwn2HqW0r_AmOLtPMt6EcSNjcRyKUc-8sPnJ-PYKIPGeHoF67WUBsDbBteD1LjaG8fQ/TK%20from%20Ling%20tosite%20sigure%20-%20unravel.mp3?extra=null
https://slider.kz/download/474499285_456610507/236/cs3-5v4/s/v1/acmp/yQ75TttqYQXX7TBSoFeRrACN7JTw4pDVbn7gQ0uYHgKyUtvarViLiz47siDjWti8wr-Udt9pYektb5tp-VY68B-nFSq8SQ1Amqo3YCROcOFMhfA-xxT__aZOT2eLjyC_8KqHeLGi6Dzc-Jo52P-30wiWCh09ScW4t8rmQGjspFzDuWMkqQ/Yung%20Anime%20-%20Unravel.mp3?extra=null
https://slider.kz/download/474499192_456485073/91/cs3-5v4/s/v1/acmp/CnNkh81L0geRARmPlCx6HDtwarMIshv51_2gNlOHnFHKfJ3sTWbOXjzT7-SBqMmIMIusJ8lRN9tsjx4djCww8g3zd5n-x8SxDVVcMxhXKx-XqtOdT1q88mABUQhKY4ZcLos_XBYe7lx7C9CzcBabYFpBYZRZ-9x9M-HX52DfvajtnggW5w/Yendorami%20-%20Unravel%20(Opening%201)%20%5BFrom%20%22Tokyo%20Ghoul%22%5D.mp3?extra=null
https://slider.kz/download/-2001941575_98941575/185/cs3-3v4/s/v1/acmp/fmdPR0KKIvzX-evg6umhtRemEihc16Hc0s0W6fanPFxDNR8vTeCaTGdId8L20cp4r43dNQO3XYNbdp4ftt0mwIxb89SFMmcn5bBMZCymCay9cegxcXx4dDhc2329j0Evv_KSB3wpIzA5IR9cBf34DyOdDkd6L1b_ygIpDGF5E1_yp4acPA/Collosia%20-%20Unravel%20(Tokyo%20Ghoul).mp3?extra=null
https://slider.kz/download/474499243_456360717/225/cs3-4v4/s/v1/acmp/i3ZW2FQYrSPIkVmjzhKxasRpNU2sHQA24dwNyYtNb7SpQTg9bTqGlKbpeknyi2-z74quCFgDKR6Vox0JFmyJJVHQmQkGrPxTgnngADz-Iaxa4q3MVUAJqzTZJgu32Ur-FRa3n-lcN3wy/Trinthepianist%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22)%20%5BPiano%20Version%5D.mp3?extra=null
https://slider.kz/download/-2001187532_103187532/144/cs3-5v4/s/v1/acmp/NuVJyRij62WtsZrXRq60T7I1G8mfLu-Q01ITWVXxnn9DvIa9hiRGNUfMX2HF7qJMNv6nLC8GzobY8nmh71W9ulIpnXI44tGkyBKcy3lyAq1rTpBjLXssQiC94zA07P238siyirG99-EQsSAFL29rMcG8UYN9hXqC5g9-Xc4hyhT5vuNsHA/KinovariSquad%20-%20Unravel.mp3?extra=null
https://slider.kz/download/474499138_456497179/330/cs3-1v4/s/v1/acmp/OClerb_S3_yMpf46bD6zx0ObnCHcC6OQTPnK5Pry3qn8XVTOSLd_xRzCzUodFJoaLYbrjnKNHktd_J6PZu2m1hkZM3N--UNOqZkPIMubcOCJfcBBloN5BoCve2WvgftZ3yWubFzzWHiVpLnosbeogAzf_bMpblrewEeJ0-xC7toQeGTJ6Q/SLSMusic%20-%20unravel%20-%20Slow%20Version%20(From%3A%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/-2001343575_62343575/235/cs3-4v4/s/v1/acmp/O6pRAZYWRIaUHjj-3d-I7wDamPE2M1MziJsmpF1XKCPGPbOKR3UV_A5vqtpbBqPSPochs34hpYKTTsHS5eAL_FiKJwwKvbUFqD3exRAAbDouMGxfW7YEpmMkN8NcpVz5QmH0lEJypGWO84b9cRnPSB8ZAkWNyKpGue26FP9Qc-jFLAj6Kw/Geek%20Music%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/474499226_456421440/235/cs3-4v4/s/v1/acmp/ErptzsuuaoawWhx_finD4Pb5GRbG05r7PrMIQy5LYmfw9QfJqjPBRGK-vRTrZnbsfSKoCA8K0odPCNvb_3rC3Mv-97bmbyv188pzC2PIKET0DntUvYYE__cD-KCVTnTyUpx3M5fe5hNdYjsb9UkYVngGlSBMNrA4OQhrp3-15Fybbvrfsw/Geek%20Music%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/474499206_456447563/243/cs3-6v4/s/v1/acmp/Ll3SqtO1lm7C2W7j7T6-bCEZ03fOS_MAFwvXUqzizClbi3AJS4LYDddyPM_iI531Tv-t1hUYKQFeJCUsF9MCM_ox93UUSaP_M9Zp0uVdB4PSpBXcu6xgqoKCDkbQ69CcnGpARuXxeZ5XPo6hJox-8xvMuZnfoT_b76BasOFB7rcYjFPDsA/%E6%AD%8C%E3%81%A3%E3%81%A1%E3%82%83%E7%8E%8B%20-%20unravel(%E3%82%AA%E3%83%AA%E3%82%B8%E3%83%8A%E3%83%AB%E3%82%A2%E3%83%BC%E3%83%86%E3%82%A3%E3%82%B9%E3%83%88%3ATK%20from%20%E5%87%9B%E3%81%A8%E3%81%97%E3%81%A6%E6%99%82%E9%9B%A8)%5B%E3%82%AC%E3%82%A4%E3%83%89%E3%83%A1%E3%83%AD%E3%83%87%E3%82%A3%E7%84%A1%E3%81%97%E3%82%AB%E3%83%A9%E3%82%AA%E3%82%B1%5D.mp3?extra=null
https://slider.kz/download/474499222_456568196/98/psv4/s/v1/amp2/G5Qoz-PWlD8w86QFGNvVLmXfps93Kv7CVsbbVFpo4e5LjDQRJ_yuY4Ha3Nrp10xVXf5hur40alIvROP88ncZTLhPICCjkBn3mM7QGpaFgBQf3SEib-JCMELaZoMg36gpsc6vdsRKzSqjecjnhDAvjkixFTc05wTyL3b7/Piano%20Cat%2C%20Anime%20Cat%20-%20Unravel%20Tokyo%20Ghoul%20(Winter).mp3?extra=null
https://slider.kz/download/474499266_456496174/238/cs3-2v4/s/v1/acmp/ZSnEUsWBGhnXj5XDpV5_RYtliMXOCl747j9MfPXEJut-jDHGl9lI3z-Wyrmo1fnrb7QYMdmCyCFGChtLb_Jlnhj47w9llLDSfOeT-ejt6HrEFCVeG9pSJfb1x2Orv9ls1AWiprtX9TBylxyJYkTTMCT8lXPxyT_1Lhh63I_Vclx__QWHAg/The%20Unknown%20Songbird%20-%20Unravel%20(Tokyo%20Ghoul).mp3?extra=null
https://slider.kz/download/-2001091207_65091207/243/cs3-4v4/s/v1/acmp/KQtj35XBV_qATTaJDm3ULd5piLxC8bhqrobCvIFfHM35M5YDIBlSZQyfTvcmGktv5upmL40wFiQuX65il7CSkTyvGgCubXJXch30BTk_8VgSnOQitmFNebJzW_ADxbaBwFpJsZRZ1La5UKahLdafzttj-qiWjk7B9uuiI2PejUViDS7oGQ/Bibi%20Gamino%20-%20Unravel%20(Opening%20de%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/-2001815387_86815387/189/cs3-5v4/s/v1/acmp/ao_2H9LXIsXS3wGH47_DjS8MTmTGiH8wBvOjh7ZTyXXT6gGCzwqO9j3fW940H4MSUnTlarmjLDf4kFN9izLPboP6opybIVo1zv0efcOASrk03xIjwIFbsZuB7Ub1vpl_U9jevxKne9xT7-8ecNhrm8_MrrcqEt2yUw6ApNIIDiFIjnPm9Q/Baur%20Karbon%20-%20Unravel.mp3?extra=null
https://slider.kz/download/-2001343001_91343001/91/cs9-18v4/s/v1/acmp/zwfryFVsrFggm-hEVOd6RiXeyEUooW9tZMLBrntNjiPJOxatNavKMSQ6tdcBDb5DfN-5oz89p6eXNd61Td00hMgQXQ3pfU_uRhQXPtSNRXH26a_vH5x1z6VVndmwxWikty-kmyuctXkSLygSr-__9qnxcZYaNplAvE79Hl5maQC14zIq3A/Ren%20Avel%20-%20Unravel%20(Tokyo%20Ghoul).mp3?extra=null
https://slider.kz/download/-2001459950_102459950/222/cs3-5v4/s/v1/acmp/dXsMP_kZ98lEdBVsmwcSr0zvYGcr5Spyn8RNNVYxfMDehWA3Jbcq7BpGMLoaKcgCMpQ4PpHszAAC3bkW4wNbKiFD6LumIhx0_T2C2xekVJsfR1xb8cVtOgwLUMFnffOWz3GULiHXBKoPS2xjpWf1TmkQXdDhcpmKqokCzhx0yvKBagDO0Q/Jonathan%20Young%20-%20Unravel%20(Full%20Version).mp3?extra=null
https://slider.kz/download/474499249_456500220/95/cs9-5v4/s/v1/acmp/kI7HhI6sOV-jdnMnoifzOJwbZwidKai2LQ6nQOuVeZ73W2eyOcQUDIDLEuDWL6qhNZM4ATnhITDeeI1W34s3kaY03Bf9CGzRm2RTJHtIbF7U1ZHL5GJ_lKj8DR27Ulz_4whswvN7p4sUAuL8LHEyYCFjLrgQVlLTb_xfVL9j7q5_4ndF5w/Tony%20Gu%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22)%20%5BOrchestral%20Arrangement%5D.mp3?extra=null
https://slider.kz/download/-2001472007_90472007/95/cs9-2v4/s/v1/acmp/D25FbaAzhvubfYuHGk82DiPeG6gHBuhK6YwgRt3UV2gnf-s7ynvbhcNomwhgm7kqj6RZ5cqVRgRhzUIVCLvLPjnuChQiPRefKOp8IH35ZjCBLRB5-CWFnNKMc2Z3Z_jQzeyxzikPI9PI5u9fErheiCWVb0HnR5k8FSg_guhCQaOBeYKgbA/Radio%20Control%20-%20Unravel.mp3?extra=null
https://slider.kz/download/474499153_456452837/238/cs3-6v4/s/v1/acmp/QOuii-Czus3bM2CyRJILUYMWsCNb-6TEgDNKyBO_qJYdxeoLM5YEqyW03di_3mSuFpbTNqCD8cEN8ijmtjMRESRZCKya9Wj89Hyo8A6s7oQFURgmc0OAQw1ifU6FNPgNuLt0cM8UuVu5ONMZcXLxQB_X-vyVga9enPIDvzadwIPzMLrkKA/Kevin%20Remisch%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/474499122_456591314/97/cs3-7v4/s/v1/acmp/lOsM3wtU0cPI-POYId2zCQKMuFocZYxke-zM9kGXjLh_HWnKmpt9FT_BoUqdG_QMSQyXfnUdjzt11bJW4Z6hkc8izzQ_xo7UA70AhMVu2Ik-d5INvNV2W7xAbavVROqzFEPxMEmmYpazITlYQEy07UapD3KiFWUY1WRDbr_-1v30o018GA/Lil%20Bubb1egum%20-%20unravel%20but%20is%20it%20okay%20if%20it's%20lofi%3F.mp3?extra=null
https://slider.kz/download/-2001077985_82077985/238/psv4/s/v1/amp2/VllxAjiDhUD2kGGN_1QYYlEi6ljH-VawyGBR5t7gRwmBd4bnyoy7Q9TP42q83gu6qs0Q3xjHT7HEgi-HkPQAs3w417-ESpSQGMQEySHxg8O43I_0gt0dJ6edsjnIsWzRIrOtSJwBSeqtoUKitAfdgVRjP5-6egt8gLqr/%E6%AD%8C%E3%81%A3%E3%81%A1%E3%82%83%E7%8E%8B%20-%20unravel%20(%E5%8E%9F%E6%9B%B2%E6%AD%8C%E6%89%8B%3ATK%20from%20%E5%87%9B%E3%81%A8%E3%81%97%E3%81%A6%E6%99%82%E9%9B%A8)%5B%E3%82%AC%E3%82%A4%E3%83%89%E7%84%A1%E3%81%97%E3%82%AB%E3%83%A9%E3%82%AA%E3%82%B1%5D.mp3?extra=null
https://slider.kz/download/474499138_456433311/238/cs3-6v4/s/v1/acmp/VNdbMoXjNPrw1jhq0upwNnZfbolEFriZNKeUEHSf1ncOJUmACaSbyYEZiu1N8JghIVspFPzHWO3STGmE0lGuFvgo0bF6Y74ykyvwwSiDgcRqMbUBbn2ySHSA5RSLN8nCinRGvDvJ-5YSbYmvgZbwaNVWVFVJ0R7D4gwA5RPvt7BQj7eaeQ/PelleK%20-%20Unravel%20(Tokyo%20Ghoul).mp3?extra=null
https://slider.kz/download/-2001778472_74778472/238/cs9-2v4/s/v1/acmp/z1oe_35fnQwSwR-yAjBnQ4MlCd2PNIEnRSLz_ZCKF_5PldV40_lerssvNhZazYLx7YpdpVjqm_HNA8PIuAqrlvKioKNyrOu3GL048jLLCz_xJL38tDuU22qmn8mhf5YJ7_XzQygG4tmjHFh88hASl-wYpTowxw4kmOAcS0perZf4inwKbw/senakisaragi%20-%20Unravel.mp3?extra=null
https://slider.kz/download/474499220_456599167/246/psv4/s/v1/amp2/ZyCoku2jGfwD1BaUlyq7-q6TDH-IwTCPHDVoyJSz9TRj4hLvzyeFs3j5um5duLh8mru1sc22DqkxA7Ajara2-Y9GKl7PomX_F88WidLCB1MARvkw2iYrizN-q4Et4QsAWJm2VaSamp1vRDksmw5SUgN4a74Qh58sRx2Y/Matteo%20Leonetti%20-%20Unravel%20(Japanese%20Version)%20%5BTokyo%20Ghoul%5D.mp3?extra=null
https://slider.kz/download/-2001804556_66804556/238/cs3-5v4/s/v1/acmp/gIyB4Vft-_97FyJAE9_0hQlav32398RrusRMOaXgEkSsoBNW93224d76-9tGbFvAT07TSJEwetYGewbka3fjyJfSui8-paQTAJMVI5lMtwgMuTU9e6fB_cRtlIsXuN6vZhQORVl47To-JGxYgfN1-zRNimSOEBj4Ve_p121ne1UwhTnDLQ/V0RA%20-%20Unravel%20(Tokyo%20Ghoul%20OP).mp3?extra=null
https://slider.kz/download/474499291_456568508/102/cs9-6v4/s/v1/acmp/fKvZmSOyvLXOxRXgOatLErO-EzVCD2cn-tB1Dys0NFpugNtd7A4hZlhBPUZMXAKZIevqS49l2ZAscFh-k3RRHX02kgrbPM6H6AhcAZ_jbvNstSTtxywJvz68KeUWJ5y0Bp8ll3fL7g-rNSYGQEzIxoTjQ6LXqfB1eolI4ZUsqC8vUfoRcg/Piano%20Cat%2C%20Anime%20Cat%20-%20Unravel%20Tokyo%20Ghoul%20(Fall).mp3?extra=null
https://slider.kz/download/-2001227120_98227120/186/psv4/s/v1/amp2/cLi6IzWrhQhwJVpx-nbHAVgRnIxbVTGQQZfcqBf1JyjYIYi55BhiY6bc_fFMdWxKV5lMiYXFcAbUTzrketBr5NXGs3a4B682hU58z4nXP-E3AET0RRFi2-zBVQkK8zeHEOfBGk4-RH8HbJvY4f_oJHfeSSfhVPyxOKdg/Olga%20Scheps%20-%20Start%20Anew%20(from%20%22Unravel%20Two%22).mp3?extra=null
https://slider.kz/download/474499232_456549884/90/psv4/s/v1/amp2/G_D-kC0LEQYYW7ekPdSMg06ozpZssPpm4_4ke1P8RhdHN2a8EH0t27UjRhyBw98-veAur6NiLRcicS9vp4nzOPQIsqaPr_-v-x9h8m2L3yIzpDcVKTftL2Th9rB_H4g4tHTUl8orsWQU6npEA7sTWysshIwpFs_nqWQq/Mary%20Antoine%20-%20Unravel.mp3?extra=null
https://slider.kz/download/-2001667855_68667855/186/psv4/s/v1/amp2/xznk6i9L8LRfJar58xq3PjlC8dSayqHAQnXP8JIbQTSUFF6dtmf4AG1_Sr7dR-6gYxW05dzwebqzJJU-KkNJvqu6ieBiK_trxotPhLkBA3mmBcskTm8z1q6VnpE3YPOYyiJIOFl_NLvxbdT7GqlNW1UYnKScRVumta8m/Track%20-%20Unravel.mp3?extra=null
https://slider.kz/download/474499146_456450407/186/psv4/s/v1/amp2/wEP-qGrwzxqR21c80CCBheBmiu2fIZVI4VsGd7CuyGxTkVI1boxILGBdheTs6tX0N6XCuLO1j4bAs5B8XGAmAEdD9ijXYGTW5cOYDQ7psn31PqmHWKubTdiefBdP9ztaYS6szPgmhtQ9FN9OoKKOH3-WWs3qW_IvYAFM/Track%20-%20Unravel.mp3?extra=null
https://slider.kz/download/474499318_456592131/96/psv4/s/v1/amp2/1Ngs7lG1WBh6TJ1IhzauBhSNBeazCty_k25JatXLvLqpadvgBgoQ0lY9ISeBhfts6PV2hs5QtYk9EZFXczQq7GVshKDrg8ovPv0OnPTolvNdOhhJjLiKPoF2dUewl09LW4yvJmX5ogCX7q_Pq-F3bWeLpa20NankBa_c/Vangakuz%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/474499173_456432829/137/cs3-1v4/s/v1/acmp/rT3ABEaA8kmBYfjWSUgBN1i_wGr6c7BO5YQXfOdt474aADbZ0Tjg4OWB08dgPLHIcx6o3-rRRELc59NKPIhWhfVQq8jatvyFzxR6k6bo-wp5dJFVptg21nj7NTtWPOMcgvjXxblmVfZrXtZKGKCfFoqb-sO2sDKOM3prAMESLmk_HQieCg/Kenzie%20Smith%20Piano%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/-2001400485_76400485/187/psv4/s/v1/amp2/wQrrOm95FEb1EpWPN05KTxkNgjEfDh_faLEMRAA1Qk4F4cCJSO_9sd8XjDlc714dR_OK9JyY_9AQCe6moPsLeNnM6iCbXzFZrdWObICKCxggwhxuCtyaM4liHWk5TgNcFC1GLQp8Lu7Hh8v2yiC062n2nfsvZHjenrho/Simpsonill%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/-2001521828_57521828/551/cs3-6v4/s/v1/acmp/dZXnCT1fUfJ6PdA2mzGe8yHpA9s-DY_9UYw3SUYt3z_QoxPkXt6m4epU3ilm_7hieMGGJIe-HQeq6UYmj1QZYvJPezdSZyd6wBiy288Zdv2zwMEqQsWrKziLnBIDpenTzJfh8Lht97PQ4hqkQ7mOGO8ucyZm1evH5kQFymrzeFQcstVf5A/Alveol%20-%20Unravel.mp3?extra=null
https://slider.kz/download/474499237_456436594/88/psv4/s/v1/amp2/8Bo41ZWBvHso221zqJhGjyABsxjNh8o27U_XLhVWL93AEuMhcNxyhU2Ur43TuL48o6nFZQDF5HpQj43ra3EM0ddi_ptqUCJRgVr1vF5ADTqFXYlzqdIO_Jg64Vku2kEfVPwd5KYOHoJD_VCsRTFN83sW5epkBJm1c798/MattxAJ%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/-2001915683_101915683/90/psv4/s/v1/amp2/QaH-avVk5DSEc-4Xw50z0JX3TZLUGZ0nk4PblwUkmscM8QYZh2u6t8rFGB7D_GzbyTwxltM4Rp0kE1x0WfCO_QHO5IHZe047S93z7D2a_-DEnr8v_bwDtLv_10NQRabdX77HE8h9iWlSs_doa76LQMm3Gm6fYtCutX29/Music%20Legends%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/474499272_456394915/236/cs3-4v4/s/v1/acmp/WDQwU05V9M89alMuCTi786gxWCQDoeFFwYlNzojOyQ5G5oeCmgknrMPfQGlX1dE1nn-JHFfaJOOhgXQJWptsR070hN8OufTiDgs4dzg43TzgDhi_Cn6mXxnfAq3qK5xDWSY4Kamlydi4hVH6xcC5uRrpy70lK5xEce7eq01yornEDvVkcA/Shiroku%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/-2001529426_101529426/255/cs3-2v4/s/v1/acmp/yz7am2kIPxn76UUaQEQFAeOfeTQciBwOu8g151LzAMMKMQ5NRg6nI92UZBjPePHnH9rV7svs2udN-fiGIpg2s8QqIbP9lWCyG-J1qUB9CywlXwu8tfg6CzSTN2GgbRp8iQM-QKlGrRgLxi56Up3r0U7oWubLYL0hEOKr3ysemG_9tB_Q8w/HalcyonMusic%20-%20Unravel%20(from%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/474499176_456433059/241/psv4/s/v1/amp2/EZVr_Le7Wnc33wId8JajbbOo03wZBaoT8f8_ZyHVTiL5h8hs8HtGSVdm3BLNaW_AH5znzjmROfyAYYPoYnLQ-pY789RfVtTwX034IVdl_tM097Dube-bzIGaa_7COKBlo3Dp_1lHWfn1EjrgHEI3YhYPhnVyxOjkRsrd/I%20am%20Justice%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/-2001621142_58621142/68/psv4/s/v1/amp2/Oc2oMY7m2oW3hfpZDQIW24g3lWZIUDD1idJn8Wn5ZN5LJy29qZTBNwcHYCcxvtLPz5YsK4hWSvi71bW03O1eKw4PhvRXcqWreiPMJI9BJsHU6vwjXk1OieqMk3zC-stJLo5ZRIMxUpuefWK8HQlv7HfCAqrba3PI1_yG/Gianni%20Tallone%20Della%20Cavallotta%20-%20Unravel.mp3?extra=null
https://slider.kz/download/-2001052300_100052300/89/cs9-11v4/s/v1/acmp/jENDq0_xjfWlA1Fau3Cd16w9dQT3Won_n8YIMUI3daFMPE4P3PEuYPxpnn37TYSDXtZsXjyDSPLFO2HbnzGi3yDkvfePGfIrY2lK4y19tTEY9Y6aRUZIbAN5eS-IWG91t19G8soWuPs8B09mV82_4lj6lW3900E11hWClLA-Xl7jCoN3bQ/Fonzi%20M%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz/download/474499157_456624310/212/psv4/s/v1/amp2/WpHC4MCzE7o7pBv5lzq2K9K3pf4fZPUgW_GycJUP-92L5fd1yNx9fOLzJol5M9F5Mkwv8BApcjZkjLw4nC75G1rX7bnR-oZNOBP1pSw0EgJZi1o_a2soIKK3YLTMmcHdcdxXGNTJ3IG0YasN9w9n_8pzlc8mYoQYkOf1/Love%2C%20Lies%20and%20Fiction%20-%20Unravel.mp3?extra=null
https://slider.kz/download/371745435_456449924/271/cs3-2v4/s/v1/acmp/2vGBvFG1ejsCoSv88vOr1JwaXPjPCw6aFRXBB_8fqa1j_LxgWXH7bzy_9JbcymQd5ACAr1bYoLnwBwfthYgqYHMWaL2DGTCu5Lp67BHZXWHRyGFeD3iKM5OeKLh4642GtJSGxVkgApp_9cRCvvIj-sRUAI1a0jgd_zjfAafM8eE8pf8WTA/The%20Blue%20Notes%20-%20Unravel%20from%20%22tokyo%20Ghoul%22.mp3?extra=null
https://slider.kz/download/-2001260329_90260329/94/cs9-15v4/s/v1/acmp/9QTEytbysHCNmBriy3cvC2OOMqLUBoFMaUgslyo1b_QU4KGSKxWTJ-1x9bmqlxOIaCKJKTrsStWyE01pTlyCUU-tImpXxgFKyqcdpwQOvNAtLT_jiCrxmVpKGGryMeOgDp7xqcsbwWrDTqUYh685cY_Gc4avJ7va_QDg935M6qq_TxcxIg/Piano%20Cat%2C%20Anime%20Cat%20-%20Unravel%20Tokyo%20Ghoul%20(Fall).mp3?extra=null
https://slider.kz/#Contaminated
https://slider.kz/#Excarnate
https://slider.kz/#Extremity
https://slider.kz/#Serpent%20Of%20Gnosis
https://slider.kz/#Petrification
https://slider.kz/#Skullsmasher
https://slider.kz/#Hissing
https://slider.kz/#Mortuous
https://slider.kz/#Skelethal
https://slider.kz/#stabbed
https://slider.kz/#Winds%20of%20Leng
https://slider.kz/#Usurpress
https://slider.kz/#Encoffinized
https://slider.kz/#Glacial%20Tomb
https://slider.kz/#Overlord
https://slider.kz/#RESIN%20TOMB
https://slider.kz/#Descent
https://slider.kz/#Hurricane%20Death
https://slider.kz/#Gutter%20Instinct
https://slider.kz/#Crypt%20Crawler
https://slider.kz/#This%20Game
https://slider.kz/#LIFETAKER
https://slider.kz/#Autokrator
https://slider.kz/#Vanhelgd
https://slider.kz/#ATARAXY
How do i fix this.I tried randomizing my user-agent but the same result.The website doesnt have any log/sign in.
Chrome version 98
Python 3.8.8

This page uses JavaScript to download data and generate links. But requests/BeautifulSoup can't run JavaScript.
You may try to find url which use JavaScript to get data and later use it with requests. Usually JavaScript sends JSON data which doesn't need BeautifulSoup.
Using DevTools in Firefox/Chrome (tab: Network, filter: XHR) you can see URL which it use to get data and try to use with requests.
It gets JSON data from http://slider.kz/vk_auth.php?q=unravel but it gets numbers, names, etc, and you would have to compare with real url to see how to use them to create real url.
At this moment server sends only messasge error so maybe too many users try to get data
import requests
response = requests.get('http://slider.kz/vk_auth.php?q=unravel')
print('text:', response.text[:100])
data = response.json()
for item in data['audios']:
print(item)
It gets also other JSON data - similar artists - from url https://slider.kz/similar/artist/unravel - and it sends data without problem so I show how code can looks like after getting data.
response = requests.get('https://slider.kz/similar/artist/unravel')
print('text:', response.text[:1000])
data = response.json()
print('key1:', data.keys())
print('key2:', data['similarartists'].keys())
print('key2:', data['similarartists']['artist'][0].keys())
for item in data['similarartists']['artist']:
print('name:', item['name'])
print('url:', item['url'])
print('image:', item['image'][0]['#text'])
print('---')
Result:
key1: dict_keys(['similarartists'])
key2: dict_keys(['artist', '#attr'])
key2: dict_keys(['name', 'mbid', 'match', 'url', 'image', 'streamable'])
name: Contaminated
url: https://www.last.fm/music/Contaminated
image: https://lastfm.freetls.fastly.net/i/u/34s/2a96cbd8b46e442fc41c2b86b821562f.png
---
name: Excarnate
url: https://www.last.fm/music/Excarnate
image: https://lastfm.freetls.fastly.net/i/u/34s/2a96cbd8b46e442fc41c2b86b821562f.png
---
name: Extremity
url: https://www.last.fm/music/Extremity
image: https://lastfm.freetls.fastly.net/i/u/34s/2a96cbd8b46e442fc41c2b86b821562f.png
---
name: Serpent Of Gnosis
url: https://www.last.fm/music/Serpent+Of+Gnosis
image: https://lastfm.freetls.fastly.net/i/u/34s/2a96cbd8b46e442fc41c2b86b821562f.png
---

If you use the browser inspector (dev tools - network tab) you can see that, every time you hit "Search" it makes a get request to this kind of url: https://slider.kz/vk_auth.php?q=unravel
The response is a JSON with the results.
So, if you want to use requests you should get this url and then extract the information from the response JSON.

Now it's working.Please just run the code.
from bs4 import BeautifulSoup
import time
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
url = 'https://slider.kz/#unravel'
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get(url)
time.sleep(4)
soup = BeautifulSoup(driver.page_source, 'html.parser')
#driver.close()
for link in soup.select('div.sm2-360ui + a'):
url=link.get('href')
abs_url='https://slider.kz'+ url
print(abs_url)
Output:
https://slider.kz//download/371745468_456554892/238/cs3-3v4/s/v1/acmp/QRhl3TT5g1GEHPv-bkQJXT4UPbfj2OAAOI01kg3IFGV3PQUawPjulWC4U1oquAFZgADZF4QaOxk8CrEfBzLih4EpJf2a8qTr3WYT0yJQiCNDV78tnhuZAGmi1p2I4wwdw_743Rfe6Y9lvMXcVHwnKd7NdRyUlKLz5XxjVAUTGxP17pY_qA/TK%20from%20Ling%20tosite%20sigure%20-%20Unravel.mp3?extra=null
https://slider.kz//download/474499200_456477539/242/cs3-2v4/s/v1/acmp/AsZGD44ViRPS8iviU4fbOjb7I095Guqb8LMt4EBRZEwgTYR2BFlFAI4HtLEu_JTeohwHwakRgcJ9pnUurkL98oL6co9QTzepES6zqJ_tDYp4oUN5D3E5pm-m2LZGkqeCNcLsZmA6PeuxNSkpEvbfnq7W-in6XtIpASiP9qQ1UdSb1wXIgA/Anata%20No%20Senin%20-%20Unravel.mp3?extra=null
https://slider.kz//download/-2001620886_66620886/237/cs3-1v4/s/v1/acmp/gOSXhnJZc_7qVqSRt7soH679KeuB1qEa_Odx7P2YY9BwEAbmB9L2N-j8Qcm2roEWAxA-45rEGOQ1KT3r6CxnNYFZKC6t0Gg0J-wIjnC0sfDNhJrQpogT_kHrV7rhUSyFPLq6mrP3A7Tj1Dgl_KzZdjB-REw-21XuVyxlrLIgQRJiApQ1vQ/FalKKonE%20-%20unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/474499141_456425057/270/cs3-1v4/s/v1/acmp/9l2EX9ERaANRDGCKW8RS4-_VZR2MZYDlMRLy-cTAlKW4SP7QsodWeBObK7VaJa3vQiD7DOC6YEdOxTZHUUUksY6Xj7l1OgMyaJ1uzDLiVxtTy7NfJ98tdknxtnDhvNk3gXnoTZI5Fhs18fSDusmQkFAcDYkC8A7Czpng3gLvsvtoHQeQjQ/Dima%20Lancaster%20-%20unravel%20(Tokyo%20Ghoul%20OP).mp3?extra=null
https://slider.kz//download/474499133_456468907/240/psv4/s/v1/amp/84PrmRmfu7EBaFB3FMcMuLy9CrIgecvqzu74ZHTGdBVyhe1rQRRhSBfVA56cHNnwSPP4GNXVvQ88IZJq0exhcN8Fvn5k9rIdI-PJWu8QviAWxX4aYZqon1Sflf1Cbg/Yung%20Anime%20-%20Unravel.mp3?extra=null
https://slider.kz//download/-2001476389_94476389/229/cs3-6v4/s/v1/acmp/YRjjyO0TiBxN0blkh9T8e-6g6SepkCtdIMtBqNLYvHbfqOCYLBqpvwS6RmWXLeYNUKMy_FutPbQ9LLRPSHFO-TYBQwAM4AoT6Z5_vvSepusrUJu33dDVYMAIWntUc8ABhjmnLRfVPeBbHYzGnx22n2Je_U15qYFD1we3aWiaw2qV9CxbTg/Nika%20Lenina%20-%20Unravel.mp3?extra=null
https://slider.kz//download/-2001355458_97355458/244/cs3-1v4/s/v1/acmp/f-WX05d925zpwPtfj0RS6oUftizqO337PogbUWwnM1M8hsXNV26bPGdtEgze2yNpsEQygL5qeip-xc5s7LBFOtY5X_4n0fQo-oWz9vkdRvD3sUl9XEPgibbUob0VFJ4itJXtm1l1D1LRHTyDcNeA3xDj2S2n1s_7Bi59L8RpZgIGWmvtcg/TK%20from%20Ling%20tosite%20sigure%20-%20unravel.mp3?extra=null
https://slider.kz//download/474499285_456610507/236/cs3-5v4/s/v1/acmp/QMrplAB509ufV1tZgQn756q7IsLtpEmITRyaCLzQBrMEFmK-Zc-IkEWdOI_eE5D4-Nz1i2Y97wu7cmlF69HxYWPqjcp-2pWLuUzg4h1YeOUtTGh-nteiMEdfnpJYfYkyP6ZwayqLFLOh7OH0Ki9n3lNYRm1A_U47zVi-lsRS2IGOCZQAOw/Yung%20Anime%20-%20Unravel.mp3?extra=null
https://slider.kz//download/474499192_456485073/91/cs3-5v4/s/v1/acmp/ukuxjLP1ByDVop0pyoGvFwd4eQTFJnLH_6Dod2uT9uR9VCLHcTk9zoKqes1bAAb7bM8wDj1VTxi_ooy8I_AcNjW91hjWhb2uKMBYow8RR2WciYlyJ2ndYfqcBHwTOH_bj4fd1nPng0AFGG7phKNleCZW0tg3tlvzMjTHPKfsOI3w-Rypug/Yendorami%20-%20Unravel%20(Opening%201)%20%5BFrom%20%22Tokyo%20Ghoul%22%5D.mp3?extra=null
https://slider.kz//download/-2001941575_98941575/185/cs3-3v4/s/v1/acmp/kLoq7G6WCnnVNV4gIOaTAA0sZWVTO34BChRHQ-SLhB47bp0Fu3ogijAaADtrwsqaZlKDLkY6RwLlAf8f-hgIGM9TCJ11mwzLsnIOc9YJRkDeUYH74GT2WF0tIO7nlPGg9iv2dVFDld089JM6-XcbT849iJCegdNGQ3qSkcaFjrQ1NaJFbA/Collosia%20-%20Unravel%20(Tokyo%20Ghoul).mp3?extra=null
https://slider.kz//download/474499243_456360717/225/cs3-4v4/s/v1/acmp/N0jZrOAfPu-5bJlwP0HzVU_8yG96Xddn12wE1QMmBOH_3aza1-ig-cHZHrY3dx76Ahm8ppqKO-uYXOCNAYUEK_mPy2Uy6npKbUlnRKn-YHIze3gP0IRvLHaaVMCACfEx3W-srnsSQTiN/Trinthepianist%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22)%20%5BPiano%20Version%5D.mp3?extra=null
https://slider.kz//download/-2001187532_103187532/144/cs3-5v4/s/v1/acmp/TYPoa8_kCxJuIGNUL1Wl_5vYYnxcl0CrUNYlv6FzMCLCgRyI5TrNjhX7xCuuRFBsSxGeTDX8rqI8acAvhNSGX0u__Hzr84NWTzMp2rtgee6NyN793lVcVF7r499W_34RTTC6exIIGcYEIhwUxDMC-BBwIQr6pYnBctsAbVJYd4ZcD-2JKg/KinovariSquad%20-%20Unravel.mp3?extra=null
https://slider.kz//download/474499138_456497179/330/cs3-1v4/s/v1/acmp/_WbB2KC7ZDs22xTYAFfwRyk0p3GfbW8L-q9exF4raO5H_YnoX1rdRmAGXSW9xQCk_7sxBLfA-WO0UT0ilQVPRVwo9GR6upY8VLlMWyRHtntEaNOnUakzaiUBcTIE5mGRYs3ti5Ek1JGmhtioUBPEr94GVtrOMeKGRPe4s56oYHWlO3xXrg/SLSMusic%20-%20unravel%20-%20Slow%20Version%20(From%3A%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/-2001343575_62343575/235/cs3-4v4/s/v1/acmp/6pYoGLam7Gpf2Urd4PDhHInGLa0NZXgC4MiBNBCE_PG-YBTqjPZiIJNdgJty3HEEA42N4htrAN0TyudLT_l0rASk3yvwH8HSSl1GEA7xTxy8ZuQ1BfLrpHwIZ6g9MY2saaMmwaz51K32mUtBQHKXltyWuCvWzwYteRs3ZwuOhyEES2AJFQ/Geek%20Music%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/474499226_456421440/235/cs3-4v4/s/v1/acmp/ZHspQ1RQdnss3kdW6LREudZe1CVOFo2wItlxVeppU8SDGGPmfoVUH1p4451LnaaNG61RtRDrZD46eBfoTGwdmsiXmZ12Fzyhg_A5UMGplsTPHxOcL9Pw8CJ-VTQlIEwNiLJKwoWXxKIZ0e4h2A7rq3hRs49qnAzIc3lRE93yzrLCymc0ng/Geek%20Music%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/474499206_456447563/243/cs3-6v4/s/v1/acmp/26YcSjZs2EOKlDmgk6YspYb5imv6RXR1NsTkMBi6jgJMvdf5nNX3vJLWO-z9YAxl97b8JTxF3HK_u5wT7TcQfiIcFHOoLvXM7jg_Y1YMW1Vlc-sj-kqadoXFbGw_x9aI0NzZIT-VpYncsYC6N0N3cQsi_UjPPgHT5iiuIvFW6JHFBm5ROg/%E6%AD%8C%E3%81%A3%E3%81%A1%E3%82%83%E7%8E%8B%20-%20unravel(%E3%82%AA%E3%83%AA%E3%82%B8%E3%83%8A%E3%83%AB%E3%82%A2%E3%83%BC%E3%83%86%E3%82%A3%E3%82%B9%E3%83%88%3ATK%20from%20%E5%87%9B%E3%81%A8%E3%81%97%E3%81%A6%E6%99%82%E9%9B%A8)%5B%E3%82%AC%E3%82%A4%E3%83%89%E3%83%A1%E3%83%AD%E3%83%87%E3%82%A3%E7%84%A1%E3%81%97%E3%82%AB%E3%83%A9%E3%82%AA%E3%82%B1%5D.mp3?extra=null
https://slider.kz//download/474499222_456568196/98/psv4/s/v1/amp2/5MdAXyMQXscyhnNEfxrNMz4rZAhA3Ckw9tV0tLA94BDtvVCeDVz_Ycc62dFPtyJvjDaucoWBQZAguZHmkHV81InGjY22Am3TtoTYWH4rGOoiaM8coOi8DpIUQd21gda6fFcTZMkuLf0t33P5it1dH86nAmDGduot9AYu/Piano%20Cat%2C%20Anime%20Cat%20-%20Unravel%20Tokyo%20Ghoul%20(Winter).mp3?extra=null
https://slider.kz//download/474499266_456496174/238/cs3-2v4/s/v1/acmp/GkaHnOLSSRydiMt--oHvxPF2ZrdnAK-NO5Lei7J48RoxS_io3XwVnMFTQJN54MLZ5oagc9cSb4COMi_yxoCuAEm4FMnA3Kqetj5cj5D_alBQkHJEtNHS4IL1-kgE_QQ9RDsAfu5Qq2GrdyXj06rpvRr1EyKs1ITAkPez6uvlHAozli0mWQ/The%20Unknown%20Songbird%20-%20Unravel%20(Tokyo%20Ghoul).mp3?extra=null
https://slider.kz//download/-2001091207_65091207/243/cs3-4v4/s/v1/acmp/izDJAIBhtw7Gbo37AL4HCeFiLWypB9BgtdIqUnziUZ06PBBWt6OVqGY5BJIjaY4E3fG1szeoZcQeU7946ltUPsX86cPyq9jMfaPTbNS2rk4qaDYoJtK70qlGrA_OwvHUS3Ec_BQTFNq3-fqv2tJuyElVIaUesQOo4MMH7BpaTUI1T3n-BA/Bibi%20Gamino%20-%20Unravel%20(Opening%20de%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/-2001815387_86815387/189/cs3-5v4/s/v1/acmp/1q3ZcWs-DyWh-iYHwleI-jAD5A1WVgP8sN0zolnoNmBaj55OQqmHopLzZPd7AWmYP2Xs6E7Tv7BDtGn3TU1_-cDD13sbCywSWwQB2ckEsW84uZKn38qj0vcbYCN7YqDp7L4-drmoSdzIyHRE6xXrixB9cJuXgcKlTbSwkFh7-HRoU1lA9g/Baur%20Karbon%20-%20Unravel.mp3?extra=null
https://slider.kz//download/-2001343001_91343001/91/cs9-18v4/s/v1/acmp/pk4LPfaKgfokG8pHZAzxFPGhpemNTcIw-Ey_8BN1RPX_QNtv9m7H5HzsrKVY-b2p1_AKiSm_jJOdWV1iUXGsjtOF-mkjo4er3adDmZSFB6BqlCGQgXkK7F1Kn8XQDc5dc41KESB5p-JTSPKOCzitje8M8wcsdELenOJANNsvKtByjHRL6A/Ren%20Avel%20-%20Unravel%20(Tokyo%20Ghoul).mp3?extra=null
https://slider.kz//download/-2001459950_102459950/222/cs3-5v4/s/v1/acmp/dOeaiW4FJ3Q0jfOOTinST8c8Pn59ODRoRT5BgYrmn-DJw7m-u3x--lLPiUeECWRcHc7kKLGuAKoVTMUt32QfU0RDeGRCBGV2F8V8MPzNNczL3Qk6GQP9Bpj4ZL7SUUWlINmeRZyDChuD1APnUZtAzQ31DhAkQgpCH70MMHmtl6SNLNCVCQ/Jonathan%20Young%20-%20Unravel%20(Full%20Version).mp3?extra=null
https://slider.kz//download/474499249_456500220/95/cs9-5v4/s/v1/acmp/Cd6m3qhcIfjSesLIbaLc8IPOVc4E0H9ljKh_VIlj5LcIDifka-qMXBDquBCwd16bV2an-ZGX9X_NFFyV32q5DOg6AEKAMrSOdWQl9EC_mkvCLX1RLBF8HVarOoMZ9Y9oaV-gDNO9iTstvL0vBGQoxYz-CXIKok5FOfC5ptoG3dfSsRwkmA/Tony%20Gu%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22)%20%5BOrchestral%20Arrangement%5D.mp3?extra=null
https://slider.kz//download/-2001472007_90472007/95/cs9-2v4/s/v1/acmp/sqXJVOoZTIZvRB2CxjUHJZdvYHeaQkVffFqfpn4SW39S1-lImykJzWyENkb2tWUMO9eZNnOwNR0a6eZWPKvrBKdpLzLQE7JJFvV9kxKu8h2inNFyhSazlAljMtC3cn8R5Wk1B0uQdQY-17wb6-oBb7qoGSeiIBT-7C_Ob4G-JXcJ_6EUJw/Radio%20Control%20-%20Unravel.mp3?extra=null
https://slider.kz//download/474499153_456452837/238/cs3-6v4/s/v1/acmp/IQdnnyJQZtLM-cx27JQiYpqw3hrvc7Y4LGbUy5ruMBZNzC-7Bb4W6Wlw3M5CLykxFM2ALSU-aPR2xyMvggKW4gtqeXIhyfOq4yQbFNCYgG7mXXwwHp18FeyDTkXdYbdCbOZ05gQpO26e54LFLEky3FaAwvy48gQdHSUBAlCDNFN7gNUoMA/Kevin%20Remisch%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/474499122_456591314/97/cs3-7v4/s/v1/acmp/05S842jqQXODQOr-pq-LCi3z3CKmZR-6f9sqWNjGrXOaJDf-Ga0F42wMYDQmLTtChJaCKz-4kIdgGTXL-unYs4sJd3cg7ZwbywsPL2lMS5CiseljaXJ5I960mN1frRnZV-jVMiNqASMWfF9m5COgxgS_IAEvVDW5_tiXQxMxDfp_JSygOA/Lil%20Bubb1egum%20-%20unravel%20but%20is%20it%20okay%20if%20it's%20lofi%3F.mp3?extra=null
https://slider.kz//download/-2001077985_82077985/238/psv4/s/v1/amp2/TJu-2VmMhrc6VA7SF1OM2092kq003sX40joJxtIzXgAulLrdmWnzcVDMjn7L-dp5oXZVnqh-tXxyEbY-20YIvqOOWOuYlnldJHceTYDII9aJM-JhsA_Y9jkdCSOKZ0lwp_OhCWGFKEbfHu3NBJanFbms4EdBFUhNQH7K/%E6%AD%8C%E3%81%A3%E3%81%A1%E3%82%83%E7%8E%8B%20-%20unravel%20(%E5%8E%9F%E6%9B%B2%E6%AD%8C%E6%89%8B%3ATK%20from%20%E5%87%9B%E3%81%A8%E3%81%97%E3%81%A6%E6%99%82%E9%9B%A8)%5B%E3%82%AC%E3%82%A4%E3%83%89%E7%84%A1%E3%81%97%E3%82%AB%E3%83%A9%E3%82%AA%E3%82%B1%5D.mp3?extra=null
https://slider.kz//download/474499138_456433311/238/cs3-6v4/s/v1/acmp/CqmMRJm5B03c3so1ex1GshI705Z472EzHgAWm-M9eznXzjVF_6G-7lYlhkHdfPpHsWcVU6hWhLRqVr6WrYKBXcyvZ-ri0q5iV9Jv8Iwyd91t9cLu0sWVP1W3016tZKK0CSoXyfiL_oOfFhHbF9DbqmjmyUVO9ZB-McrQY6PlmOk4M-MBHg/PelleK%20-%20Unravel%20(Tokyo%20Ghoul).mp3?extra=null
https://slider.kz//download/-2001778472_74778472/238/cs9-2v4/s/v1/acmp/26oh4w3DDMsrZziue-jJNLoL_DKv7JMHYjXTixs0K4KjVjP0V6qTG2t5nmh6H9e8DVf3PH6Xoeqq1nzN718lXcKkSuutTFbWpgvA1nmV8M3DlQvm_WXyWRMReTYLwSwmwpALZ60-VbutNMFN3l8vPJ9F7lJCyi7V2ncQu9KxyWMCRD5C1Q/senakisaragi%20-%20Unravel.mp3?extra=null
https://slider.kz//download/474499220_456599167/246/psv4/s/v1/amp2/SX9vu47zGi9Uos8gQWm7XbfU1l0EKA1jqYblOj3ePK0pl5yPWLNFqITGYWgpzA1sRw6DzP_RAzgnu0C9-b2Ej5NDF4gU0px6dVSeOvO0DlKeW6XGcy2jFMJJCjHB_ag1VXEJgRb3jUYZLYRItNORbxOd45tRox8MGKBm/Matteo%20Leonetti%20-%20Unravel%20(Japanese%20Version)%20%5BTokyo%20Ghoul%5D.mp3?extra=null
https://slider.kz//download/-2001804556_66804556/238/cs3-5v4/s/v1/acmp/SGrWKb-wUFGqKV704k_bMGD7EJHh_A6sw2C9D8sWzWUA0R956jql_ZlVKEMv0MsJQ8gFsacMt_tywv3ZeXisu-rGTn4j_E8T3HS-o5RaqLzLOJauSlPoidthk8eIcx5xy4tGURb4KQkIfwIFJhriTy9HlnR7vCbaPpj_PfVNTBdlo7CU2g/V0RA%20-%20Unravel%20(Tokyo%20Ghoul%20OP).mp3?extra=null
https://slider.kz//download/474499291_456568508/102/cs9-6v4/s/v1/acmp/44hsI7dzI30x7Ho7N20vnYC06nWK2Sx-iXDPJXyN--Coele4vZUdekMfyNazmGRQ3IYMnLr1XBMPRWLVqZFkzHLk4fa_H5JN85IDDtft8EGPjsBRpSJOmbiAVAodQwQw3JcF6LW88ajmqKtJh3eUYARn9vrfQVN_J-ODVFY-dYm_akXUmg/Piano%20Cat%2C%20Anime%20Cat%20-%20Unravel%20Tokyo%20Ghoul%20(Fall).mp3?extra=null
https://slider.kz//download/-2001227120_98227120/186/psv4/s/v1/amp2/s57ppy-s9vm4n3qy-y4h8p4l0f5XyxK38-d91I3odmc9gvabMD_t42h8zd2oRL49tQkR1CD_vsPwIRA6DHsRUuW2WAkGJVwcef_VBmXUGsw7xEJSJG8kx-9AIXAUlj-8Ot32vMWg4TCiy/Olga%20Scheps%20-%20Start%20Anew%20(from%20%22Unravel%20Two%22).mp3?extra=null
https://slider.kz//download/474499232_456549884/90/psv4/s/v1/amp2/-eUtSpHonW9PPES9zCRzzNYbTSu7i_AEVujmMuifOKCuYme-q6vqFR5fXVHik2ARsaGX9_x4Za45VNNZohqsdkdCToDbne2W_6P7zgqnPJSuosGi8Z4ol0h0V_ViLH1ZIw_meOnJOkFDOTjKKyF-Ol0IvevzfvqPJtC0/Mary%20Antoine%20-%20Unravel.mp3?extra=null
https://slider.kz//download/-2001667855_68667855/186/psv4/s/v1/amp2/IwKjMHOHdlAXWq1z1B5qxNBGhN8JVmJy-pJEl_b3RrTdZrGTPB_U0KIkNnnq7NcP5e1wa5ms9jSB1LIhEdiZTN6c7PHTMyJaLxNM8TDA46JpfBIXy73o11mf_FyBbAjkOIevfweKBRPukNSdConEqZr2ZUQoVaFeT7ZK/Track%20-%20Unravel.mp3?extra=null
https://slider.kz//download/474499146_456450407/186/psv4/s/v1/amp2/NMuGmiwJOxvdnOKQLkH0gIvMLbj_I_b5UIivo2Q2qVIxnI6kkJ5ITkC9rtL3H-28k3P6SzP-SEiegRZp_CyuXtuxPOA-qcx8Jo7VJEu817oIipyKTLFF9TBnMa6_E7w4Fa9uWM7RhR_J0VBAdPCDE4hi58gKNRBJ3bl5/Track%20-%20Unravel.mp3?extra=null
https://slider.kz//download/474499318_456592131/96/psv4/s/v1/amp2/zVQkmtvUwnxkcZxoMYAgcJ-RBoo8WvzKV6PyKAif3fw1C7fr-rzaWQbimUXK7_xSaKfsZpokZ7WRNdLOzxWm8QP8rlG2NNHYNQv6b2OmOiqOpP-WqeQ3VRvHN-0FtzcWnjYsIJZKcLcBA_q89U2c9Otw2Bqcvbt4lxRq/Vangakuz%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/474499173_456432829/137/cs3-1v4/s/v1/acmp/_LgsM2-WTNOvaV85Hy6ObbzZSKnJNfppdaWt9A_NyTv3XYhfSpyJ3QXFyQDb8nISAoRbTgNEt4yXtvvbWDli7gClVqaNCdiuTpEIwDl0EUZqyUF_aJrueZAjvd1DdUJR0YhvL6Fmv5C-8KoGEKotgiL1Eq1O4hU1wFDxayNVb4TkOzwwEQ/Kenzie%20Smith%20Piano%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/-2001400485_76400485/187/psv4/s/v1/amp2/TFqk4JfH34feLtswrYZZXKfFUtIfXRTq8bOmXxLIHX99vXnkmPTp_A6-Rjj8fS9qiAQ6ao9F-VQvLZSH0bAnjBiUCM8pZbtBsL6z1v38vZzsc3kpP21MYG6kBCtfqg_jJhlj4CqDmfvQ0nW1NoUSNmutLvFy_7KTyjVG/Simpsonill%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/-2001521828_57521828/551/cs3-6v4/s/v1/acmp/kXLXCQe33bHi2WDA_-0y9UGNCgbK7Xhca4q6pig_COLFf5bNaN11Z-4Kzd3A9tanob-U9_Jni_Ix5beoPUaJurH1ozqI6XQiKuHPmkxkLKRSXZpd_tj6SQhkniNDwxBxyRANOsxVfbHe80fcXahF0OWTKc4R5mjB1L1XVLsOhILcdpyS1g/Alveol%20-%20Unravel.mp3?extra=null
https://slider.kz//download/474499237_456436594/88/psv4/s/v1/amp2/yha2bLmL0XLvXvjfVKdqa25-JSAeZQYu8QQKRZKNwVAINm3912ZetkeVDrvSt5MeOyyto6x-PvbwC7hiBAXKC7g8_PwbcXKcqmku8hiHr7xJKl9ca34WIzeqNLyayp9RcFxGjO_Gk_AuaQ0ZBgXTaChq2nE4OTl1bqbX/MattxAJ%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/-2001915683_101915683/90/psv4/s/v1/amp2/E79KiQTMA5b8mthMlpzYU5TRFcns8aqBNzqvKDFGJ14IR9Cz_C5yChPQUPQW-VFAm9_8sbo51D01NENhaegDJJIyui36C5ECVDwGaLhOcoLrhB8RKPLBGDLN01fcrb8mIxb7t0MhWr_YefPgj4Zt13xlufjor89n84eg/Music%20Legends%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/474499272_456394915/236/cs3-4v4/s/v1/acmp/2LiycWJnJntKnyikeZn_7TLKdSb8v343jFSIXZ9dbhcnsbSsjQZvlUI0jBkPWRGwGp5cK7UdprZarl3TUh_e_j_0vPrQs1SRHyogfbJlfQuM54Tr34iFNBasQLHCmZv1ktctdpEmuwNm4Yj2yi_1wLs9Iwqc9lBaZ-WmbDsfeeayQNwOLg/Shiroku%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/-2001529426_101529426/255/cs3-2v4/s/v1/acmp/em6rSRlSNNcJHpGpiuSV4-VjdhJoAsemcgDXdS5VPu3XYPF1y-45PleMXDNAZ8jNEkf-5V9h7AInOtxaLBiP8KcWDvKW5d6EVy_hhh_qqK4PgpSPLS5Kv6yhdh7gUy2vo0t-ZVt-mZcQ1-8k48Q58_JE2U4JRD2ZgV8yVVbXKAK92bDnqg/HalcyonMusic%20-%20Unravel%20(from%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/474499176_456433059/241/psv4/s/v1/amp2/ATyS1Qxr7uBxM2KN4G0dMynLla7gPAaTh9nkDH5fMGYBEycOzy3sfM8LmLIjM2JP5yO3Vml0jBuTpeLoY9BBFCcWaD4qZ81dHjXsWFddJdyXqK0lkV4aKOo8EbMdyCJwRzeCesqn3r4aa_Pvxp3f6oPzLQgubvIXUF-s/I%20am%20Justice%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/-2001621142_58621142/68/psv4/s/v1/amp2/CL5J0kD4Xm1iSLFGqD7zrODdTqubYTFwGCzC5r1n3Z_Tu6G_68RtjH3HaEY_D8qHuckrW0C0bCfUD5fXz0gxfoJD5PtEGwPpi_wQUdGi-D7dJITBjyMxx6dWz5duvvqZ7dmmW7ydKBGGaxHPI9_tofxRB_qqQmop_I11/Gianni%20Tallone%20Della%20Cavallotta%20-%20Unravel.mp3?extra=null
https://slider.kz//download/-2001052300_100052300/89/cs9-11v4/s/v1/acmp/IvCOhCCclMK3tCeswQwVtNhtyYHX5lgjJUZfQsCfskiHciXLjBpj9SzjZMop3u-kPRSuyeyFvJoJEiP-2z_owq5MPdx4uNgrbl5Mqm0xbC_JWlpl4VH8Sc_vdNn28SHpU6EJiWDKgN7kf_sRozvO5OCEq7YmcfQe2lC_awFHsJV0KU7rHg/Fonzi%20M%20-%20Unravel%20(From%20%22Tokyo%20Ghoul%22).mp3?extra=null
https://slider.kz//download/474499157_456624310/212/psv4/s/v1/amp2/-oLu8sAswnvZ-I4RRXNqDu59CLZGoMUsv6qNVAL9QTVO9TcP4SWmGmUJSgfiXAw-_MgjRO1Y1y90YBuAirIcJgFenZe0XxlsqWrtbr6CEyQypX3sfmidTTBcH91G2x-4dXiDLI6JUmSXR9vZ4KQ_Z4vl-9o8SbiPmdD8/Love%2C%20Lies%20and%20Fiction%20-%20Unravel.mp3?extra=null
https://slider.kz//download/371745435_456449924/271/cs3-2v4/s/v1/acmp/ased-v3q4z-i_oiBH_OoVrZntdBkQGYKVQre0wr9DyNB7KVvbXTnpHh6G0_EyuBf90j3bW9SbB6aUeTt8NgAKKiuezgzETKKXzUhtpacQuKQEo-upEFNHLq5VzSrFAz2IjSSA3lm-p9plN7U4DQOPnCzn1_klu8Mje4uY3TCcwoTAuPckg/The%20Blue%20Notes%20-%20Unravel%20from%20%22tokyo%20Ghoul%22.mp3?extra=null
https://slider.kz//download/-2001260329_90260329/94/cs9-15v4/s/v1/acmp/qI5QjSZzHW-fYGXdVihZndt972zUCvmUN2Qzjk9gfUgyTjh_XPeOh9ctff9Ob2ezMEbFLEL7oNkrnvgoh4EJy9JlUjJPhTQS2Fh3L1cLzqJELD-waEz9vDaZwS_IuPXr0nH2PO5img_a4xkkqRLIy-VDm64EVAdGelBPmwLf7rIj_qjF9w/Piano%20Cat%2C%20Anime%20Cat%20-%20Unravel%20Tokyo%20Ghoul%20(Fall).mp3?extra=null
Using requests module from api
import requests
import json
headers={
'X-Requested-With': 'XMLHttpRequest'}
api_url='https://slider.kz/similar/artist/unravel'
req=requests.get(api_url,headers=headers).json()
for url in req['similarartists']['artist']:
print(url['url'])
Output:
https://www.last.fm/music/Contaminated
https://www.last.fm/music/Excarnate
https://www.last.fm/music/Extremity
https://www.last.fm/music/Serpent+Of+Gnosis
https://www.last.fm/music/Petrification
https://www.last.fm/music/Skullsmasher
https://www.last.fm/music/Hissing
https://www.last.fm/music/Mortuous
https://www.last.fm/music/Skelethal
https://www.last.fm/music/stabbed
https://www.last.fm/music/Winds+of+Leng
https://www.last.fm/music/Usurpress
https://www.last.fm/music/Encoffinized
https://www.last.fm/music/Glacial+Tomb
https://www.last.fm/music/Overlord
https://www.last.fm/music/RESIN+TOMB
https://www.last.fm/music/Descent
https://www.last.fm/music/Hurricane+Death
https://www.last.fm/music/Gutter+Instinct
https://www.last.fm/music/Crypt+Crawler
https://www.last.fm/music/This+Game
https://www.last.fm/music/LIFETAKER
https://www.last.fm/music/Autokrator
https://www.last.fm/music/Vanhelgd
https://www.last.fm/music/ATARAXY

Webscraping Live data

I am currently trying to scrape live stock market data from the yahoo finance page.
I am using bs4. My current issue is that whenever I run my script, it does not update properly to reflect the current price of the stock.
If anybody has any advice on how to change that it would be appreciated.
import requests
from bs4 import BeautifulSoup
while True:
page = requests.get("https://nz.finance.yahoo.com/quote/NZDUSD=X?p=NZDUSD=X")
soup = BeautifulSoup(page.text, "html.parser")
price = soup.find("div", {"class": "My(6px) Pos(r) smartphone_Mt(6px)"}).find("span").text
print(price)

NOT POSSIBLE WITH BS4 ALONE
This website particularly uses JavaScript to update the page and urlib etc. just parses the html content of the page not Java Script or AJAX content.
PhantomJs or Selenium Web Browser provide a more mechanized browser that often can run the JavaScript codes enabling dynamic websites. Try Using this :)
Using Selenium It can be done as:
from selenium import webdriver #its the library
import time
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup as soup
#it Says that we are going to Use chrome browser
chrome_options = webdriver.ChromeOptions()
#hiding the Chrome Browser
chrome_options.add_argument("--headless")
#Initiating Chrome with all properties we need (in this case we use no specific properties
driver = webdriver.Chrome(chrome_options=chrome_options,executable_path='C:/Users/shary/Downloads/chromedriver.exe')
#URL We need to open
url = 'https://nz.finance.yahoo.com/quote/NZDUSD=X?p=NZDUSD=X'
#Starting Our Browser
driver = webdriver.Chrome()
#Accessing the url .. this will open the page just as you open in Chrome etc.
driver.get(url)
while 1:
#it will get you the html content repeatedly .. So you can get the changing price
html = driver.page_source
page_soup = soup(html,features="lxml")
price = page_soup.find("div", {"class": "D(ib) Mend(20px)"}).text
print(price)
time.sleep(5)
Note the Best Comments But Hope this you will understand it :) Else Watch a youtube tutorial to get proper idea what a Selenium Bot does
Hope This will Help. Its working perfect for me :) Accept This Answer if it helps you

Need help on retrieving site key from dynamic website

I am creating an automated account creator using Pycharm. I am facing a problem to which I have yet to find a good solution. I want to get the site-key in order to pass the captcha to a service I have bought. I have used the requests.get method but it gives back "None" as a result. I am using selenium in my program. After some thought I realized that using requests.get method, if it worked, would bring me a different key than the one that my selenium driver is currently displayed. I googled a lot and found only that there is a module named Selenium-Requests which doesn't have Edge imported. I am using Edge as it is the only browser everyone has and doesn't require the developer version of it like Chrome and Firefox.
Generally I haven't found a fix that can help me retrieve the key within my driver.
This is the retrieve code:
registerurl = requests.get(url)
registerurlstring = ''.join(str(e) for e in registerurl)
soup = BeautifulSoup(registerurlstring, features="html5lib")
hidden_tags = soup.find({"id":"recaptcha-token"})
sitekey = hidden_tags
try:
print('Sitekey = ', sitekey)
except:
print('Sitekey = Not Found')

I am not sure this is what you are after or not.To get the recaptcha value which is inside iframe so you have to target that src value of that iframe and using python request module you can get value of that input.
import requests
from bs4 import BeautifulSoup
url='https://www.google.com/recaptcha/api2/anchor?ar=1&k=6Lc3HAsUAAAAACsN7CgY9MMVxo2M09n_e4heJEiZ&co=aHR0cHM6Ly9zaWdudXAuZXVuZS5sZWFndWVvZmxlZ2VuZHMuY29tOjQ0Mw..&hl=en&v=A1Aard-wURuGsXRGA7JMOqVO&theme=dark&size=invisible&badge=bottomright&cb=ezyy1frci5ms'
registerurl = requests.get(url)
soup = BeautifulSoup(registerurl.text, features="html5lib")
hidden_tags = soup.find('input' ,attrs={"id":"recaptcha-token"})
print(hidden_tags['value'])
Output:
03AOLTBLQFd9hdHGmOesrT0xDcA8MkI6FGIiM3892Uws3aEWzPxUT8-U8IBEZHYzUEba2Jp9m3s9z_sz_fuij9OXZHABulFrI8YCD95kXV_H6xTO9vOubuZfzscleb6fdkkAE3IwUUSdTzPbXILy6SGLPI3LpPUptC1enZLIkQxQq9T8AEPPvCIsVgGe4jSE_l1jCWIRmBeBXsLgPLABZSq6ah6QWFfAngdC1rQaLMKWzLBmzh6ytEEGNYHmEG7P6UVtYcTI1IRIvq-ba-oGIUS1ELUb-1d3upQ29JWBtQ2t7_VNn237fguztf_FUDEHnAfHppUsrz-ZlkE00sMXFCuQ1XF6Qz7lH2j5g2z5KZQiODhRUBRRyd-ydjetz053bKRcgWpnNoZGNf1GBlW5inL9AtyYTkpruttw5sruAPuVgs5mrniQ5hrHNvfDIZKX905T2E21W2DsW1_07rItFYa-zkylMU83YXRQ
Hope this helps.
Updated code to get the iframe src value using webdriver.
import requests
from bs4 import BeautifulSoup
from selenium import webdriver
driver=webdriver.Chrome()
driver.get("https://signup.eune.leagueoflegends.com/en/signup/index")
url=driver.find_element_by_css_selector("iframe[role='presentation']").get_attribute('src')
registerurl = requests.get(url)
soup = BeautifulSoup(registerurl.text, features="html5lib")
hidden_tags = soup.find('input' ,attrs={"id":"recaptcha-token"})
print(hidden_tags['value'])

how to pull the shipping price from banggood.com using beautifulsoup

i'm trying to get the shipping price from this link:
https://www.banggood.com/Xiaomi-Mi-Air-Laptop-2019-13_3-inch-Intel-Core-i7-8550U-8GB-RAM-512GB-PCle-SSD-Win-10-NVIDIA-GeForce-MX250-Fingerprint-Sensor-Notebook-p-1535887.html?rmmds=search&cur_warehouse=CN
but it seems that the "strong" is empty.
i've tried few solutions but all of them gave me an empty "strong"
i'm using beautifulsoup in python 3.
for example this code led me to an empty "strong":
client = requests.get(url, headers={"User-Agent": "Mozilla/5.0"})
soup = BeautifulSoup(client.content, 'lxml')
for child in soup.find("span", class_="free_ship").children:
print(child)

The issue is the 'Free Shipping' is generated by JavaScript after the page loads, rather than being sent in the webpage.
It might obtain the Shipping price by performing a HTTP request after the page has loaded or it may be hidden within the page
You might be able to try to find the XHR Request to pulls the Shipping price using DevTools in Firefox or chrome using the 'networking' tab and using that to get the price.

Using the XHR, you can find that data:
import requests
from bs4 import BeautifulSoup
import json
url = 'https://m.banggood.com/ajax/product/dynamicPro/index.html'
payload = {
'c': 'api',
'sq': 'IY38TmCNgDhATYCmIDGxYisATHA7ANn2HwX2RNwEYrcAGAVgDNxawIQFhLpFhkOCuZFFxA'}
response = requests.get(url, params=payload).json()
data = response['result']
shipping = data['shipment']
for each in shipping.items():
print (each)
print (shipping['shipCost'])
Output:
print (shipping['shipCost'])
<b>Free Shipping</b>

Categories