I am trying to send a styled bootstrap HTML template to email with the help of smtplib and EmailMessage. I have used f string to format the file. But, I am facing ValueError: Cannot convert mixed to the alternative. I could not address the problem. It should have work but couldn't be able to think much. Where did I do wrong?.
In emailsender.py
import smtplib
import imghdr
from email.message import EmailMessage
#Credentials
sender_email = "sender127#gmail.com"
password = 'password231'
receiver_email = "reciever9882#gmail.com"
smtp = smtplib.SMTP_SSL('smtp.gmail.com',465)
smtp.login(sender_email, password)
message = EmailMessage() #Instantiate EmailMessage
with open('/email.png', 'rb') as file_obj:
fdata = file_obj.read()
ftype = imghdr.what(file_obj.name)
print('Type:'+ftype)
fname = file_obj.name
print('Name of file:'+fname)
message["Subject"] = "Email Test"
message["From"] = sender_email
message["To"] = receiver_email
message.set_content("This is a test version!")
#message.attach('django notes.pdf')
message.add_attachment(fdata, maintype ="image", subtype = ftype,
filename=fname)
message.add_alternative(f"""\n\n
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title></title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<link href="https://cdn.jsdelivr.net/npm/bootstrap#5.1.3/dist/css/bootstrap.min.css" rel="stylesheet">
<div class="container py-4">
<header class="pb-3 mb-4 border-bottom">
<a href="/" class="d-flex align-items-center text-dark text-decoration-none">
<svg xmlns="http://www.w3.org/2000/svg" width="40" height="32" class="me-2" viewBox="0 0 118 94"
role="img">
<title>Bootstrap</title>
<path fill-rule="evenodd" clip-rule="evenodd"
d="M24.509 0c-6.733 0-11.715 5.893-11.492 12.284.214 6.14-.064 14.092-2.066 20.577C8.943 39.365 5.547 43.485 0 44.014v5.972c5.547.529 8.943 4.649 10.951 11.153 2.002 6.485 2.28 14.437 2.066 20.577C12.794 88.106 17.776 94 24.51 94H93.5c6.733 0 11.714-5.893 11.491-12.284-.214-6.14.064-14.092 2.066-20.577 2.009-6.504 5.396-10.624 10.943-11.153v-5.972c-5.547-.529-8.934-4.649-10.943-11.153-2.002-6.484-2.28-14.437-2.066-20.577C105.214 5.894 100.233 0 93.5 0H24.508zM80 57.863C80 66.663 73.436 72 62.543 72H44a2 2 0 01-2-2V24a2 2 0 012-2h18.437c9.083 0 15.044 4.92 15.044 12.474 0 5.302-4.01 10.049-9.119 10.88v.277C75.317 46.394 80 51.21 80 57.863zM60.521 28.34H49.948v14.934h8.905c6.884 0 10.68-2.772 10.68-7.727 0-4.643-3.264-7.207-9.012-7.207zM49.948 49.2v16.458H60.91c7.167 0 10.964-2.876 10.964-8.281 0-5.406-3.903-8.178-11.425-8.178H49.948z"
fill="currentColor"></path>
</svg>
<span class="fs-4">Jumbotron example</span>
</a>
</header>
<div class="p-5 mb-4 bg-light rounded-3">
<div class="container-fluid py-5">
<h1 class="display-5 fw-bold">Custom jumbotron</h1>
<p class="col-md-8 fs-4">Using a series of utilities, you can create this jumbotron, just like the one
in previous versions of Bootstrap. Check out the examples below for how you can remix and restyle it
to your liking.</p>
<button class="btn btn-primary btn-lg" type="button">Example button</button>
</div>
</div>
<div class="row align-items-md-stretch">
<div class="col-md-6">
<div class="h-100 p-5 text-white bg-dark rounded-3">
<h2>Change the background</h2>
<p>Swap the background-color utility and add a `.text-*` color utility to mix up the jumbotron look.
Then, mix and match with additional component themes and more.</p>
<button class="btn btn-outline-light" type="button">Example button</button>
</div>
</div>
<div class="col-md-6">
<div class="h-100 p-5 bg-light border rounded-3">
<h2>Add borders</h2>
<p>Or, keep it light and add a border for some added definition to the boundaries of your content.
Be sure to look under the hood at the source HTML here as we've adjusted the alignment and
sizing of both column's content for equal-height.</p>
<button class="btn btn-outline-secondary" type="button">Example button</button>
</div>
</div>
</div>
<footer class="pt-3 mt-4 text-muted border-top">
© 2021
</footer>
</div>
<script src="https://cdn.jsdelivr.net/npm/#popperjs/core#2.10.2/dist/umd/popper.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap#5.1.3/dist/js/bootstrap.min.js"></script>
</body>
</html>
""" ,subtype="html")
smtp.send_message(message)
Tracebacks here->
click here to view the traceback
Related
I'm trying to build a website that generates a power hour style video from a list of music videos that the user selects by searching through the youtube api. I'm having trouble figuring out how to append selected videos to a list (addedVideos) in view.py from a button (add) in my html. I need to append each added video to a list so I can display them and loop through them in an embedded youtube player. This is my first django project so I don't have a good understanding of what the potential problems could be. index.html and views.py below:
views.py
import requests
from isodate import parse_duration
from django.conf import settings
from django.shortcuts import render, redirect
def index(request):
addedVideos = []
videos = []
if request.method == 'POST':
search_url = 'https://www.googleapis.com/youtube/v3/search'
video_url = 'https://www.googleapis.com/youtube/v3/videos'
search_params = {
'part' : 'snippet',
'q' : request.POST['search'],
'key' : settings.YOUTUBE_DATA_API_KEY,
'maxResults' : 3,
'type' : 'video'
}
#print(request.POST['submit'])
r = requests.get(search_url, params=search_params)
#print(r)
results = r.json()['items']
#print(results)
video_ids = []
for result in results:
video_ids.append(result['id']['videoId'])
if request.POST['submit'] == 'lucky':
return redirect(f'https://www.youtube.com/watch?v={ video_ids[0] }')
video_params = {
'key' : settings.YOUTUBE_DATA_API_KEY,
'part' : 'snippet,contentDetails',
'id' : ','.join(video_ids),
'maxResults' : 3
}
r = requests.get(video_url, params=video_params)
results = r.json()['items']
for result in results:
video_data = {
'title' : result['snippet']['title'],
'id' : result['id'],
'url' : f'https://www.youtube.com/watch?v={ result["id"] }',
'duration' : int(parse_duration(result['contentDetails']['duration']).total_seconds() // 60),
'thumbnail' : result['snippet']['thumbnails']['high']['url']
}
videos.append(video_data)
if request.POST['add'] == 'addValue':
print("add clicked")
context = {
'videos' : videos,
'addedVideos': addedVideos
}
return render(request, 'search/index.html', context)
index.html
{% load static %}
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<title>Search YouTube</title>
<!-- Bootstrap core CSS -->
<link href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-ggOyR0iXCbMQv3Xipma34MD+dH/1fQ784/j6cY/iJTQUOhcWr7x9JvoRxT2MZw1T" crossorigin="anonymous">
<style>
.bd-placeholder-img {
font-size: 1.125rem;
text-anchor: middle;
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;
user-select: none;
}
#media (min-width: 768px) {
.bd-placeholder-img-lg {
font-size: 3.5rem;
}
}
</style>
<!-- Custom styles for this template -->
<link href="{% static 'search/album.css' %}" rel="stylesheet">
</head>
<body>
<main role="main">
<section class="jumbotron text-center">
<div class="container">
<h1 class="jumbotron-heading">Build Your Power Hour</h1>
<p class="lead text-muted">Select music videos to add to your power hour</p>
<form method="POST">
<div class="input-group mb-3">
{% csrf_token %}
<input type="text" name="search" class="form-control" aria-label="Username">
</div>
<p>
<button type="submit" name="submit" value="search" class="btn btn-primary my-2">YouTube Search</button>
<button type="submit" name="submit" value="lucky" class="btn btn-secondary my-2">I'm Feeling Lucky</button>
</p>
</form>
</div>
</section>
<div class="album py-5 bg-light">
<div class="container">
<div class="row">
{% for video in videos %}
<div class="col-md-4">
<div class="card mb-4 shadow-sm">
<img class="bd-placeholder-img card-img-top" width="100%" height="225" src="{{ video.thumbnail }}" preserveAspectRatio="xMidYMid slice" focusable="false" role="img" aria-label="Placeholder: Thumbnail"></img>
<div class="card-body">
<p class="card-text">{{ video.title }}</p>
<div class="d-flex justify-content-between align-items-center">
<div class="btn-group">
<form method="POST">
{% csrf_token %}
<button type="submit" name="add" value="addValue" id='{{ video }}' class="btn btn-sm btn-outline-secondary">Add</button>
</form>
</div>
<small class="text-muted">{{ video.duration }} mins</small>
</div>
</div>
</div>
</div>
{% endfor %}
</div>
</div>
</div>
</main>
</body>
</html>
Running this as is gives a MultiValueDictKeyError at 'q' : request.POST['search'],
Any help would be appreciated, thanks.
You should put / in front of search/index.html instead of search/index.html.
And you don't have to have the https:// you can just put // instead.
I have tried several answers in Stack Overflow. When I print the webpage I can only see the equivalent of viewing the page source in Chrome, rather than the full DOM tree you would get from inspecting the web page. As you can see I have put a wait in but this hasn't changed anything, should I try Firefox instead of Chrome?
Is it possible the website I'm trying to use has anti-scraping measures? What else could I try?
def selenium_start(url):
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('chromedriver',chrome_options=options)
driver.get(url)
try:
driver = WebDriverWait(driver, 5).until\
(EC.presence_of_element_located((By.ID, "koex")))
except:
print('Sorry!')
return driver
webpage_driver = selenium_start('https://getbootstrap.com/docs/4.0/components/collapse/')
"""
div_container = webpage_driver.find_element(By.CLASS_NAME, 'maincontent')
html = webpage_driver.execute_script('return document.documentElement.outerHTML')
#inner_div = div_container.get_attribute('outerHTML')
"""
print(page_soup)
To extract the Page Source you have to induce WebDriverWait for the visibility_of_element_located() of an element within the webpage and you can use the following Locator Strategies:
Code Block:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("https://getbootstrap.com/docs/4.0/components/collapse/")
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "h1.bd-title")))
print(driver.page_source)
driver.quit()
Console Output:
<html lang="en"><head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="Toggle the visibility of content across your project with a few classes and our JavaScript plugins.">
<meta name="author" content="Mark Otto, Jacob Thornton, and Bootstrap contributors">
<meta name="generator" content="Jekyll v3.7.0">
<title>Collapse · Bootstrap</title>
<link rel="canonical" href="https://getbootstrap.com/docs/4.0/components/collapse/">
<!-- Bootstrap core CSS -->
<style class="anchorjs"></style><link href="/docs/4.0/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
<!-- Documentation extras -->
<link href="https://cdn.jsdelivr.net/npm/docsearch.js#2/dist/cdn/docsearch.min.css" rel="stylesheet">
<link href="/docs/4.0/assets/css/docs.min.css" rel="stylesheet">
<!-- Favicons -->
<link rel="apple-touch-icon" href="/docs/4.0/assets/img/favicons/apple-touch-icon.png" sizes="180x180">
<link rel="icon" href="/docs/4.0/assets/img/favicons/favicon-32x32.png" sizes="32x32" type="image/png">
<link rel="icon" href="/docs/4.0/assets/img/favicons/favicon-16x16.png" sizes="16x16" type="image/png">
<link rel="manifest" href="/docs/4.0/assets/img/favicons/manifest.json">
<link rel="mask-icon" href="/docs/4.0/assets/img/favicons/safari-pinned-tab.svg" color="#563d7c">
<link rel="icon" href="/docs/4.0/assets/img/favicons/favicon.ico">
<meta name="msapplication-config" content="/docs/4.0/assets/img/favicons/browserconfig.xml">
<meta name="theme-color" content="#563d7c">
<!-- Twitter -->
<meta name="twitter:card" content="summary">
<meta name="twitter:site" content="#getbootstrap">
<meta name="twitter:creator" content="#getbootstrap">
<meta name="twitter:title" content="Collapse">
<meta name="twitter:description" content="Toggle the visibility of content across your project with a few classes and our JavaScript plugins.">
<meta name="twitter:image" content="https://getbootstrap.com/docs/4.0/assets/brand/bootstrap-social-logo.png">
<!-- Facebook -->
<meta property="og:url" content="https://getbootstrap.com/docs/4.0/components/collapse/">
<meta property="og:title" content="Collapse">
<meta property="og:description" content="Toggle the visibility of content across your project with a few classes and our JavaScript plugins.">
<meta property="og:type" content="website">
<meta property="og:image" content="http://getbootstrap.com/docs/4.0/assets/brand/bootstrap-social.png">
<meta property="og:image:secure_url" content="https://getbootstrap.com/docs/4.0/assets/brand/bootstrap-social.png">
<meta property="og:image:type" content="image/png">
<meta property="og:image:width" content="1200">
<meta property="og:image:height" content="630">
<script async="" src="https://www.google-analytics.com/analytics.js"></script><script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-146052-10', 'getbootstrap.com');
ga('set', 'anonymizeIp', true);
ga('send', 'pageview');
</script>
<script id="_carbonads_projs" type="text/javascript" src="https://srv.carbonads.net/ads/CKYIKKJL.json?segment=placement:getbootstrapcom&callback=_carbonads_go"></script></head>
<body>
<a id="skippy" class="sr-only sr-only-focusable" href="#content">
<div class="container">
<span class="skiplink-text">Skip to main content</span>
</div>
</a>
There's a newer version of Bootstrap 4!
<header class="navbar navbar-expand navbar-dark flex-column flex-md-row bd-navbar">
<a class="navbar-brand mr-0 mr-md-2" href="/" aria-label="Bootstrap"><svg class="d-block" width="36" height="36" viewBox="0 0 612 612" xmlns="http://www.w3.org/2000/svg" focusable="false"><title>Bootstrap</title><path fill="currentColor" d="M510 8a94.3 94.3 0 0 1 94 94v408a94.3 94.3 0 0 1-94 94H102a94.3 94.3 0 0 1-94-94V102a94.3 94.3 0 0 1 94-94h408m0-8H102C45.9 0 0 45.9 0 102v408c0 56.1 45.9 102 102 102h408c56.1 0 102-45.9 102-102V102C612 45.9 566.1 0 510 0z"></path><path fill="currentColor" d="M196.77 471.5V154.43h124.15c54.27 0 91 31.64 91 79.1 0 33-24.17 63.72-54.71 69.21v1.76c43.07 5.49 70.75 35.82 70.75 78 0 55.81-40 89-107.45 89zm39.55-180.4h63.28c46.8 0 72.29-18.68 72.29-53 0-31.42-21.53-48.78-60-48.78h-75.57zm78.22 145.46c47.68 0 72.73-19.34 72.73-56s-25.93-55.37-76.46-55.37h-74.49v111.4z"></path></svg>
</a>
<div class="navbar-nav-scroll">
<ul class="navbar-nav bd-navbar-nav flex-row">
<li class="nav-item">
<a class="nav-link " href="/" onclick="ga('send', 'event', 'Navbar', 'Community links', 'Bootstrap');">Home</a>
</li>
<li class="nav-item">
<a class="nav-link active" href="/docs/4.0/getting-started/introduction/" onclick="ga('send', 'event', 'Navbar', 'Community links', 'Docs');">Documentation</a>
</li>
<li class="nav-item">
<a class="nav-link " href="/docs/4.0/examples/" onclick="ga('send', 'event', 'Navbar', 'Community links', 'Examples');">Examples</a>
</li>
<li class="nav-item">
<a class="nav-link" href="https://themes.getbootstrap.com/" onclick="ga('send', 'event', 'Navbar', 'Community links', 'Themes');" target="_blank" rel="noopener">Themes</a>
</li>
<li class="nav-item">
<a class="nav-link" href="https://expo.getbootstrap.com/" onclick="ga('send', 'event', 'Navbar', 'Community links', 'Expo');" target="_blank" rel="noopener">Expo</a>
</li>
<li class="nav-item">
<a class="nav-link" href="https://blog.getbootstrap.com/" onclick="ga('send', 'event', 'Navbar', 'Community links', 'Blog');" target="_blank" rel="noopener">Blog</a>
</li>
</ul>
</div>
<ul class="navbar-nav flex-row ml-md-auto d-none d-md-flex">
<li class="nav-item dropdown">
<a class="nav-item nav-link dropdown-toggle mr-md-2" href="#" id="bd-versions" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
v4.0
</a>
<div class="dropdown-menu dropdown-menu-right" aria-labelledby="bd-versions">
<a class="dropdown-item" href="/docs/4.1/">Latest (v4.1.x)</a>
<a class="dropdown-item active" href="/docs/4.0/">v4.0.0</a>
<div class="dropdown-divider"></div>
<a class="dropdown-item" href="https://v4-alpha.getbootstrap.com/">v4 Alpha 6</a>
<a class="dropdown-item" href="https://getbootstrap.com/docs/3.3/">v3.3.7</a>
<a class="dropdown-item" href="https://getbootstrap.com/2.3.2/">v2.3.2</a>
</div>
</li>
<li class="nav-item">
<a class="nav-link p-2" href="https://github.com/twbs/bootstrap" target="_blank" rel="noopener" aria-label="GitHub"><svg class="navbar-nav-svg" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 499.36" focusable="false"><title>GitHub</title><path d="M256 0C114.64 0 0 114.61 0 256c0 113.09 73.34 209 175.08 242.9 12.8 2.35 17.47-5.56 17.47-12.34 0-6.08-.22-22.18-.35-43.54-71.2 15.49-86.2-34.34-86.2-34.34-11.64-29.57-28.42-37.45-28.42-37.45-23.27-15.84 1.73-15.55 1.73-15.55 25.69 1.81 39.21 26.38 39.21 26.38 22.84 39.12 59.92 27.82 74.5 21.27 2.33-16.54 8.94-27.82 16.25-34.22-56.84-6.43-116.6-28.43-116.6-126.49 0-27.95 10-50.8 26.35-68.69-2.63-6.48-11.42-32.5 2.51-67.75 0 0 21.49-6.88 70.4 26.24a242.65 242.65 0 0 1 128.18 0c48.87-33.13 70.33-26.24 70.33-26.24 14 35.25 5.18 61.27 2.55 67.75 16.41 17.9 26.31 40.75 26.31 68.69 0 98.35-59.85 120-116.88 126.32 9.19 7.9 17.38 23.53 17.38 47.41 0 34.22-.31 61.83-.31 70.23 0 6.85 4.61 14.81 17.6 12.31C438.72 464.97 512 369.08 512 256.02 512 114.62 397.37 0 256 0z" fill="currentColor" fill-rule="evenodd"></path></svg>
</a>
</li>
<li class="nav-item">
<a class="nav-link p-2" href="https://twitter.com/getbootstrap" target="_blank" rel="noopener" aria-label="Twitter"><svg class="navbar-nav-svg" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 416.32" focusable="false"><title>Twitter</title><path d="M160.83 416.32c193.2 0 298.92-160.22 298.92-298.92 0-4.51 0-9-.2-13.52A214 214 0 0 0 512 49.38a212.93 212.93 0 0 1-60.44 16.6 105.7 105.7 0 0 0 46.3-58.19 209 209 0 0 1-66.79 25.37 105.09 105.09 0 0 0-181.73 71.91 116.12 116.12 0 0 0 2.66 24c-87.28-4.3-164.73-46.3-216.56-109.82A105.48 105.48 0 0 0 68 159.6a106.27 106.27 0 0 1-47.53-13.11v1.43a105.28 105.28 0 0 0 84.21 103.06 105.67 105.67 0 0 1-47.33 1.84 105.06 105.06 0 0 0 98.14 72.94A210.72 210.72 0 0 1 25 370.84a202.17 202.17 0 0 1-25-1.43 298.85 298.85 0 0 0 160.83 46.92" fill="currentColor"></path></svg>
</a>
</li>
<li class="nav-item">
<a class="nav-link p-2" href="https://bootstrap-slack.herokuapp.com" target="_blank" rel="noopener" aria-label="Slack"><svg class="navbar-nav-svg" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512" focusable="false"><title>Slack</title><path fill="currentColor" d="M210.787 234.832l68.31-22.883 22.1 65.977-68.309 22.882z"></path><path d="M490.54 185.6C437.7 9.59 361.6-31.34 185.6 21.46S-31.3 150.4 21.46 326.4 150.4 543.3 326.4 490.54 543.34 361.6 490.54 185.6zM401.7 299.8l-33.15 11.05 11.46 34.38c4.5 13.92-2.87 29.06-16.78 33.56-2.87.82-6.14 1.64-9 1.23a27.32 27.32 0 0 1-24.56-18l-11.46-34.38-68.36 22.92 11.46 34.38c4.5 13.92-2.87 29.06-16.78 33.56-2.87.82-6.14 1.64-9 1.23a27.32 27.32 0 0 1-24.56-18l-11.46-34.43-33.15 11.05c-2.87.82-6.14 1.64-9 1.23a27.32 27.32 0 0 1-24.56-18c-4.5-13.92 2.87-29.06 16.78-33.56l33.12-11.03-22.1-65.9-33.15 11.05c-2.87.82-6.14 1.64-9 1.23a27.32 27.32 0 0 1-24.56-18c-4.48-13.93 2.89-29.07 16.81-33.58l33.15-11.05-11.46-34.38c-4.5-13.92 2.87-29.06 16.78-33.56s29.06 2.87 33.56 16.78l11.46 34.38 68.36-22.92-11.46-34.38c-4.5-13.92 2.87-29.06 16.78-33.56s29.06 2.87 33.56 16.78l11.47 34.42 33.15-11.05c13.92-4.5 29.06 2.87 33.56 16.78s-2.87 29.06-16.78 33.56L329.7 194.6l22.1 65.9 33.15-11.05c13.92-4.5 29.06 2.87 33.56 16.78s-2.88 29.07-16.81 33.57z" fill="currentColor"></path></svg>
</a>
</li>
</ul>
<a class="btn btn-bd-download d-none d-lg-inline-block mb-3 mb-md-0 ml-md-3" href="https://github.com/twbs/bootstrap/archive/v4.0.0.zip">Download</a>
</header>
<div class="container-fluid">
<div class="row flex-xl-nowrap">
<div class="col-12 col-md-3 col-xl-2 bd-sidebar">
<form class="bd-search d-flex align-items-center">
<span class="algolia-autocomplete" style="position: relative; display: inline-block; direction: ltr;"><input type="search" class="form-control ds-input" id="search-input" placeholder="Search..." aria-label="Search for..." autocomplete="off" spellcheck="false" role="combobox" aria-autocomplete="list" aria-expanded="false" aria-owns="algolia-autocomplete-listbox-0" dir="auto" style="position: relative; vertical-align: top;"><pre aria-hidden="true" style="position: absolute; visibility: hidden; white-space: pre; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 16px; font-style: normal; font-variant: normal; font-weight: 400; word-spacing: 0px; letter-spacing: normal; text-indent: 0px; text-rendering: auto; text-transform: none;"></pre><span class="ds-dropdown-menu" role="listbox" id="algolia-autocomplete-listbox-0" style="position: absolute; top: 100%; z-index: 100; display: none; left: 0px; right: auto;"><div class="ds-dataset-1"></div></span></span>
<button class="btn btn-link bd-search-docs-toggle d-md-none p-0 ml-3" type="button" data-toggle="collapse" data-target="#bd-docs-nav" aria-controls="bd-docs-nav" aria-expanded="false" aria-label="Toggle docs navigation"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 30 30" width="30" height="30" focusable="false"><title>Menu</title><path stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-miterlimit="10" d="M4 7h22M4 15h22M4 23h22"></path></svg>
</button>
</form>
.
<div class="bd-example">
<div id="accordion">
<div class="card">
<div class="card-header" id="headingOne">
<h5 class="mb-0">
<button class="btn btn-link" data-toggle="collapse" data-target="#collapseOne" aria-expanded="true" aria-controls="collapseOne">
Collapsible Group Item #1
</button>
</h5>
</div>
<div id="collapseOne" class="collapse show" aria-labelledby="headingOne" data-parent="#accordion">
<div class="card-body">
Anim pariatur cliche reprehenderit, enim eiusmod high life accusamus terry richardson ad squid. 3 wolf moon officia aute, non cupidatat skateboard dolor brunch. Food truck quinoa nesciunt laborum eiusmod. Brunch 3 wolf moon tempor, sunt aliqua put a bird on it squid single-origin coffee nulla assumenda shoreditch et. Nihil anim keffiyeh helvetica, craft beer labore wes anderson cred nesciunt sapiente ea proident. Ad vegan excepteur butcher vice lomo. Leggings occaecat craft beer farm-to-table, raw denim aesthetic synth nesciunt you probably haven't heard of them accusamus labore sustainable VHS.
</div>
</div>
</div>
<div class="card">
<div class="card-header" id="headingTwo">
<h5 class="mb-0">
<button class="btn btn-link collapsed" data-toggle="collapse" data-target="#collapseTwo" aria-expanded="false" aria-controls="collapseTwo">
Collapsible Group Item #2
</button>
</h5>
</div>
<div id="collapseTwo" class="collapse" aria-labelledby="headingTwo" data-parent="#accordion">
<div class="card-body">
Anim pariatur cliche reprehenderit, enim eiusmod high life accusamus terry richardson ad squid. 3 wolf moon officia aute, non cupidatat skateboard dolor brunch. Food truck quinoa nesciunt laborum eiusmod. Brunch 3 wolf moon tempor, sunt aliqua put a bird on it squid single-origin coffee nulla assumenda shoreditch et. Nihil anim keffiyeh helvetica, craft beer labore wes anderson cred nesciunt sapiente ea proident. Ad vegan excepteur butcher vice lomo. Leggings occaecat craft beer farm-to-table, raw denim aesthetic synth nesciunt you probably haven't heard of them accusamus labore sustainable VHS.
</div>
</div>
</div>
<div class="card">
<div class="card-header" id="headingThree">
<h5 class="mb-0">
<button class="btn btn-link collapsed" data-toggle="collapse" data-target="#collapseThree" aria-expanded="false" aria-controls="collapseThree">
Collapsible Group Item #3
</button>
</h5>
</div>
.
<table class="table table-bordered table-striped">
<thead>
<tr>
<th style="width: 100px;">Name</th>
<th style="width: 50px;">Type</th>
<th style="width: 50px;">Default</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>parent</td>
<td>selector | jQuery object | DOM element </td>
<td>false</td>
<td>If parent is provided, then all collapsible elements under the specified parent will be closed when this collapsible item is shown. (similar to traditional accordion behavior - this is dependent on the <code>card</code> class). The attribute has to be set on the target collapsible area.</td>
</tr>
<tr>
<td>toggle</td>
<td>boolean</td>
<td>true</td>
<td>Toggles the collapsible element on invocation</td>
</tr>
</tbody>
</table>
.
<table class="table table-bordered table-striped">
<thead>
<tr>
<th style="width: 150px;">Event Type</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>show.bs.collapse</td>
<td>This event fires immediately when the <code>show</code> instance method is called.</td>
</tr>
<tr>
<td>shown.bs.collapse</td>
<td>This event is fired when a collapse element has been made visible to the user (will wait for CSS transitions to complete).</td>
</tr>
<tr>
<td>hide.bs.collapse</td>
<td>This event is fired immediately when the <code>hide</code> method has been called.</td>
</tr>
<tr>
<td>hidden.bs.collapse</td>
<td>This event is fired when a collapse element has been hidden from the user (will wait for CSS transitions to complete).</td>
</tr>
</tbody>
</table>
<div class="bd-clipboard"><button class="btn-clipboard" title="" data-original-title="Copy to clipboard">Copy</button></div><figure class="highlight"><pre><code class="language-js" data-lang="js"><span class="nx">$</span><span class="p">(</span><span class="s1">'#myCollapsible'</span><span class="p">).</span><span class="nx">on</span><span class="p">(</span><span class="s1">'hidden.bs.collapse'</span><span class="p">,</span> <span class="kd">function</span> <span class="p">()</span> <span class="p">{</span>
<span class="c1">// do something…</span>
<span class="p">})</span></code></pre></figure>
</main>
</div>
</div>
<script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>
<script>window.jQuery || document.write('<script src="/docs/4.0/assets/js/vendor/jquery-slim.min.js"><\/script>')</script>
<script src="/docs/4.0/assets/js/vendor/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script><script src="/docs/4.0/dist/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script><script src="https://cdn.jsdelivr.net/npm/docsearch.js#2/dist/cdn/docsearch.min.js"></script><script src="/docs/4.0/assets/js/docs.min.js"></script>
</body></html>
Maybe the DOM nodes you are looking for are inside iframes. In that case, you need to look for them inside those iframes by:
driver = webdriver.Chrome()
driver.get("https://www.google.com")
iframes = driver.find_elements_by_tag_name('iframe')
for pos, iframe in enumerate(iframes):
src = iframe.get_attribute('src')
driver.switch_to.frame(iframe)
# driver = WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.ID, "...")))
source = getattr(driver, "page_source", "no page source")
print(pos, src, len(source), source[:1000])
driver.switch_to.default_content()
It's hard to tell from context but if you have a string containing the page html source then parsing it with Beautiful Soup will do. Maybe not ideal if you need to keep the number of dependencies as small as possible but thats an easy fix.
This webpage opens fine manually, but directly goes to a "maintenance" error message when using Selenium !
from selenium import webdriver
driver = webdriver.Chrome(executable_path="chromedriver")
driver.get("https://www.winamax.fr/paris-sportifs/")
Is there a way to avoid this behaviour ?
Sometimes the websites checks your user-agent. Maybe you can change that.
Otherwise, I would recommend using Auto-It to do the job.
You could also try loading in your personal Chrome profile into Selenium.
A bit unclear why you felt website blocking Selenium. However I was able to access the website following the solution below:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://www.winamax.fr/paris-sportifs/')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "section#above-content a[href='/paris-sportifs']")))
print(driver.page_source)
Console Output:
<html class="app-desktop js flexbox canvas canvastext webgl no-touch geolocation postmessage websqldatabase indexeddb hashchange history draganddrop websockets rgba hsla multiplebgs backgroundsize borderimage borderradius boxshadow textshadow opacity cssanimations csscolumns cssgradients cssreflections csstransforms csstransforms3d csstransitions fontface generatedcontent video audio localstorage sessionstorage webworkers applicationcache svg inlinesvg smil svgclippaths pointerevents cssremunit" lang="fr" style=""><!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--><!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--><!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--><!--[if gt IE 8]><!--><!--<![endif]--><head>
<title>Paris Sportifs - Parier en ligne avec Winamax</title>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<meta name="viewport" content="width=1024">
<meta name="verify-v1" content="divGwlYLgkmoS558zISY8BYg1KLLvQLFmRf0CmPJ1kc=">
<meta name="google-site-verification" content="DB-aVdWJ00FHClrX_fsXFGgZzaYUKGVcvY7uSfOkXsw">
<meta name="format-detection" content="telephone=no">
<meta name="apple-mobile-web-app-title" content="Winamax">
<meta property="og:description" content="Agréé par l’Arjel – Pariez sur le sport sur Winamax! Faites un premier dépôt et votre premier pari sera remboursé si il est perdant.">
<meta property="og:image" content="https://operator-front-static-cdn.winamax.fr/img/content/betting/ParisSportif_Facebook.jpg?v=20150401">
<meta property="fb:admins" content="519584907">
<meta property="og:site_name" content="Winamax.fr">
<meta property="og:type" content="game">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:site" content="#WinamaxSport">
<meta name="twitter:title" content="Paris Sportifs - Parier en ligne avec Winamax">
<meta name="twitter:description" content="Agréé par l’Arjel – Pariez sur le sport sur Winamax! Faites un premier dépôt et votre premier pari sera remboursé si il est perdant.">
<meta name="twitter:image:src" content="https://operator-front-static-cdn.winamax.fr/img/content/betting/ParisSportif_Facebook_twitter.jpg?v=20150401">
<meta name="twitter:domain" content="winamax.fr">
<meta property="og:url" content="https://www.winamax.fr/paris-sportifs/">
<link rel="stylesheet" id="normalize" href="https://operator-front-static-cdn.winamax.fr/style/v2/normalize.css?v=20191219-1" type="text/css" media="all">
<link rel="stylesheet" id="reset" href="https://operator-front-static-cdn.winamax.fr/style/v2/reset.css?v=20191219-1" type="text/css" media="all">
<link rel="stylesheet" id="fontawesome" href="https://operator-front-static-cdn.winamax.fr/style/v2/fontawesome.min.css?v=20191219-1" type="text/css" media="all">
<link rel="stylesheet" id="magnific-popup" href="https://operator-front-static-cdn.winamax.fr/style/v2/magnific-popup.css?v=20191219-1" type="text/css" media="all">
<link rel="stylesheet" id="spritesheet" href="https://operator-front-static-cdn.winamax.fr/style/v2/spritesheet.css?v=20191219-1" type="text/css" media="all">
<link rel="stylesheet" id="common" href="https://operator-front-static-cdn.winamax.fr/style/v2/common.css?v=20191219-1" type="text/css" media="all">
<link rel="stylesheet" id="fancybox" href="https://operator-front-static-cdn.winamax.fr/style/v2/jquery.fancybox.css?v=20191219-1" type="text/css" media="all">
<link rel="stylesheet" id="doubleslider" href="https://operator-front-static-cdn.winamax.fr/style/v2/doubleslider.css?v=20191219-1" type="text/css" media="all">
<link rel="stylesheet" id="source-sans-pro" href="https://operator-front-static-cdn.winamax.fr/style/fonts/SourceSansPro/source-sans-pro.css?v=20191219-1" type="text/css" media="all">
<link rel="icon" href="https://operator-front-static-cdn.winamax.fr/img/style/v2/favicon.ico" type="image/png">
<link rel="manifest" href="/manifest.json">
<link rel="apple-touch-icon" href="https://operator-front-static-cdn.winamax.fr/img/style/v2/20170721_touch_icon_winamax.png">
<link rel="canonical" href="https://www.winamax.fr/paris-sportifs/">
<link rel="alternate" hreflang="fr" href="https://www.winamax.fr/paris-sportifs/">
<link rel="alternate" hreflang="en" href="https://www.winamax.fr/en/sports-betting/">
<link rel="alternate" hreflang="de" href="https://www.winamax.fr/de/sportwetten/">
<script async="" src="https://www.google-analytics.com/analytics.js"></script><script type="text/javascript" async="" src="https://ssl.google-analytics.com/ga.js"></script><script type="text/javascript" src="https://operator-front-static-cdn.winamax.fr/script/swfobject.js?v=20191219-1"></script>
<script type="text/javascript" src="https://operator-front-static-cdn.winamax.fr/script/v2/betting-helpers.js?v=20191219-1"></script>
<script type="text/javascript" src="https://operator-front-static-cdn.winamax.fr/script/numeral.js?v=20191219-1"></script>
<script type="text/javascript" src="https://operator-front-static-cdn.winamax.fr/script/numeral.languages.js?v=20191219-1"></script>
<script src="https://operator-front-static-cdn.winamax.fr/script/v2/lib/modernizr.custom.js?v=20191219-1"></script>
<!--[if lte IE 8]>
<script src="https://operator-front-static-cdn.winamax.fr/script/v2/lib/jquery-1.10.1.min.js?v=20191219-1"></script>
<script src="https://operator-front-static-cdn.winamax.fr/script/v2/lib/selectivizr-min.js?v=20191219-1"></script>
<![endif]-->
<script language="Javascript" type="text/javascript" src="https://operator-front-static-cdn.winamax.fr/script/messages/messages_fr.js?v=20191219-1"></script>
<script>
var $siteLanguage = 'FR';
var $siteLanguagePath = '';
var $siteLicense = 'FR';
</script>
<style data-styled-components=""></style><script charset="utf-8" src="https://operator-front-static-cdn.winamax.fr/betting/client/1.55.1/main.30da6bcf4d935f4ad03c.js"></script><script charset="utf-8" src="https://operator-front-static-cdn.winamax.fr/betting/client/1.55.1/main.6dcf45fd436abb91aedd.js"></script><script charset="utf-8" src="https://operator-front-static-cdn.winamax.fr/betting/client/1.55.1/main.1812cb976eb7abff1666.js"></script><script charset="utf-8" src="https://operator-front-static-cdn.winamax.fr/betting/client/1.55.1/main.fbe533b04c719597d791.js"></script><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="jquery" src="https://operator-front-static-cdn.winamax.fr/script/v2/lib/jquery-1.12.4.min.js?v=20191219-1"></script><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="swipe" src="https://operator-front-static-cdn.winamax.fr/script/v2/lib/swipe.js?v=20191219-1"></script><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="touch" src="https://operator-front-static-cdn.winamax.fr/script/v2/lib/jquery.touch.js?v=20191219-1"></script><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="magnific" src="https://operator-front-static-cdn.winamax.fr/script/v2/lib/jquery.magnific-popup.min.js?v=20191219-1"></script><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="fancybox" src="https://operator-front-static-cdn.winamax.fr/script/v2/lib/jquery.fancybox.pack.js?v=20191219-1"></script><style id="detectElementResize" type="text/css">#keyframes resizeanim { from { opacity: 0; } to { opacity: 0; } } .resize-triggers { animation: 1ms resizeanim; visibility: hidden; opacity: 0; } .resize-triggers, .resize-triggers > div, .contract-trigger:before { content: " "; display: block; position: absolute; top: 0; left: 0; height: 100%; width: 100%; overflow: hidden; z-index: -1; } .resize-triggers > div { background: #eee; overflow: auto; } .contract-trigger:before { width: 200%; height: 200%; }</style><style type="text/css">.fancybox-margin{margin-right:17px;}</style><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="script/v2/common" src="https://operator-front-static-cdn.winamax.fr/script/v2/common.js?v=20191219-1"></script><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="script/v2/mobile" src="https://operator-front-static-cdn.winamax.fr/script/v2/mobile.js?v=20191219-1"></script><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="script/v2/gallery" src="https://operator-front-static-cdn.winamax.fr/script/v2/gallery.js?v=20191219-1"></script><script type="text/javascript" charset="utf-8" async="" data-requirecontext="_" data-requiremodule="script/v2/doubleslider" src="https://operator-front-static-cdn.winamax.fr/script/v2/doubleslider.js?v=20191219-1"></script></head>
<body class="lang-fr license-fr">
<div id="wrap-menu-overlay"></div>
<!-- Facebook -->
<div id="fb-root"></div>
<script>$fbLocalized = 'fr_FR';</script>
<div id="doc">
<div id="inner-wrap"><div style="cursor: pointer; background: url("https://static.winamax.fr/img/style/v2/common/avertissement.png?v=2") center center no-repeat rgb(0, 0, 0); height: 60px; z-index: 112; display: block;"></div>
<script>
var arjelBannerImgPath = Math.random() > 0.5 ? "https://static.winamax.fr/img/style/v2/common/avertissement.png?v=2" : "https://static.winamax.fr/img/style/v2/common/avertissement-alt.png";
function setupArjelBanner() {
var ref = document.getElementById("inner-wrap");
if (ref) {
var banner = document.createElement("div");
banner.style.cursor = "pointer";
banner.style.background = "url('" + arjelBannerImgPath + "') no-repeat center center #000";
banner.style.height = "60px";
banner.style.zIndex = Math.floor(50 + Math.random() * 100);
function goArjelInfos() {
window.open("https://www.winamax.fr/CLIC/PREVENTION/JIS_HOME/www.joueurs-info-service.fr");
}
function WidthChange(mq) {
if (mq.matches) {
var arjelBannerImgPath = Math.random() > 0.5 ? "https://static.winamax.fr/img/style/v2/common/avertissement-mobile.png?v=2" : "https://static.winamax.fr/img/style/v2/common/avertissement-alt-mobile.png";
banner.style.background = "url('" + arjelBannerImgPath + "') no-repeat center center #000";
banner.style.backgroundSize = "100% auto";
} else {
banner.style.display = "block";
}
}
banner.addEventListener("click", goArjelInfos, false);
ref.insertBefore(banner, ref.firstChild);
if (matchMedia) {
var mq = window.matchMedia("(max-width: 480px)");
mq.addListener(WidthChange);
WidthChange(mq);
}
}
}
setupArjelBanner();
</script>
<div class="top-notif-wrapper"><div class="notification top-notif medium cookie-consent">
<i class="icon-warning-sign"></i>
<p><span>En poursuivant votre navigation sur ce site, vous acceptez l’utilisation de Cookies afin de réaliser des statistiques de visites et vous proposer des promotions adaptées. S’y opposer - Plus d’informations sur les cookies</span></p>
<i class="icon-remove"></i></div>
</div><header id="masthead">
<div class="container">
<a href="/" id="logo">
<img src="https://operator-front-static-cdn.winamax.fr/img/style/v2/common/logo.png" alt="Winamax">
<img class="hover" src="https://operator-front-static-cdn.winamax.fr/img/style/v2/common/logo-highlight.png" alt="">
</a>
<div class="content">
<div id="login-toggle" class="toggle">
<a href="/account/login.php?redir=/paris-sportifs/">
<svg id="user-icon" viewBox="0 0 32 24">
<path d="M10.3,13.9c0,0-7.1-0.1-9.8,5.3v4.7h9.8h9.8v-4.7C17.5,13.8,10.3,13.9,10.3,13.9z"></path>
<ellipse cx="10.3" cy="6.4" rx="4.9" ry="6.2"></ellipse>
</svg>
</a>
</div>
<div id="login-container" class="expandable">
Se connecter
S'inscrire
<div id="login-popup">
<form id="login-form" method="post" action="/paris-sportifs/" autocomplete="off">
<p class="field">
<label>Adresse email</label>
<input type="email" name="email" id="loginbox_email" placeholder="vous#exemple.com" tabindex="1" autocomplete="off" value="">
</p>
<p class="field">
<label>Mot de passe</label>
<input type="password" id="loginbox_password" name="password" placeholder="" tabindex="2" autocomplete="off">
<a class="forgot-password" href="/account/lost_password.php">Mot de passe oublié ?</a>
</p>
<p class="field birthdate">
<label for="day-input">Date de naissance</label>
<input type="text" id="loginbox_birthday" name="birth_day" autocomplete="off" placeholder="JJ" tabindex="3" pattern="[0-9]*" size="2" maxlength="2" value="">
<input type="text" id="loginbox_birthmonth" name="birth_month" autocomplete="off" placeholder="MM" tabindex="4" pattern="[0-9]*" size="2" maxlength="2" value="">
<input type="text" id="loginbox_birthyear" name="birth_year" autocomplete="off" placeholder="AAAA" tabindex="5" pattern="[0-9]*" size="4" maxlength="4" value="">
</p>
<button type="submit" id="login-button" name="submitlogin" class="secondary-button" tabindex="6"><i class="icon-lock"></i>Connexion</button>
</form>
<div id="no-account">
<span>Pas encore de compte ?</span> Inscrivez-vous gratuitement
</div>
</div>
<div id="pin-popup">
</div>
</div>
<div id="search" class="expandable" style="height:28px">
</div>
<div id="nav-toggle" class="toggle">
<svg id="list-icon" viewBox="0 0 24 24">
<rect x="6" y="17" width="18" height="3"></rect>
<rect x="6" y="10.5" width="18" height="3"></rect>
<rect x="6" y="4" width="18" height="3"></rect>
<rect x="0" y="4" width="3" height="3"></rect>
<rect x="0" y="10.5" width="3" height="3"></rect>
<rect x="0" y="17" width="3" height="3"></rect>
</svg>
</div>
</div>
</div>
<nav id="main-nav" class="">
<ul class="main-nav-list">
<li class="">
JOUER AU POKER
</li>
<li class="focus">
PARIS SPORTIFS
</li>
<li class="">
GRILLES
</li>
<li class="">
JEU DE L’ENTRAINEUR
</li>
<li class="">
PROMOS
</li>
<li class="">
VIP
</li>
<li class="">
ACTUS
</li>
</ul>
</nav>
</header>
<section id="above-content">
<nav id="secondary-nav" class="container">
<div id="section-title">
Paris Sportifs
</div>
<ul>
<li>À la Une</li>
<li>LIVE</li>
<li>Multiplex</li>
<li>Winamax TV</li>
<li>Calendrier</li>
<li>Mes paris</li>
<li>Stats</li>
<li>Résultats</li>
</ul>
</nav>
</section>
I want to get 'a' tags inside div with the class 'x'. I'm trying this code:
u = urllib.urlopen('http://www.full-hd-wallpaper.com/all')
data = u.read()
soap = BeautifulSoup(data)
print soap.select("div.wallpaper_item a")
but the result is empty. I'm sure the selector is correct. I also tried this simple selector:
print soap.select("div")
but nothing returned. what's wrong with my code?
this is the input:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US" xmlns:fb="http://www.facebook.com/2008/fbml">
<head>
<link rel="stylesheet" type="text/css" href="/templates/darkbrush/style.css?20140420:8" />
<script type="text/javascript">
SITE_URL = '';
SEO_ON = '3';
COMMENT_WAIT = 'Please wait 60 seconds between comments';
COMMENT_ERROR = 'An error occured in sending your comment';
WALLPAPER_SUBMIT_COMMENT = 'Submit comment';
ADDING_COMMENT = 'Adding comment';
COMMENT_ADDED = 'Comment added';
function WallpaperAddHit(id) {
AjaxPost("/includes/wallpaper/ajax/wallpaper_hit.php", "wallpaper_id="+id,
function () {}
)
}
</script>
<script type="text/javascript" src="/includes/wss.js"></script>
<link rel="alternate" type="application/rss+xml" title="" href="/rss.php" />
<title>All wallpapers - Full HD Wallpaper</title><meta name="description" content="All wallpapers All our wallpapers in one place!" />
<meta name="keywords" content="Full HD Wallpapers, wallpapers 2013, full hd wallpaper,wallpapers, HD Wallpapers, HD Wallpaper, desktops, downloads,Wallpaper,hd wallpaper download, hd wallpaper nature, hd wallpaper 1920x1080, HD Wallpaper 1080p, hd wallpaper for android, " /><meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" />
<link rel="icon" href="/favicon.ico" type="image/x-icon" />
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-42003300-1', 'full-hd-wallpaper.com');
ga('send', 'pageview');
</script>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-45774874-1']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</head>
<body>
<!-- Report popup and overlay !-->
<div id="ava-popup">
<div id="ava-popup-header">
<div id="ava-popup-title"></div>
<div id="popup-close-button" onclick="HidePopup('ava-popup');"></div>
</div>
<div id="ava-popup-content"></div>
</div>
<div id="overlay" onclick="HidePopup('ava-popup')"></div>
<div class="header">
<div class="header_logo">
<img src="/templates/darkbrush/images/logo.png" alt="Full HD Wallpaper" />
</div>
<div class="header_right">
<!--ads-->
</div>
<br style="clear:both;" />
</div>
<div class="menu">
<div class="menu_left">
<div class="menu_item">
Homepage</div><div class="menu_item">
New wallpapers</div><div class="menu_item">
Most Downloads</div><div class="menu_item">
Top rated</div><div class="menu_item"></div><div class="menu_item">Copyright policy</div><div class="menu_item">Contact Us </div>
</div>
<div class="menu_right">
<form action="/index.php?task=search" method="get" onsubmit="searchSubmit('', ''); return false;">
<input name="task" type="hidden" value="search" />
<div class="search_contain">
<div class="search_contain_left">
<input name="q" type="text" size="20" id="search_textbox" value="Search..." onclick="clickclear(this, 'Search...')" onblur="clickrecall(this,'Search...')" class="search_box" />
</div>
<div class="search_contain_right">
<input type="image" style="margin-top:7px;" src="/templates/darkbrush/images/search_button.png" />
</div>
</div>
</form>
</div>
</div>
<div class="mc_bg">
<div class="main_container">
<div class="secondary_container">
<div class="left_sidebar">
News
Subscribe
Links<br />
<form action="http://feedburner.google.com/fb/a/mailverify" method="post" target="popupwindow" onsubmit="window.open('http://feedburner.google.com/fb/a/mailverify?uri=FullHdWallpaper', 'popupwindow', 'scrollbars=yes,width=550,height=520');return true"><input type="hidden" name="loc" value="en_US" /><input type="hidden" value="FullHdWallpaper" name="uri" /><h5>Email Subscription</h5>
<input type="text" class="email_textbox" name="email" size="20" />
<input type="submit" class="email_button" value="Subscribe" />
</form>
<br />
<h2>Categories</h2>
<div class="category_menu_item">
All wallpapers</div><div class="category_menu_item">Abstract</div><div class="category_menu_item">Animals</div><div class="category_menu_item">Nature</div><div class="category_menu_item">Men</div><div class="category_menu_item">Children</div><div class="category_menu_item">Girls and Women</div><div class="category_menu_item">World</div><div class="category_menu_item">Foods</div><div class="category_menu_item">Cars</div><div class="category_menu_item">Technology</div><div class="category_menu_item">Holiday</div><div class="category_menu_item">Other</div><div class="category_menu_item">Flower</div>
<br />
<h2>Tags</h2>
<div class="tag_cloud">Adriana lima wallpaper Alessandra ambrosio wallpaper Amber heard wallpaper Beyonce wallpaper Britney Spears wallpaper Candice swanepoel wallpaper Cheryl cole wallpaper Doutzen kroes wallpaper Elisha cuthbert wallpaper Ewelina olczak wallpapers Inna wallpaper Jennifer lawrence wallpapers Jennifer lopez wallpaper Jessica alba wallpaper Kate upton wallpaper Lady Gaga wallpaper Lindsey stirling wallpaper Marloes horst wallpaper Natalia vodianova wallpaper Nicki minaj wallpaper Nicole scherzinger wallpaper Rihanna wallpaper Robert pattinson wallpaper Scarlett johansson wallpaper Taylor swift wallpaper Woman with Car wallpaper alexandra stan wallpaper alfa romeo wallpaper aston martin wallpaper audi wallpaper bar refaeli wallpaper barbara palvin wallpaper benz wallpaper bmw wallpaper bugatti wallpaper emma watson wallpaper eva mendes wallpaper ferrari wallpaper ford wallpaper irina shayk wallpaper kate beckinsale wallpaper katy perry wallpaper kelly clarkson wallpaper lamborghini wallpaper megan fox wallpaper miranda kerr wallpaper motorcycle wallpaper porsche wallpaper rosie huntington wallpaper selena gomez wallpaper suzuki wallpaper </div><br />
<h2>New</h2>
<div class="module_wallpaper">
<a href="/nature/windmill-morning-sunrise-2">
<img src="/image.php?width=229&height=129&id=5997&nocache=1&dothumb=1" alt="windmill morning sunrise" width="150" height="85" />
</a>
</div><div class="module_wallpaper">
<a href="/nature/landscapes-nature-roads">
<img src="/image.php?width=229&height=129&id=5996&nocache=1&dothumb=1" alt="landscapes nature roads" width="150" height="85" />
</a>
</div><div class="module_wallpaper">
<a href="/girls-and-women/candice-swanepoel-model">
<img src="/image.php?width=229&height=129&id=5995&nocache=1&dothumb=1" alt="candice swanepoel model" width="150" height="85" />
</a>
</div><br /> </div>
<div class="right_sidebar">
<h2>Your account</h2>
<form method="post" action="/login.php?done=1">
<div class="mini_login_form">
<p>Username</p>
<input name="username" type="text" id="username" class="mini_login_textbox" /><br />
<p>Password</p>
<input name="password" type="password" id="password" class="mini_login_textbox" /><br />
<p><label><input type="checkbox" name="remember" id="remember" checked="checked" /> Keep me logged in</label></p>
<input type="submit" name="Submit" value="Login" class="mini_login_button" />
Forgotten your password?
</div>
</form>
Register new account
<br />
<h2>Recently viewed</h2>
<div class="no_recents">None...</div><br />
<h2>Stats</h2>
<ul class="stats_ul">
<li><strong>317</strong> users online</li>
<li><strong>5729</strong> wallpapers</li>
<li><strong>76823</strong> members</li>
<li><strong>0</strong> news posts</li>
<li><strong>31</strong> comments</li>
<li><strong>19</strong> categories</li>
</ul><br />
<h2>Links</h2>
<ul><li>Fashion Shows</li><li>wallpapers gallery</li><li>Girls Wallpapers</li><li>High Definition Wallpapers</li></ul><div class="more_links">More links</div>
</div>
<div class="center_column">
<div class="center_container">
<center>
<div class="gas">
<script async="async" src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
<!-- wallpaper-text-728 -->
<ins class="adsbygoogle" style="display:inline-block;width:728px;height:15px" data-ad-client="ca-pub-3574787538747201" data-ad-slot="8438048800"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script> </div>
</center>
<div class="header_overflow"><h1>Homepage » All wallpapers</h1></div>
<div class="ad_banner_misc">
</div>
<div class="category_sort_options">
Sort options: <a class="sort_bold" href="/all">Newest</a> | <a class="sort_notbold" href="/all/oldest/1">Oldest</a> | <a class="sort_notbold" href="/all/rating/1">Top Rated</a> | <a class="sort_notbold" href="/all/downloads/1">Most Downloads</a> | <a class="sort_notbold" href="/all/nameasc/1">A-Z</a> | <a class="sort_notbold" href="/all/namedesc/1">Z-A</a> <select class="select" id="resolution_filter" name="resolution_filter" onchange="setResFilter();">
<option value="all">All resolutions</option><optgroup label="4:3"> <option value="1600x1200">1600x1200 </option><option value="1400x1050">1400x1050 </option><option value="1280x960">1280x960 </option><option value="1024x768">1024x768 </option><option value="800x600">800x600 </option></optgroup><optgroup label="16:9"> <option value="2560x1440">2560x1440 </option><option value="1920x1080">1920x1080 (1080p) </option><option value="1600x900">1600x900 </option><option value="1280x720">1280x720 (720p) </option></optgroup><optgroup label="16:10"> <option value="2880x1800">2880x1800 </option><option value="2560x1600">2560x1600 </option><option value="1920x1200">1920x1200 </option><option value="1680x1050">1680x1050 </option><option value="1440x900">1440x900 </option><option value="1280x800">1280x800 </option></optgroup><optgroup label="Apple"> <option value="2048x2048">Retina iPad </option><option value="1024x1024">iPad / iPad mini </option><option value="640x1136">iPhone 5 (& iPod) </option><option value="640x960">iPhone 4/4S/iPod </option><option value="320x480">Older iPhone & iPod </option></optgroup><optgroup label="Blackberry"> <option value="360x480">360x480 </option><option value="320x240">320x240 </option></optgroup><optgroup label="Google Android"> <option value="720x1280">720x1280 </option><option value="540x960">540x960 </option><option value="480x854">480x854 </option><option value="480x800">480x800 </option><option value="320x480">320x480 </option></optgroup><optgroup label="Netbook"> <option value="1366x768">1366x768 </option><option value="1024x600">1024x600 </option><option value="800x480">800x480 </option></optgroup><optgroup label="Other resolutions"> <option value="960x544">960x544 PS Vita </option><option value="480x272">480x272 PSP </option></optgroup><optgroup label="Windows Phone 7/8"> <option value="768x1280">768x1280 </option><option value="720x1280">720x1280 </option><option value="480x800">480x800 </option></optgroup></select> </div>
<script async="async" src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
<!-- wallpaper-720 -->
<ins class="adsbygoogle" style="display:inline-block;width:728px;height:90px" data-ad-client="ca-pub-3574787538747201" data-ad-slot="1054382805"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script><br /><br />
<div class="category_container">
<div class="wallpaper_item">
<a href="/nature/windmill-morning-sunrise-2">
<img src="/image.php?width=229&height=129&id=5997&nocache=1&dothumb=1" alt="windmill morning sunrise" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/windmill-morning-sunrise-2">
windmill morning sunrise </a>
</div>
</div><div class="wallpaper_item">
<a href="/nature/landscapes-nature-roads">
<img src="/image.php?width=229&height=129&id=5996&nocache=1&dothumb=1" alt="landscapes nature roads" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/landscapes-nature-roads">
landscapes nature roads </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/candice-swanepoel-model">
<img src="/image.php?width=229&height=129&id=5995&nocache=1&dothumb=1" alt="candice swanepoel model" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/candice-swanepoel-model">
candice swanepoel model </a>
</div>
</div><div class="wallpaper_item">
<a href="/cars/bugatti-veyron-grand-sport-2014">
<img src="/image.php?width=229&height=129&id=5994&nocache=1&dothumb=1" alt="bugatti veyron grand spor…" />
</a>
<div class="wallpaper_item_name">
<a href="/cars/bugatti-veyron-grand-sport-2014">
bugatti veyron grand spor… </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/beyonce-knowles-singer">
<img src="/image.php?width=229&height=129&id=5993&nocache=1&dothumb=1" alt="beyonce knowles singer" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/beyonce-knowles-singer">
beyonce knowles singer </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/vintage-girl">
<img src="/image.php?width=229&height=129&id=5992&nocache=1&dothumb=1" alt="vintage girl" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/vintage-girl">
vintage girl </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/jennifer-lopez-2014">
<img src="/image.php?width=229&height=129&id=5991&nocache=1&dothumb=1" alt="jennifer lopez 2014" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/jennifer-lopez-2014">
jennifer lopez 2014 </a>
</div>
</div><div class="wallpaper_item">
<a href="/cars/bmw-m3">
<img src="/image.php?width=229&height=129&id=5990&nocache=1&dothumb=1" alt="bmw m3" />
</a>
<div class="wallpaper_item_name">
<a href="/cars/bmw-m3">
bmw m3 </a>
</div>
</div><div class="wallpaper_item">
<a href="/nature/tree-leaves-fog">
<img src="/image.php?width=229&height=129&id=5989&nocache=1&dothumb=1" alt="tree leaves fog" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/tree-leaves-fog">
tree leaves fog </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/hand-lips-girl">
<img src="/image.php?width=229&height=129&id=5988&nocache=1&dothumb=1" alt="hand lips girl" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/hand-lips-girl">
hand lips girl </a>
</div>
</div><div class="wallpaper_item">
<a href="/nature/spring-sheet">
<img src="/image.php?width=229&height=129&id=5987&nocache=1&dothumb=1" alt="spring sheet" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/spring-sheet">
spring sheet </a>
</div>
</div><div class="wallpaper_item">
<a href="/cars/srt-viper-2014">
<img src="/image.php?width=229&height=129&id=5986&nocache=1&dothumb=1" alt="srt viper 2014" />
</a>
<div class="wallpaper_item_name">
<a href="/cars/srt-viper-2014">
srt viper 2014 </a>
</div>
</div><div class="wallpaper_item">
<a href="/animals/yellow-butterfly-red">
<img src="/image.php?width=229&height=129&id=5985&nocache=1&dothumb=1" alt="yellow butterfly red" />
</a>
<div class="wallpaper_item_name">
<a href="/animals/yellow-butterfly-red">
yellow butterfly red </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/art-girl-butterflies">
<img src="/image.php?width=229&height=129&id=5984&nocache=1&dothumb=1" alt="art girl butterflies" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/art-girl-butterflies">
art girl butterflies </a>
</div>
</div><div class="wallpaper_item">
<a href="/cities/eiffel-tower-2">
<img src="/image.php?width=229&height=129&id=5983&nocache=1&dothumb=1" alt="eiffel tower" />
</a>
<div class="wallpaper_item_name">
<a href="/cities/eiffel-tower-2">
eiffel tower </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/hair-face-wind">
<img src="/image.php?width=229&height=129&id=5982&nocache=1&dothumb=1" alt="hair face wind" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/hair-face-wind">
hair face wind </a>
</div>
</div><div class="wallpaper_item">
<a href="/cars/bmw-f30">
<img src="/image.php?width=229&height=129&id=5981&nocache=1&dothumb=1" alt="bmw f30" />
</a>
<div class="wallpaper_item_name">
<a href="/cars/bmw-f30">
bmw f30 </a>
</div>
</div><div class="wallpaper_item">
<a href="/nature/purple-ears-field">
<img src="/image.php?width=229&height=129&id=5980&nocache=1&dothumb=1" alt="purple ears field" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/purple-ears-field">
purple ears field </a>
</div>
</div><div class="wallpaper_item">
<a href="/other/bike">
<img src="/image.php?width=229&height=129&id=5979&nocache=1&dothumb=1" alt="bike" />
</a>
<div class="wallpaper_item_name">
<a href="/other/bike">
bike </a>
</div>
</div><div class="wallpaper_item">
<a href="/cars/porsche-carrera">
<img src="/image.php?width=229&height=129&id=5978&nocache=1&dothumb=1" alt="porsche carrera" />
</a>
<div class="wallpaper_item_name">
<a href="/cars/porsche-carrera">
porsche carrera </a>
</div>
</div><div class="wallpaper_item">
<a href="/nature/mountain-nature-landscape">
<img src="/image.php?width=229&height=129&id=5977&nocache=1&dothumb=1" alt="mountain nature landscape" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/mountain-nature-landscape">
mountain nature landscape </a>
</div>
</div><div class="wallpaper_item">
<a href="/cars/suzuki-gsx-r600-motorcycle">
<img src="/image.php?width=229&height=129&id=5976&nocache=1&dothumb=1" alt="suzuki gsx r600 motorcycl…" />
</a>
<div class="wallpaper_item_name">
<a href="/cars/suzuki-gsx-r600-motorcycle">
suzuki gsx r600 motorcycl… </a>
</div>
</div><div class="wallpaper_item">
<a href="/nature/road-mountain">
<img src="/image.php?width=229&height=129&id=5975&nocache=1&dothumb=1" alt="road mountain" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/road-mountain">
road mountain </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/car-2">
<img src="/image.php?width=229&height=129&id=5973&nocache=1&dothumb=1" alt="car" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/car-2">
car </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/brunette-dress">
<img src="/image.php?width=229&height=129&id=5972&nocache=1&dothumb=1" alt="brunette dress" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/brunette-dress">
brunette dress </a>
</div>
</div><div class="wallpaper_item">
<a href="/girls-and-women/model-dress">
<img src="/image.php?width=229&height=129&id=5971&nocache=1&dothumb=1" alt="model dress" />
</a>
<div class="wallpaper_item_name">
<a href="/girls-and-women/model-dress">
model dress </a>
</div>
</div><div class="wallpaper_item">
<a href="/animals/africa-elephants">
<img src="/image.php?width=229&height=129&id=5970&nocache=1&dothumb=1" alt="africa elephants" />
</a>
<div class="wallpaper_item_name">
<a href="/animals/africa-elephants">
africa elephants </a>
</div>
</div><div class="wallpaper_item">
<a href="/nature/green-trees-forest-akes">
<img src="/image.php?width=229&height=129&id=5969&nocache=1&dothumb=1" alt="Green trees forest akes" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/green-trees-forest-akes">
Green trees forest akes </a>
</div>
</div><div class="wallpaper_item">
<a href="/animals/wild-cheetah-alone">
<img src="/image.php?width=229&height=129&id=5968&nocache=1&dothumb=1" alt="wild cheetah alone" />
</a>
<div class="wallpaper_item_name">
<a href="/animals/wild-cheetah-alone">
wild cheetah alone </a>
</div>
</div><div class="wallpaper_item">
<a href="/nature/lake-3">
<img src="/image.php?width=229&height=129&id=5967&nocache=1&dothumb=1" alt="lake" />
</a>
<div class="wallpaper_item_name">
<a href="/nature/lake-3">
lake </a>
</div>
</div> </div>
<br />
<script async="async" src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
<!-- wallpaper-720 -->
<ins class="adsbygoogle" style="display:inline-block;width:728px;height:90px" data-ad-client="ca-pub-3574787538747201" data-ad-slot="1054382805"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script><br />
<div class="category_pages">
<div class="pagination_wrap">
<b><font color="#2B6FE4">1</font></b> 2 3 4 5 6 7 8 ... 188 189 Next » </div>
</div>
<br />
<div style="padding: 10px">
<center>
</center>
</div>
</div>
</div>
</div>
<div class="ad_banner_footer"><br style="clear:both" />
<script async="async" src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script>
<!-- wallpaper-text-728 -->
<ins class="adsbygoogle" style="display:inline-block;width:728px;height:15px" data-ad-client="ca-pub-3574787538747201" data-ad-slot="8438048800"></ins>
<script>
(adsbygoogle = window.adsbygoogle || []).push({});
</script>
</div>
</div>
</div>
<div class="footer">
Powered by Wallpaper Site Script - Copyright AV Scripts 2014 </div>
</body>
</html>
I shared a note through evernote and a HTML page generated.
I wanna get title & content of this note ,so code below:
import re
resource = '<!DOCTYPE html>\n<!--[if lt IE 7 ]> <html class="ie6">; <![endif]--><!--[if IE 7 ]> <html class="ie7"> <![endif]--><!--[if IE 8 ]> <html class="ie8"> <![endif]--><!--[if IE 9 ]> <html class="ie9"> <![endif]--><!--[if gt IE 9]> <html> <![endif]--><!--[if !IE]><!--> <html> <!--<![endif]--><head><meta name="en:locale" content="en" />\n <meta charset="utf-8" />\n <meta http-equiv="X-UA-Compatible" content="IE=9,chrome=1" />\n <meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,minimum-scale=1,user-scalable=0" />\n\n <meta property="og:title" content="python re"/>\n <meta property="og:type" content="article"/>\n <meta property="og:description" content="a question about python re\n "/>\n <meta property="og:url" content="https://www.evernote.com/shard/s61/sh/396b4a1f-ae9c-40aa-b740-5aa19e301489/3de6deff539dec4772bdc4f1057a437d"/>\n <meta property="og:image"\n content="https://www.evernote.com/shard/s61/sh/396b4a1f-ae9c-40aa-b740-5aa19e301489/3de6deff539dec4772bdc4f1057a437d/thm/note/396b4a1f-ae9c-40aa-b740-5aa19e301489"/>\n <meta property="og:site_name" content="Evernote"/>\n <meta property="og:created_time" content="1350193749000"/>\n <meta property="og:updated_time" content="1350193786000"/>\n <link rel="Shortcut Icon" href="/favicon.ico" type="image/x-icon" />\n\n <link rel="stylesheet" href="/redesign/global/css/fonts.css" />\n <link rel="stylesheet" href="/redesign/global/css/header.css" />\n\n <link rel="stylesheet" href="/redesign/sharing/css/sharedNote.css" />\n <title>python re</title>\n <link rel="stylesheet" href="/redesign/modules/SharingMenu/SharingMenu.css"><link rel="stylesheet" href="/redesign/modules/LinkUrlDialog/LinkUrlDialog.css"></head><body class="wrapper"><div class="logo-bar">\n \n <a class="save-button save-button-desktop" href="/saveNote/s61/396b4a1f-ae9c-40aa-b740-5aa19e301489/3de6deff539dec4772bdc4f1057a437d">\n Save to Evernote</a>\n\n <div class="switch-account-div">\n <div class="switch-account-icon"></div>\n <span class="switch-account-name"></span>\n <div class="switch-account-arrow"></div>\n <div class="switch-account-dropdown">\n <div class="switch-dropdown-arrow"></div>\n <div class="switch-account-menuitem">\n Switch Account</div>\n <div class="switch-account-logout">\n Sign Out</div>\n </div>\n </div>\n\n </div>\n\n <div id="message-container">\n <div id="message">\n <div id="message-checkmark"></div>\n <span></span>\n </div>\n </div>\n\n <div id="container-boundingbox" class="wrapper">\n <div id="container" class="wrapper">\n <div class="sharing-imagegallery">\n <div class="SharingMenu"><div class="sharing-menu">\n <div class="share-button-container">\n <div class="label-container">\n <span class="label">\n Share</span>\n <div class="label-icon facebook-icon">\n </div>\n </div>\n <div class="icon-container"\n title="Share">\n <div class="icon">\n </div>\n </div>\n </div>\n <div class="menu-bar">\n <div class="menu-bar-div">\n <div class="menu-bar-icon facebook-icon"></div>\n <span class="menu-bar-label">\n Facebook</span>\n </div>\n <div class="menu-bar-div">\n <div class="menu-bar-icon twitter-icon"></div>\n <span class="menu-bar-label">\n Twitter</span>\n </div>\n <div class="menu-bar-div">\n <div class="menu-bar-icon linkedin-icon"></div>\n <span class="menu-bar-label">\n LinkedIn</span>\n </div>\n <div class="menu-bar-div">\n <div class="menu-bar-icon link-icon"></div>\n <span class="menu-bar-label">\n Link</span>\n </div>\n </div>\n </div>\n</div></div>\n <div class="shared-by-mobile">\n Shared by flowerszhong</div>\n <div class="shared-by shared-by-desktop">\n <div class="shared-by-left"></div>\n Shared by flowerszhong<div class="shared-by-right"></div>\n </div>\n <h2 class="note-title">python re</h2>\n <div class="vtop">\n <div class="note-updated">\n <span>\n Updated Today</span>\n </div>\n </div>\n <div class="divider"></div>\n <div class="note-content">\n <div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="ennote">\na question about python re\n<div><br/></div></div></div>\n <a class="save-button save-button-mobile" href="/saveNote/s61/396b4a1f-ae9c-40aa-b740-5aa19e301489/3de6deff539dec4772bdc4f1057a437d">\n Save to Evernote</a>\n <div class="clearfix" style="clear: both;"></div>\n</div>\n </div>\n\n\n <div class="footer">\n <div>\n Evernote makes it easy to remember things big and small from your everyday life using your computer, tablet, phone and the web.</div>\n <div class="footer-logo"></div>\n </div>\n\n <div class="LinkUrlDialog"><script id="linkUrlDialog" type="text/html">\n <div class="link-url-dialog">\n <div class="dialog-head">\n Link to Note</div>\n <div class="dialog-body">\n <p>Paste this link into an email or IM to share it.</p>\n <p>Anyone with the link will be able to view the note.</p>\n </div>\n <div class="url-container">\n <div class="url-title">\n Note URL:</div>\n <input type="text" class="url-input" value="{{url}}" readonly>\n <div class="copy-container">\n <button type="button" class="copy-button">\n Copy to Clipboard</button>\n </div>\n </div>\n </div>\n </script>\n</div><script src="/redesign/global/js/respond.min.js"></script>\n <script src="/redesign/global/js/require.min.js"></script>\n <script src="/redesign/global/js/config-require.js"></script>\n <script type="text/javascript">\n define("actionBean", [], function() {return {"shareNoteUri":"/shard/s61/sh/396b4a1f-ae9c-40aa-b740-5aa19e301489/3de6deff539dec4772bdc4f1057a437d?shareNote&service=","foodNote":false,"skitchNote":false,"userName":"","switchAccountUri":"/saveNote/s61/396b4a1f-ae9c-40aa-b740-5aa19e301489/3de6deff539dec4772bdc4f1057a437d?switch","logoutUri":"/saveNote/s61/396b4a1f-ae9c-40aa-b740-5aa19e301489/3de6deff539dec4772bdc4f1057a437d?logout","userStatus":"","images":false,"userLoggedIn":false};});\n </script>\n <!-- Google Analytics -->\n<script type="text/javascript">\nvar _gaq = _gaq || [];\n_gaq.push([\'_setAccount\', \'UA-285778-5\']);\n\n\n _gaq.push([\'_trackPageview\', \'/sh/{noteGuid}/{noteKey}/{suffix}\']);\n \n\n(function() {\n var ga = document.createElement(\'script\'); ga.type = \'text/javascript\'; ga.async = true;\n ga.src = (\'https:\' == document.location.protocol ? \'https://ssl\' : \'http://www\') + \'.google-analytics.com/ga.js\';\n var s = document.getElementsByTagName(\'script\')[0]; s.parentNode.insertBefore(ga, s);\n})();\n</script>\n<!-- End of Google Analytics -->\n<script type="text/javascript">\n var _gaq = _gaq || [];\n _gaq.push([\'_setCustomVar\',\n 4, // Slot 4 - required\n \'contentClass\', // Category - required\n \'\', // Value - required\n 3 // Page-level scope\n ]);\n\n _gaq.push([\'_setCustomVar\',\n 5, // Slot 5 - required\n \'sourceApplication\', // Category - required\n \'\', // Value - required\n 3 // Page-level scope\n ]);\n _gaq.push([\'_trackPageview\', \'/singleNote\']);\n </script>\n <script type="text/javascript" src="/redesign/modules/SharingMenu/SharingMenu.js"></script><script type="text/javascript" src="/redesign/modules/LinkUrlDialog/LinkUrlDialog.js"></script><script type="text/javascript" src="/redesign/sharing/SharedNoteViewAction/SharedNoteViewAction.js"></script></body></html>'
title_pattern = re.compile('(?<=<title>).+(?=</title>)')
content_pattern = re.compile('(?<=class=\"divider\"></div>).+(?=<a class=\"save-button)')
title= re.search(title_pattern,resource)
content = re.search(content_pattern,resource)
if title:
print title.group()
if content:
print content.group()
# if __name__=='__main__':main()
output:
python re
why only get title? and how to get the content of this note?
Your problem is that the content contains newlines. ., by default, doesn't match newlines.
Therefore, you should use re.DOTALL:
content_pattern = re.compile('(?<=class=\"divider\"></div>).+(?=<a class=\"save-button)', re.DOTALL)
to make . match newlines. Then it works.
I don't fully understand what you want to do but it seems like BeautifulSoup can help you out.