I am using IronPython 2.7.9.0 on Grasshopper and Rhino to web scrape data from a specific widget on this link: https://vemcount.app/embed/widget/uOCRuLPangWo5fT?locale=en
The code I am using is as follows
import urllib
import os
web = urllib.urlopen(url)
html = web.read()
web.close()
The html output contains all the html code from this link except for the parts I need. When I inspect it on chrome it has a "flex" button next to it such as the following image.
image that summarizes the issue I am facing
Anything that is rooted under the line with a "flex" button does not appear in the scraping result and comes as a blank line.
This is the output html I get:
<!DOCTYPE html>
<html lang="en">
<head>
<title>Central Library - Duhig North & Link</title>
<meta charset="utf-8">
<meta name="google" content="notranslate">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<meta name="csrf-token" content="">
<link rel="stylesheet" href="/build/app.css?id=2fefc4f9faa59eebcb4b">
<link rel="stylesheet" href="https://vemcount.app/fonts/hamburg_serial/stylesheet.css">
<style>
#embed, #main {
height: 100vh;
}
.vue-grid-item {
margin-bottom: 0px !important;
}
.powered_by {
position: absolute;
bottom: 0px;
right: 0px;
background-color: rgba(0, 0, 0, 0.18);
color: #fff;
padding: 2px 5px;
font-size: 9px;
}
.powered_by:hover, .powered_by:link, .powered_by:visited {
text-decoration: none;
display: none;
}
.dashboard-widget .relative {
overflow: hidden !important;
}
</style>
<script>
window.App = {"socketAppKey":"eJSkWUHWpwolvjVcT2ZxUJZXnDpxtRljdZl74fKr","socketCluster":null,"socketHost":"websocket.vemcount.com","socketPort":443,"socketSecurePort":443,"socketDisableStats":true,"socketEncrypted":true,"locale":"en","settings":[{"name":"type","value":"{\"count_in\":\"column\"}"},{"name":"period","value":"[\"yesterday\"]"},{"name":"period_step","value":"hour"},{"name":"hide_datalabel","value":"0"},{"name":"currency","value":"AUD"},{"name":"show_days","value":"[0,1,2,3,4,5,6]"},{"name":"show_months","value":"[1,2,3,4,5,6,7,8,9,10,11,12]"},{"name":"show_hours_from","value":"00:00"},{"name":"show_hours_to","value":"23:45"},{"name":"data_heatmap","value":"blue"},{"name":"weather_metrics","value":"0"},{"name":"first_day_of_week","value":"1"},{"name":"time_format24","value":"time_format24"},{"name":"date_time_format","value":"2"},{"name":"number_grouping","value":","},{"name":"number_decimal","value":"."},{"name":"opening_hours_overlap","value":"0"},{"name":"data_output","value":"count_in"}],"sound":null};
</script>
<script src="/build/lang/en.js?v=2022.04.4"></script>
</head>
<body class="bg-transparent">
<main id="main">
<div id="embed" >
<div class="w-full h-full vue-grid-item cssTransforms" style="position: absolute;">
<live-inside :embedded="true" :widget="{"id":81438,"pane_id":4005,"title":"Central Library - Duhig North & Link","description":"Live occupancy \/ Seating capacity","x":0,"y":0,"w":2,"h":1,"bg_color":"red","text_color":"black","type":"live-inside","secret":"uOCRuLPangWo5fT","internal":"VRg4JTIRrtJ7Pwg","embeddable":1,"content":{"target":1100,"bidirectional":true,"target_enable":true,"prettify":false,"target_type":"donut","target_donut_hide_metric":false,"target_donut_target_hide_label":false,"target_visual_inside_text":null,"target_visual_available_text":null,"target_screen_ok_title":null,"target_screen_ok_text":null,"target_screen_ok_color":"#38A169","target_screen_ok_image":-1,"target_screen_warning_title":null,"target_screen_warning_pe</live-inside>
</div>
</div>
</main>
<a title=" Vemco Group A/S " class="powered_by" target="_blank"
href="http://vemcount.com">Powered by
<b>vemcount.com</b>
</a>
<script src="/build/manifest.js?id=7f2e9aa3431c681a4683"></script>
<script src="/build/vendor.js?id=19867aae3b960cda7d79"></script>
<script src="/build/embed.js?id=2ff0173dd78c5c1f99c6"></script>
</body>
</html>
As you can see it is missing some lines, which are the lines that have a flex button next to them. (btw I have shortended the code that is in so I dont reach the 30000 character limit).
I am interested in the number 311 which changes every 2 seconds in the live link and it can be found in the html code between
<span>311</span>
Is there a way I can get this value, as well as any other value, using IronPython?
P.S. I am a noob in actual coding, that's why I might have issues with terminologies, but have a fair background in visual scripting. Your help is much appreciated. Thanks.
Just in case you had the same query or were struggling with dynamic web scraping. You have to use CPython and install a webscraper such as Playwright or BS + Selenium
I used playwright which is far more straightforward and has a very much appreciated inner_html() function which reads straight into the dynamic flex HTML code. Here is the code for reference.
#part of the help to write the script I got from https://stackoverflow.com/questions/64303326/using-playwright-for-python-how-do-i-select-or-find-an-element
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(slow_mo=1000)
page = browser.new_page()
page.goto('https://vemcount.app/embed/widget/uOCRuLPangWo5fT')
central = page.query_selector("p.w-full span");
print({'central': central.inner_html()})
browser.close()
Afterwards I am trying to run the .py script remotely from Grasshopper through a batch file and read the output through a txt or CSV file from within Grasshopper.
If there is a better way I am more than happy to hear your suggestions.
Yours,
A Beginner in Python. :)
I want to change the font size of a specific markdown cell in jupyter lab, and not the whole output. I will convert my project at the end into an html file.
I already tried this:
<html>
<head>
<style>
div.a {
font-size: 300%;
}
</style>
<body>
<div class="a">My text in here</div>
</body>
</head>
</html>
But this is not changing my text size after I run my cell under Markdown.
I also don't want to use in order to not give a automatic number to that particular text.
Thanks in advance
I see you want to change the font-size:
<html>
<head>
<style type="text/css">
.a{ //for class use .(class-name)
font-size: 300%;
}
</style>
</head> // all the styling is to be done inside the head tag.
<body>
<div class="a">My text in here</div>
</body>
</html>
there is a problem about html5 webcam
this is the error i have
Uncaught TypeError: Failed to execute 'createObjectURL' on 'URL': No function was found that matched the signature provided.
at photo.js:17
photo.js :17
video.src=vendorUrl.createObjectURL(stream);
please check my code
thank you so much!
takeing_photo.html
{% load staticfiles %}
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Document</title>
<link href="{% static 'css/photo.css' %}" rel="stylesheet">
</head>
<body>
<div class="booth">
<video id="video" width="400" height="300"></video>
Take photo
<canvas id="canvas" width="400" height="300"></canvas>
<img id="photo" src="http://placekitten.com/g/400/300" alt="photo of you">
</div>
<script src="{% static 'js/photo.js' %}"></script>
</body>
</html>
photo.js
(function(){
var video = document.getElementById('video'),
photo = document.getElementById('photo'),
context = canvas.getContext('2d'),
phto = document.getElementById('photo');
vendorUrl = window.URL || window.webkitURL;
navigator.getMedia = navigator.getUserMedia ||
navigator.webkitGetUserMedia ||
navigator.mozGetUserMedia ||
navigator.msGetUserMedia;
navigator.getMedia({
video:true,
audio:false
}, function(stream){
video.src=vendorUrl.createObjectURL(stream);
video.play();
}, function(error){
});
document.getElementById('capture').addEventListener('click', function(){
context.drawImage(video, 0, 0, 400, 300);
photo.setAttribute('src', canvas.toDataURL('image/png'))
});
})();
photo.css
.booth{
width:400px;
background-color: #ccc;
border:10px solid #ddd;
margin:0 auto;
}
.booth-capture-button {
display:block;
margin:10px 0;
padding:10px 20px;
background-color: cornflowerblue;
color: #fff;
text-align: center;
text-decoration: none;
}
#canvas {
display :none;
}
i just want to make webcam properly
and i m wondering there is a way to save the pics into the folder when i put the button "take of you"
please give me advice. thank you so much.
This error is caused because the function createObjectURL is deprecated. You need to update your code to set srcObject to the video object directly.
video.srcObject=stream;
I have tried using #fontface css style, but the font doesn't get rendered.
Is there another way of doing this using python/flask??
<!DOCTYPE html>
<html>
<style type="text/css">
#font-face {
font-family: trial;
src: url_for('CENTRALESANSCND-BOLD.otf') ;
font-weight:bold; }
</style>
<body>
<p style="font-family:trial; font-weight: bold"> Hello </p>
</body>
</html>
The above is my HTML template.
Unfortunately, when I render it using Flask, it doesn't reflect the font.
The following is my .py code:
from flask import Flask, render_template
app=Flask(__name__)
app.secret_key='1234'
#app.route('/', methods=['post', 'get'])
def output():
c=2+3
print c
return render_template('f.html', c=c)
if __name__=='__main__':
app.run(port=9876)
Thanks a lot for all your help.. A combination of all the suggestions finally worked.
Posting the solution here:
CSS file:
#font-face{
font-family: trial;
src: url('CENTRALESANSCND-BOLD.otf');
font-weight: bold;}
body {font-family: georgia; color:green}
h1 {font-family:trial; color: pink}
HTML file:
<!DOCTYPE html>
<html>
<link type="text/css" rel="stylesheet" href="{{url_for('static', filename='mystyle.css')}}"/>
<body>
<p > Hello... </p>
<h1> welcome </h1>
</body>
</html>
url_for takes a function name, and you need to wrap it in double curly brackets... try:
<style type="text/css">
#font-face {
font-family: trial;
src: {{ url_for('static', filename='CENTRALESANSCND-BOLD.otf') }};
font-weight:bold; }
</style>
<style>
#font-face {
font-family: FontName;
src: url("{{ url_for('static', filename='FontFile.otf') }}");
}
</style>
i need to know how to redirect the user in Django views to a certain page after he logs in.
let's say we have 3 types of users and 3 types of pages, i want each type to be directed to a certain page, and in the same time doesn't has the permission to view the other pages.
You can do something like this:
from django.http import HttpResponseRedirect
from django.core.urlresolvers import reverse
from django.contrib.auth.decorators import login_required
#login_required
def home(request):
return HttpResponseRedirect(
reverse(custom_view,
args=[request.user.username]))
Here, custom_view should be your user specific view. This is assuming you have:
LOGIN_REDIRECT_URL = '/profiles/home'
and you have configured a urlpattern:
(r'^profiles/home', 'myapp.views.home')
You can add a check for account type and redirect to the correct view.
I wrote some similar functionality recently by sub-classing the LoginView provided by django.contrib.auth. Okay, make your first page from the root directory, call it login:
python manage.py startapp login. Make this file exist <projectRoot>/login/templates/registration/login.html Add this code inside said file, it's more or less cut and pasted from bootstrap's login form, but with some standard django template language to bind to the expected AuthenticationForm fields.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
<meta name="description" content="">
<meta name="author" content="">
<link rel="icon" href="../../favicon.ico">
<title>Signin to yourWebsite.com</title>
<script src="https://code.jquery.com/jquery-3.2.1.slim.min.js" integrity="sha384-KJ3o2DKtIkvYIK3UENzmM7KCkRr/rE9/Qpg6aAZGJwFDMVNA/GpGFF93hXpG5KkN" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.12.9/umd/popper.min.js" integrity="sha384-ApNbgh9B+Y1QKtv3Rn7W3mgPxhU9K/ScQsAP7hUibX39j7fakFPskvXusvfa0b4Q" crossorigin="anonymous"></script>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/css/bootstrap.min.css" integrity="sha384-Gn5384xqQ1aoWXA+058RXPxPg6fy4IWvTNh0E263XmFcJlSAwiGgFAW/dAiS6JXm" crossorigin="anonymous">
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.0.0/js/bootstrap.min.js" integrity="sha384-JZR6Spejh4U02d8jOt6vLEHfe/JQGiRRSQQxSfFWpi1MquVdAyjUar5+76PVCmYl" crossorigin="anonymous"></script>
</head>
<body class="text-center">
<form class="form-signin" method="post" action="{% url 'login' %}?next=clockin">
{% csrf_token %}
<img class="mb-4" src="https://getbootstrap.com/assets/brand/bootstrap-solid.svg" alt="" width="72" height="72">
<h1 class="h3 mb-3 font-weight-normal">Please sign in</h1>
<label for="inputEmail" class="sr-only">Email address</label>
<!--input id="inputEmail" class="form-control" placeholder="Email address" required="True" autofocus="" type="email"-->
{{form.username}}
<label for="id_password" class="sr-only">Password</label>
{{form.password}}
<!--input id="inputPassword" class="form-control" placeholder="Password" required="True" type="password"-->
<div class="checkbox mb-3">
<label>
<input value="remember-me" type="checkbox"> Remember me
</label>
</div>
<button class="btn btn-lg btn-primary btn-block" type="submit">Sign in</button>
<p class="mt-5 mb-3 text-muted">© 2018</p>
</form>
</body>
</html>
<script>
$("#id_password").attr({
"class":"form-control",
"placeholder":"Password",
"required":"True",
"type":"password",
});
$("#id_username").attr({"class":"form-control",
"placeholder":"Email address",
"required":"True",
"type":"email",
});
</script>
<style>
html,
body {
height: 100%;
}
body {
display: -ms-flexbox;
display: -webkit-box;
display: flex;
-ms-flex-align: center;
-ms-flex-pack: center;
-webkit-box-align: center;
align-items: center;
-webkit-box-pack: center;
justify-content: center;
padding-top: 40px;
padding-bottom: 40px;
background-color: #f5f5f5;
}
.form-signin {
width: 100%;
max-width: 330px;
padding: 15px;
margin: 0 auto;
}
.form-signin .checkbox {
font-weight: 400;
}
.form-signin .form-control {
position: relative;
box-sizing: border-box;
height: auto;
padding: 10px;
font-size: 16px;
}
.form-signin .form-control:focus {
z-index: 2;
}
.form-signin input[type="email"] {
margin-bottom: -1px;
border-bottom-right-radius: 0;
border-bottom-left-radius: 0;
}
.form-signin input[type="password"] {
margin-bottom: 10px;
border-top-left-radius: 0;
border-top-right-radius: 0;
}
</style>
Next, subclass the built in view and override the redirect part. Inside login.views.py, add this:
from django.contrib.auth.views import LoginView
class CustomLoginview(LoginView):
def get_redirect_url(self):
if self.request.user.groups.filter(name="customer").exists():
return 'invoice'
return super().get_redirect_url()
Finally, update urls.py:
from login.views import CustomLoginview
urlpatterns = [
path('', CustomLoginview.as_view(), name='login'),
I'm telling the login page to go to my next app 'invoice' if the user is a customer, otherwise it goes to the default that I specified in the settings file. Obviously you could expound upon the concept for 3 types of users, which is different than routing based on the names of users, not sure how that got 2 upvotes.