My scrap script that works pretty well on Windows 10, so i am trying to use a VPS (from contabo) to make file.py running 24/7. My Server is a Ubuntu 22.04.1 without a GUI, just the CLI, python 3.10.6 and playwright 1.27.1 in both machines.
Tried: xvfb-run with --auto-servernum --server-num=1 --server-args='-screen 0, 1920x1080x24' and without it, also tried import os ;os.environ['DISPLAY'] = ':1'.
Minimum Reproducible:
from playwright.async_api import async_playwright
import asyncio
async def run(browser):
context = await browser.new_context(viewport={'width':1500, 'height':1000})
page = await context.new_page()
await page.goto("https://www.pixbetrei.com/", timeout=60000)
await page.get_by_role("button", name="Entrar").click(timeout=60000)
await context.close()
async def start():
async with async_playwright() as playwright:
browser = await playwright.firefox.launch(headless = True)
tasks = [run(browser)]
await asyncio.gather(*tasks)
await browser.close()
asyncio.run(start())
Traceback:
playwright._impl._api_types.TimeoutError: Timeout 60000ms exceeded.
=========================== logs ===========================
waiting for selector "role=button[name="Entrar"i]"
============================================================
Related
I'm making a discord bot for playing music (ik very unoriginal) and everytime I try to play music, it gives this error:
2023-02-18 12:31:54 INFO discord.player ffmpeg process 4024 has not terminated. Waiting to terminate...
2023-02-18 12:31:54 INFO discord.player ffmpeg process 4024 should have terminated with a return code of -9.
It is in the voice chat and didn't play anything. Here's my code if you need it:
print("defining functions")
import discord, pytube, os, threading
from moviepy import editor
from discord.ext.commands import Bot
intents=discord.Intents.default()
intents.message_content = True
intents.voice_states = True
bot = Bot("b!",intents=intents)
check = False
token = "redacted"
#bot.command()
async def play(ctx, arg):
await ctx.send("Downloading...")
try:
yt = pytube.YouTube(arg)
print("STREAM: "+str(yt.streams.filter(only_audio=True,mime_type="audio/webm",type="audio",abr="160kbps")))
def download(yt):
print("DOWNLOADING STREAM TO AUDIO.WEBM")
yt.streams.filter(only_audio=True,mime_type="audio/webm",type="audio",abr="160kbps").order_by("abr").first().download(filename="audio.webm")
print("EXPORTING TO MP3")
editor.AudioFileClip("audio.webm").write_audiofile("audio.webm"[:-5] + ".mp3")
os.remove("audio.webm")
print("DONE")
thread = threading.Thread(target=download, args=(yt,))
thread.start()
thread.join()
try:
channel = ctx.author.voice.channel
try:
vc = await channel.connect()
except Exception:
ctx.send("already in vc, one sec")
except Exception:
pass
vc.play(discord.FFmpegPCMAudio(source="audio.mp3", executable="./ffmpeg"))
except Exception:
pass
bot.run(token)
It worked on my computer in VS Code, but I wanted to test it on Replit and only there I got the error. I tried:
Reinstalling discord, pynacl, even pytube and moviepy.
Using a local copy of ffmpeg on replit, see line 36.
Adding the intents that were described in a different thread.
One thing that always happens is the exit code being -9 but the process number changing (ofc). I saw a GitHub Issue about this too which is not showing a fix so how do I fix that?
Good day everyone. I ran this code and it works perfectly well.the main purpose is to capture websocket traffic and the problem is that it closes after a minute or there about.. please how can I fix this.. I want it to stay alive forever
import asyncio
from pyppeteer import launch
async def main():
browser = await launch(
headless=True,
args=['--no-sandbox'],
autoClose=False
)
page = await browser.newPage()
await page.goto('https://www.tradingview.com/symbols/BTCUSD/')
cdp = await page.target.createCDPSession()
await cdp.send('Network.enable')
await cdp.send('Page.enable')
def printResponse(response):
print(response)
cdp.on('Network.webSocketFrameReceived', printResponse) # Calls printResponse when a websocket is received
cdp.on('Network.webSocketFrameSent', printResponse) # Calls printResponse when a websocket is sent
await asyncio.sleep(100)
asyncio.get_event_loop().run_until_complete(main())
I have this code and I want to run it on google colab. It works great on my PC but on colab I always get errors like these:
SyntaxError: 'async with' outside async function
or
RuntimeError: You must use "async with" if the event loop is running (i.e. you are inside an "async def")
sometimes it does not wait for getting new messages and finish after one running.
import json
import time
import telethon as tlt
import asyncio
from telethon import events,TelegramClient
chat_name = "sample"
telegram_session="sample_1"
api_id = "0000000"
api_hash = ""
client = TelegramClient(None , api_id, api_hash)
#client.on(events.NewMessage(chats=chat_name))
async def handler(event):
get_message = event.message.to_dict()
get_message['date'] = get_message['date'].strftime("%Y-%m-%d %H:%M:%S")
message_json = json.dumps(get_message)
print(message_json)
async with client:
client.run_until_disconnected()
You need to put async with inside of async def:
...
async def main():
async with client:
await client.run_until_disconnected()
client.loop.run_until_complete(main())
So I wrote a Discord bot in Python and it worked. But, when I close my Python shell (where I run my code) the bot stops working. Is there anything I can do so my bot keeps working no matter if the shell is closed or not? Just in case if you need the code, here it is
import os
import random
import datetime
import discord
bot = discord.Client()
#bot.event
async def on_member_join(member):
if member.id == bot.id:
return
channel = discord.utils.get(bot.guilds[0].channels,name="genral")
member_id = member.id
response = f"Welcome to For Testing,<#{member_id}> !"
await channel.send(response)
#bot.event
async def on_message(message):
if message.author == bot.user:
return
Keywords = ["sup","lol","yeet"]
channel = message.channel
for keyword in Keywords:
if keyword.lower() in message.content.lower():
user_id = message.author.id
response = f"Did someone say {keyword.lower()}? I guess it was <#{user_id}>"
await channel.send(response)
#bot.event
async def reminder():
while(True):
await bot.wait_until_ready()
online_members = []
for member in bot.get_all_members():
online_members.append(member.id)
if len(online_members) > 0:
user = random.choice(online_members)
channel = discord.utils.get(bot.guilds[0].channels,name="genral")
response = f"So what are u up to <#{user.id}>"
await channel.send(response)
await asyncio.sleep(3600)
bot.loop.create_task(reminder())
real_token = "THE TOKEN"
bot.run("THE TOKEN")
You can save your script as a file with the file extension .py and then run it through your file manager (e.g. double-click the file in Windows). That should run your script without needing the shell open.
https://realpython.com/run-python-scripts/#how-to-run-python-scripts-from-a-file-manager
On Windows, you can use pythonw.exe in order to run a python script as a background process:
Python scripts (files with the extension .py) will be executed by python.exe by default. This executable opens a terminal, which stays open even if the program uses a GUI. If you do not want this to happen, use the extension .pyw which will cause the script to be executed by pythonw.exe by default (both executables are located in the top-level of your Python installation directory). This suppresses the terminal window on startup.
For example,
C:\ThanosDodd\Python3.6\pythonw.exe C:\\Python\Scripts\moveDLs.py
or
pythonw.exe pythonFile.py
You can export code onto repl.it and make one more file called keep_alive.py
from flask import Flask
from threading import Thread
app = Flask('')
#app.route('/')
def main():
return "Your bot is alive!"
def run():
app.run(host="0.0.0.0", port=<any Port>)
def keep_alive():
server = Thread(target=run)
server.start()
then do
from keep_alive import keep_alive
then call the keep_alive function before bot.run
this makes a flask server then u can take the server link and make a monitor in http://uptimerobot.com/
For more info check the link
https://repl.it/talk/learn/Hosting-discordpy-bots-with-replit/11008
I've got two issues that I can't solve it at them moment.
1. I would like to keep the browser running so I could just re-connect using pyppeteer.launcher.connect() function but it seems to be closed imidiately even if I don't call pyppeteer.browser.Browser.close().
test01.py:
import asyncio
from pyppeteer import launch, connect
async def fetch():
browser = await launch(
headless=False,
args=['--no-sandbox']
)
print(f'Endpoint: {browser.wsEndpoint}')
await browser.disconnect()
loop = asyncio.get_event_loop()
loop.run_until_complete(fetch())
$ python test01.py
Endpoint: ws://127.0.0.1:51757/devtools/browser/00e917a9-c031-499a-a8ee-ca4090ebd3fe
$ curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" http://127.0.0.1:51757
curl: (7) Failed to connect to 127.0.0.1 port 51757: Connection refused
2. How do I create CDP session. This code should open another browser window but it doesn't work as expected:
test02.py
import asyncio
import time
from pyppeteer import launch, connect
async def fetch():
browser = await launch(
headless=False,
args=['--no-sandbox']
)
page = await browser.newPage()
cdp = await page.target.createCDPSession()
await cdp.send('Target.createBrowserContext')
time.sleep(5)
await browser.disconnect()
loop = asyncio.get_event_loop()
loop.run_until_complete(fetch())
$ python test02.py
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.',)>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.
How to keep the browser running
You just need to use autoClose flag, here's the docs:
autoClose (bool): Automatically close browser process when script
completed. Defaults to True.
In this case you test01.py would look as follows:
import asyncio
from pyppeteer import launch, connect
async def fetch():
browser = await launch(
headless=False,
args=['--no-sandbox'],
autoClose=False
)
print(f'Endpoint: {browser.wsEndpoint}')
await browser.disconnect()
loop = asyncio.get_event_loop()
loop.run_until_complete(fetch())
CDP session
Here it is:
import asyncio
import time
from pprint import pprint
from pyppeteer import launch, connect
from pyppeteer.browser import BrowserContext
async def fetch():
browser = await launch(
headless=False,
args=['--no-sandbox'],
autoClose=False
)
page = await browser.newPage()
cdp = await page.target.createCDPSession()
raw_context = await cdp.send('Target.createBrowserContext')
pprint(raw_context)
context = BrowserContext(browser, raw_context['browserContextId'])
new_page = await context.newPage()
await cdp.detach()
await browser.disconnect()
loop = asyncio.get_event_loop()
loop.run_until_complete(fetch())
Inspired by Browser.createIncognitoBrowserContext from pyppeteer itself.
Notice creating additional sessions via CDP doesn't seem to be such a great idea because browser._contexts won't be updated and will become inconsistent. It's also likely that Browser.createIncognitoBrowserContext might fit your needs without resorting to CDP whatsoever