Currently, the below code returns all the posts from the profile:
profilename = 'oneplus'
loader = instaloader.Instaloader()
profile = Profile.from_username(loader.context,profilename)
posts = profile.get_posts()
If the profile has, say 3000 posts, it takes a lot of time to load for instaloader.
However, I just want most recent n posts, and n<10, how do I do that?
finally came across something handy!
Firstly:
pip install func_timeout
Imagine your function was:
def sum(x,y):
return x+y
Then use it like so:
from func_timeout import func_timeout, FunctionTimedOut
try:
seconds = 5 # times out after 5 secs
result=func_timeout(seconds,sum,args=(5,3))
except FunctionTimedOut:
print('timeout')
except Exception as e:
print(e)
For me, it works like a charm.
Note: if you are thinking of downloading the posts, make sure you also timeout control the download function, as the function scans all the comments, which can get pretty lengthy.
If in trouble, visit the website where I found it: https://pypi.org/project/func-timeout/
Cheers
Related
I am building a Flask webapp and it uses the OpenBabel API (chemistry toolkit) to generate some files for me. When I call this particular function it seems to work fine and generate the files in the directory that I want. However, once it gets back to Flask it crashes and Flask does not render the html template, instead redirecting me to This page isn’t working 127.0.0.1 didn’t send any data. I found that when removing the code in the function it works normally. So it's likely a problem with OpenBabel. The Openbabel function does not ouput any errors itself and seemingly even returns at the end, based on debugging.
I have tried many things from other SO answers, including changing my host to 0.0.0.0, adding threaded=True and some other solutions. All to no avail. I tried debugging it for a long time, but now I am lost because I have tried everything I know. All I could get was a SystemExit exception from Flask. Sometimes it was able to run the print statement following it, but more often it crashes immediately. I have no clue where the problem may lie. Appreciate any help I can get. A sample of the code (shortened it a bit):
#app.route("/", methods=["POST", "GET"])
def form_handler():
if request.method == "POST":
smiles = request.form["smiles_molecule"]
pdb_file = request.files["pdb_molecule"]
no_conformers = int(request.form["no_conformers"])
scoring = request.form["score"]
if smiles:
pattern = re.compile('[^A-Za-z0-9]+')
smiles_no_special_chars = re.sub(pattern, "", smiles)
mol_path = os.path.join(app.config["MOLECULE_UPLOADS"], smiles_no_special_chars[0:10])
if os.path.exists(mol_path):
shutil.rmtree(mol_path)
os.mkdir(mol_path)
os.chdir(mol_path)
x = conf_gen.gen_and_write_confs(smiles, no_conformers, scoring) #<- breaks down here
print(x)
return render_template("index.html", mole=smiles_no_special_chars[0:10])
The function that is called:
def gen_and_write_confs(molecule, no_confs, scoring):
"""Generate and write the conformers to PDB. Takes mol, number of conformers and
scoring method: RCS, MECS and ECS: OBRMSDConformerScore,
OBMinimizingEnergyConformerScore and OBEnergyConformerScore. See OpenBabel docs."""
mole = pybel.readstring("can", molecule)
mole.addh()
mole.make3D()
mole = mole.OBMol
mole.SetChainsPerceived(True)
cs_obj = ob.OBConformerSearch()
cs_obj.Setup(mole, no_confs, 5, 5, 25)
if scoring == "RCS":
score = ob.OBRMSDConformerScore()
elif scoring == "MECS":
score = ob.OBMinimizingEnergyConformerScore()
else:
score = ob.OBEnergyConformerScore()
cs_obj.SetScore(score)
cs_obj.Search()
cs_obj.GetConformers(mole)
mole.DeleteNonPolarHydrogens()
return "Test"
If needed I can upload the project on Github. It does require installing a few dependencies and I am using conda for that right now, but I could make it available through pip too since OpenBabel is available in pip.
Ps: no single error message is shown after it crashes
I have the algorithm below that performs likes and comments on instagram photos via URL, my goal is to run this script on an AWS machine within 5 minutes which will generate 288 comments per day. However, whenever he realizes that he already executes LIKE in the photo, he returns "LIKED 0 images | ALREADY LIKED: 1", and aborts the process. I wanted to create a deal for when LIKE already exists, just execute the comment, is it possible? If yes you can help me with provisioning examples or the solution in question, thank you!
from instapy import InstaPy
from instapy import smart_run
def connect(insta_username,insta_password):
try:
session = InstaPy(username=insta_username,
password=insta_password,
headless_browser=False)
return session
except Exception as e:
print("[FAILED] Caused by: {}".format(e))
def action(url,insta_username,insta_password):
session = connect(insta_username,insta_password)
try:
with smart_run(session):
# Always follow and like before, the algorithm understands how it has already been executed when used in place of the like function
# Define interaction settings
session.set_do_comment(enabled=True, percentage=100)
session.set_comments(["#user-id"])
session.set_user_interact(amount=1, randomize=False, percentage=100, media='Photo')
# Start the feature
session.interact_by_URL(urls=url, randomize=False, interact=True)
session.end()
except Exception as e:
print("[FAILED] Caused by: {}".format(e))
if __name__ == "__main__":
pics = ["https://www.instagram.com/p/id-here"]
for url in pics:
action(url,'username','password')
I have recently started to learn and work on Django i came across this error multiple times sometimes it was due to repetitive use of a name in an html and sometimes it was due the part for example "variable_name=request.POST['variable_name']"
I visited various sites regarding this error and was able to find a solution i.e, once i have typed in "variable_name=request.POST['variable_name']" in a function in views.py
the next time i tried to POST the data again "variable_name=request.POST['variable_name']" I had to do something like this "variable_name=request.POST.get('variable-name', 'blank_or_whatever_canBeFalseAlso')
The Above statement helped me solve that error I got this solution from A Thread in StackOverflow itself Quite a useful site.
Now for the Question am just Curious to know whats the difference between
Variable_Name=request.POST['Variable_Name']
Variable_Name=request.POST.get('Variable_Name','Whatever')
Code (views.py) Is given Below:
from django.shortcuts import render
from . import views
from datetime import date
from .models import Batch, Deleted_Batch
from django.utils.datastructures import MultiValueDictKeyError
# Create your views here.
today = date.today()
Date_Batch = str(today.day)+'/'+str(today.month)+'/'+str(today.year)
def Reg_Batch(request):
return render(request, 'Reg_Batch.html',{'Date_Batch':Date_Batch})
def Reg_Batch_Receive(request):
Region_Code=request.POST['Region_Code']
Branch_Code=request.POST['Branch_Code']
Batch_Number=request.POST['Batch_Number']
Farm_Code=request.POST['Farm_Code']
Farm_Status=request.POST['Farm_Status']
Farmer_Name=request.POST['Farmer_Name']
Farmer_Address=request.POST['Farmer_Address']
Farmer_Phone_Number=request.POST['Farmer_Phone_Number']
Farmer_Email=request.POST['Farmer_Email']
return render(request, 'Reg_Batch_Receive.html')
def Reg_Batch_View(request):
Region_Code=request.POST.get('Region_Code', 'Blank')
Branch_Code=request.POST.get('Branch_Code', 'Blank')
Farm_Code=request.POST.get('Farm_Code', 'Blank')
Farm_Status=request.POST.get('Farm_Status', 'Blank')
return render(request, 'Reg_Batch_View.html',{'Region_Code':Region_Code,'Branch_Code':Branch_Code,'Farm_Code':Farm_Code,'Farm_Status':Farm_Status})
def Reg_Batch_View_Receive(request):
Date_Batch= '20/08/2000'
Region_Code='586001'
Branch_Code='586001'
Batch_Number='586001'
Farm_Code='5484712'
Farm_Status='Active'
Farmer_Name='Joel Liao'
Farmer_Address='jojojojojojojo'
Farmer_Phone_Number='8088078086'
Farmer_Email='kuroneko#gmail.com'
return render(request,'Reg_Batch_View_Receive.html',{'Date_Batch':Date_Batch,'Region_Code':Region_Code,'Branch_Code':Branch_Code,'Batch_Number':Batch_Number,'Farm_Code':Farm_Code,'Farm_Status':Farm_Status,'Farmer_Name':Farmer_Name,'Farmer_Address':Farmer_Address,'Farmer_Phone_Number':Farmer_Phone_Number,'Farmer_Email':Farmer_Email})
def Reg_Batch_Delete(request):
Region_Code='586001'
Branch_Code='586001'
Batch_Number='586001'
Farm_Code='5484712'
Farm_Status='Active'
Farmer_Name='Joel Liao'
Farmer_Address='jojojojojojojo'
Farmer_Phone_Number='8088078086'
Farmer_Email='kuroneko#gmail.com'
Deleted_Batch_DataBase=Deleted_Batch({'Region_Code':Region_Code,'Branch_Code':Branch_Code,'Batch_Number':Batch_Number,'Farm_Code':Farm_Code,'Farm_Status':Farm_Status,'Farmer_Name':Farmer_Name,'Farmer_Address':Farmer_Address,'Farmer_Phone_Number':Farmer_Phone_Number,'Farmer_Email':Farmer_Email}),
return render(request,'Reg_Batch_Delete.html',{'Batch_Number':Batch_Number})
def Reg_Batch_Delete_Receive(request):
Batch_Number='586001'
return render(request,'Reg_Batch_Delete_Receive.html',{'Batch_Number':Batch_Number})
request.POST is a dictionary-like object (see docs).
So if you do
request.POST['foobar']
you can access the posted variable foobar. But if this is not being posted and therefore not set inside POST, you will get the MultiValueDictKeyError.
This is similar to access a key inside a default dictionary which is not set, e.g.
d = {}
d['foobar'] # raises Exception
In order to avoid this error, you can then do
request.POST.get('foobar', 'default-value')
in order to get the posted field and a default if it is not present.
So if foobar isn't set inside your POST-data, the received value is default-value.
This is similar to the default dictionary handling:
d = {}
d.get('foobar', 'random') # returns 'random'
I have problem because I can not find the reason why my function in Django views.py sometimes runs two times. When I go to url, which call function create_db in Django view, function read json files from directory, parse it and write the data in the database. Most of the time it works perfectly, but sometimes for no reason it runs two times and write the same data in the data base. Does anyone know what can be the reason why code is sometimes done twice and how can I solve the problem?
Here is my create_db function:
def create_db(request):
response_data = {}
try:
start = time.time()
files = os.listdir()
print(files)
for filename in files:
if filename.endswith('.json'):
print(filename)
with open(f'{filename.strip()}', encoding='utf-8') as f:
data = json.load(f)
for item in data["CVE_Items"]:
import_item(item)
response_data['result'] = 'Success'
response_data['message'] = 'Baza podatkov je ustvarjena.'
except KeyError:
response_data['result'] = 'Error'
response_data['message'] = 'Prislo je do napake! Podatki niso bili uvozeni!'
return HttpResponse(json.dumps(response_data), content_type='application/json')
The console output that I expect:
['nvdcve-1.0-2002.json', 'nvdcve-1.0-2003.json', 'nvdcve-1.0-2004.json', 'nvdcve-1.0-2005.json', 'nvdcve-1.0-2006.json', 'nvdcve-1.0-2007.json', 'nvdcve-1.0-2008.json', 'nvdcve-1.0-2009.json', 'nvdcve-1.0-2010.json', 'nvdcve-1.0-2011.json', 'nvdcve-1.0-2012.json', 'nvdcve-1.0-2013.json', 'nvdcve-1.0-2014.json', 'nvdcve-1.0-2015.json', 'nvdcve-1.0-2016.json', 'nvdcve-1.0-2017.json']
nvdcve-1.0-2002.json
nvdcve-1.0-2003.json
nvdcve-1.0-2004.json
nvdcve-1.0-2005.json
nvdcve-1.0-2006.json
nvdcve-1.0-2007.json
nvdcve-1.0-2008.json
nvdcve-1.0-2009.json
nvdcve-1.0-2010.json
nvdcve-1.0-2011.json
nvdcve-1.0-2012.json
nvdcve-1.0-2013.json
nvdcve-1.0-2014.json
nvdcve-1.0-2015.json
nvdcve-1.0-2016.json
nvdcve-1.0-2017.json
Console output when error happened:
['nvdcve-1.0-2002.json', 'nvdcve-1.0-2003.json', 'nvdcve-1.0-2004.json', 'nvdcve-1.0-2005.json', 'nvdcve-1.0-2006.json', 'nvdcve-1.0-2007.json', 'nvdcve-1.0-2008.json', 'nvdcve-1.0-2009.json', 'nvdcve-1.0-2010.json', 'nvdcve-1.0-2011.json', 'nvdcve-1.0-2012.json', 'nvdcve-1.0-2013.json', 'nvdcve-1.0-2014.json', 'nvdcve-1.0-2015.json', 'nvdcve-1.0-2016.json', 'nvdcve-1.0-2017.json']
nvdcve-1.0-2002.json
['nvdcve-1.0-2002.json', 'nvdcve-1.0-2003.json', 'nvdcve-1.0-2004.json', 'nvdcve-1.0-2005.json', 'nvdcve-1.0-2006.json', 'nvdcve-1.0-2007.json', 'nvdcve-1.0-2008.json', 'nvdcve-1.0-2009.json', 'nvdcve-1.0-2010.json', 'nvdcve-1.0-2011.json', 'nvdcve-1.0-2012.json', 'nvdcve-1.0-2013.json', 'nvdcve-1.0-2014.json', 'nvdcve-1.0-2015.json', 'nvdcve-1.0-2016.json', 'nvdcve-1.0-2017.json']
nvdcve-1.0-2002.json
nvdcve-1.0-2003.json
nvdcve-1.0-2003.json
nvdcve-1.0-2004.json
nvdcve-1.0-2004.json
nvdcve-1.0-2005.json
nvdcve-1.0-2005.json
nvdcve-1.0-2006.json
nvdcve-1.0-2006.json
nvdcve-1.0-2007.json
nvdcve-1.0-2007.json
nvdcve-1.0-2008.json
nvdcve-1.0-2008.json
nvdcve-1.0-2009.json
nvdcve-1.0-2009.json
nvdcve-1.0-2010.json
nvdcve-1.0-2010.json
nvdcve-1.0-2011.json
nvdcve-1.0-2011.json
nvdcve-1.0-2012.json
nvdcve-1.0-2012.json
nvdcve-1.0-2013.json
nvdcve-1.0-2013.json
nvdcve-1.0-2014.json
nvdcve-1.0-2014.json
nvdcve-1.0-2015.json
nvdcve-1.0-2015.json
nvdcve-1.0-2016.json
nvdcve-1.0-2016.json
nvdcve-1.0-2017.json
nvdcve-1.0-2017.json
The problem is not in the code which you show us. Enable logging for the HTTP requests which your application receives to make sure the browser sends you just a single request. If you see two requests, make sure they use the same session (maybe another user is clicking at the same time).
If it's from the same user, maybe you're clicking the button twice. Could be a hardware problem with the mouse. To prevent this, use JavaScript to disable the button after the first click.
I am not a python geek but have tried to solve this problem using information from several answers to similar questions but none seem to really work in my case. Here it is:
I am calling a function from a python script:
Here is the function:
def getsom(X):
#some codes
try:
st = get data from site 1 using X
except:
print "not available from site 1, getting from site 2"
st = get data from site 2 using X
#some codes that depend on st
I am calling this from a python script as such:
#some codes
for yr in range(min_yr,max_yr+1):
day=1
while day<max_day:
st1 = getsom(X)
#some code that depends on st1
day+=1
This works fine when data is available on either site 1 or 2 for a particular day, but breaks down when it is unavailable on both sites for another day.
I want to be able to check for the next day if data is unavailable for a particular day for both sites. I have tried different configurations of try and except with no success and would appreciate any help on the most efficient way to do this.
Thanks!
***Edits
Final version that worked:
in the function part:
def getsom(X):
#some codes
try:
st = get data from site 1 using X
except:
print "not available from site 1, getting from site 2"
st = get data from site 2 using X
try:
st = get data from site 2 using X
except:
print "data not available from sites 1 and 2"
st=None
if st is not None:
#some codes that depend on st
In order to iterate to the next day on the script side, I had to handle the none case from the function with another try/except block:
#some codes
for yr in range(min_yr,max_yr+1):
day=1
while day<max_day:
try:
st=getsom(X)
except:
st=None
if st is not None:
#some codes that depend
As mentioned in the comments you seem like you want to catch the exception in first-level exception handler. You can do it like this:
def getsom(X):
#some codes
try:
st = get data from site 1 using X
except:
print "not available from site 1, getting from site 2"
try:
st = get data from site 2 using X
except:
print "Not available from site 2 as well."
# Here you can assign some arbitrary value to your variable (like None for example) or return from function.
#some codes that depend on st
If data is not available on neither of the sites you can assign some arbitrary value to your variable st or simply return from the function.
Is this what you are looking for?
Also, you shouldn't simply write except without specifying the type of exception you expect - look here for more info: Should I always specify an exception type in `except` statements?
Edit to answer the problem in comment:
If you have no data about certain day you can just return None and handle it like this:
#some codes
for yr in range(min_yr,max_yr+1):
day=1
while day<max_day:
st1 = getsom(X)
if st1 is not None:
#some code that depends on st1
day+=1
Why don't you create a separate function for it?
def getdata(X):
for site in [site1, site2]: # possibly more
try:
return get_data_from_site_using_X()
except:
print "not available in %s" % site
print "couldn't find data anywhere"
Then getsom becomes:
def getsom(X):
#some codes
st = getdata(X)
#some codes that depend on st