Django no module named elasticsearch_dsl.connections - python

I'm trying to connect my Django model to the Elasticsearch server on local host but when I try
from elasticsearch_dsl.connections import connections
I get the error "ImportError: No module named elasticsearch_dsl.connections".
When I use this same command in the Django shell, it works fine.
search.py
from elasticsearch_dsl.connections import connections
from elasticsearch_dsl import DocType, Text, Date, Boolean, Integer, Keyword, fields
from elasticsearch.helpers import bulk
from elasticsearch import Elasticsearch
from .models import HomeGym, Country, Rating
connections.create_connection()
class HomeGymIndex(DocType):
title = Text()
price = fields.FloatField()
tags = Keyword()
city = Text()
country = Text()
rate = Integer()
opusApproved = Boolean()
def bulk_indexing():
HomeGymIndex.init()
es = Elasticsearch()
bulk(client=es, actions=(b.indexing() for b in HomeGym.objects.all().iterator()))
This leads to an ImportError on line 1. "No module named elasticsearch_dsl.connections"
The same import statement works in the shell though.
I've already done a pip install of elasticsearch and elasticsearch-dsl inside my virtualenv.
Here is the file structure
my_website/
elasticsearch/
#elasticsearch files pulled from github
elasticsearch-5.5.2-SNAPSHOT/
#elasticsearch files
bin/
elasticsearch
opus/
manage.py
homegymlistings/
models.py
search.py
#other standard app files
opus/
#standard files for main django branch
my_virtualenv/
bin/
activate
Why does my import statement only fail when called inside the search.py file located inside the homegymlistings app?

Run this
pip install elasticsearch_dsl

Apparently I had to pip install elasticsearch and elasticsearch-dsl outside of the virtualenv. The error went away after that.

Related

Python script not run in Django environment

Terminal says that i didn't defined DJANGO_SETTINGS_MODULE but i tried as i thought every method to do it and so far nothing helper (coding on windows)
Can someone help me out please? I am stuck and dont know what to do
Maby this will help - i run my virtual env through Anaconda -
conda activate djangoenv
raise ImproperlyConfigured(django.core.exceptions.ImproperlyConfigured: Requested setting INSTALLED_APPS, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
from faker import Faker
from app.models import AccessRecrod, Webpage, Topic
import random
import django
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
django.setup()
fakegen = Faker()
topics = ['Search', 'Social', 'Marketplace', 'News', 'Games']
def add_topic():
t = Topic.objects.get_or_create(top_name=random.choice(topics))[0]
t.save()
return t
def populate(N=5):
for entry in range(N):
# get topic for entry
top = add_topic()
# create fake data for entry
fake_url = fakegen.url()
fake_date = fakegen.date()
fake_name = fakegen.company()
# create new webpage entry
webpg = Webpage.objects.get_or_create(
topic=top, url=fake_url, name=fake_name)[0]
# create fake access record
acc_rec = AccessRecrod.objects.get_or_create(
name=webpg, date=fake_date)[0]
if __name__ == '__main__':
print('populating script!')
populate(20)
print('populating complete!')
The Problem is simply your import of models. You just need to setup django before you import anything related to django:
import django
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project.settings')
django.setup()
from faker import Faker
from app.models import AccessRecrod, Webpage, Topic
import random
rest of your code
If you check the code in manage.py and django code on github you will essentially find the same sequence of code (besides a lot of additional stuff) when you run a management command by "python manage.py your_command"
To use the management command directly as described in the answer from #Code-Apprentice would be the other "normal" or "more django way" to integrate your script.
But you asked for the reason of the error message in your way of doing it.
If you study the complete error trace you will also find that it starts at the import command.
Instead of trying to shoe horn a regular python script into the Django environment like this, I suggest you create a command which you can run with ./manage.py. You can learn how to create your own custom commands here. When you do this correctly, you can run a command like ./manage.py create_topics or whatever name you give your command. This allows manage.py to load the Django environment for you.
Alternatively, you could look at creating fixtures with ./manage.py dumpdata and ./manage.py loaddata. These commands will allow you to create data and save them to a .json file which you can load into your database at any time.

No module named 'django' in standalone Django App

I installed Django on Windows 10, Django Version 3, Python 3.8 from Conda, with env, on VS Code
My Django works fine without any problem and my App and Project also works fine, but I decided to use Faker to generate some fake data in my DB, I have to mention that my Model works and connected successfully to my DB and migration process was successful without any problem.
I write this Standalone app for Faker to run it whenever I need manually:
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'first_project.settings')
import django
django.setup()
# Implement Fake Population script here.
import random
from first_app.models import AccessRecord, Topic, Webpage
from faker import Faker
fakegen = Faker()
topics = ['Search', 'Social', 'Marketplace', 'News', 'Games']
def add_topic():
t = Topic.objects.get_or_create(top_name=random.choice(topics))[0]
t.save()
return t
def populate(N=5):
for entry in range (N):
# Get the topic for the entry
top = add_topic()
# Create the fake data for that entry
fake_url = fakegen.url()
fake_date = fakegen.date()
fake_name = fakegen.company()
# Create the new webpage entry
webpg = Webpage.objects.get_or_create(topic=top,url=fake_url, name=fake_name)[0]
# Create a fake access record for that webpage
acc_rec = AccessRecord.objects.get_or_create(name=webpg,date=fake_date)[0]
if __name__ == "__main__":
print('Population Script!')
populate(20)
print('Population Completed!')
But whenever I run this code, I get this error:
> (myDjangoEnv) C:\my_path\first-project
> C:/Users/HPTav/anaconda3/python.exe
> c:/my_path/first_project/populate_first_app.py
Traceback (most recent call last):
File "c:/my_path/first_project/populate_first_app.py", line 4, in
<module>
import django
ModuleNotFoundError: No module named 'django'
> (myDjangoEnv) C:\my_path\first-project>
I replaced the path with my_path to make it easier for you when you read the console error.
This is my Django project structure:
I am sure I have Django, as I said my Django app and project works fine.
I checked it by this way to make sure I have Django on this folder:
This project is available in GitHub:
https://github.com/hptavakoli/django_first_app
Sounds like you don't have django installed, install it using this command:
pip install django
In your Command prompt.
Visit this site for additional info: Installing django using pip
What you can do is this:
"Add 'python' while calling populate_first_app.py in the terminal."
python populate_first_app.py

How to use functions from a custom module in Python Django shell

I have a python file called my_functions.py in which I have the following code:
from core.models import Blog
def news():
b = Blog(name='New Blog', tagline='All the latest news.')
b.save()
My main app folder in django is called core and I have put my python file in there. In the shell I am able to do the import: from core import my_functions
However I get an error AttributeError: module 'core.my_functions' has no attribute 'news' when I try to run the code my_functions.news().
How can I run the news function in the shell?
My tree structure is as follows:
core
-__init__.py
-admin.py
-apps.py
-models.py
-my_functions.py
-tests.py
-urls.py
-views.py
Everthing else works as normal but I just cant seem to figure why I cant do this simple import and run the function. I'm using VSCode.
Make sure there's an __init__.py file in the core directory. Then:
from core.my_functions import news
Also you have to restart your shell if you make changes to any file in your project, since the django shell will load all modules in memory at launch time.
Create your file and write your function. In my case was:
# /home/$(USER)/repos/bo/apps/base/queries.py
# "apps.base", added to INSTALLED_APPS list in settings.py
from apps.base.models import Player
from core.models import Blog
def news():
b = Blog(name='New Blog', tagline='All the latest news.')
b.save()
return b
def count_player_overs():
all_playerovers = Player.objects.all()
count_all_playerovers = all_playerovers.count()
return count_all_playerovers
print('count_all_playerovers: {}'.format(count_player_overs()))
print('saved news object: {}'.format(news()))
# this folder has manage.py
cd /home/$(USER)/repos/bo/
And run:
./manage.py shell -c "from apps.base import queries;"
Result:
count_all_playerovers: 376
saved news object: New Blog
You can change the code and run again ./manage.py shell -c "from apps.base import queries;" and it will re-import the module and run the function with all your changes.

pytest can't access django classes

I have a Django application, working fine. When I run the tests with pytest, it only works with utility classes (so not Django related).
For example, a test from package A calling an utility class from this package, or another, works fine.
However, I'm facing an error as soon as I import a django class.
example 1 :
I import my model (in the test), starting the test class with :
from app.common.models import Country
--> ImportError: No module named django.db
[django.db is called in models.py]
example 2 : I import an url resolver (in the test), starting the test class with :
from django.core.urlresolvers import reverse
--> ImportError: No module named django.core.urlresolvers
first try of fix
Following another topic, I set the content of PYTHONPATH :
/home/user/pyenv/lib/python3.5/site-packages
This folder contains installed packages in the virtualenv : django, pytest, psycopg2, and more.
1) If I set DJANGO_SETTINGS_MODULE to the same file as the application, "py.test" gives this error ending with :
File "/home/user/pyenv/lib/python3.5/site-packages/django/db/backends/postgresql/base.py", line 24, in
raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
2) If I set DJANGO_SETTINGS_MODULE to a smaller test setting file, containing only the database infos, I face a different error (self.client failing in a test) :
class Test(unittest.TestCase):
def testUrls(self):
response = self.client.get('/countries/all/get')
self.assertEqual(response.status_code, 200)
--> AttributeError: 'Test' object has no attribute 'client'
more infos :
1) The python interpreter, for the application, is located in a virtualenv.
2) conftest.py is located in application root folder, so same place as manage.py, and has the following content:
import os
import sys
sys.path.append(os.path.dirname(__file__))
3) pytest.ini is located in same folder as manage.py too, with this content :
[pytest]
python_files = test_*.py test*.py
4) most important : the application works fine, so db settings are valid
If you have any idea of what's wrong, and how to test django classes, any idea will be welcomed. Thank you in advance.
I use pytest-django and I also explicitly set DJANGO_SETTINGS_MODULE=project.my_settings in pytest.ini. Works great every time.
I faced other issues trying the standard Django testing tool + coverage.
Using Pytest with "--cov" provided me what I needed, in one command only.
I finally found a fix for the issue. After trying again, and so restarting everything (pc, virtualenv), it worked fine.
So :
1) start virtualenv and move to the application folder
2) don't set PYTHONPATH
3) set DJANGO_SETTINGS_MODULE to application settings in pytest.ini
(so only one settings file in the project)
4) run the test with : py.test --cov=my_application_name
Now I can test models or forms without error.
For the previously mentioned test about urls, it works now with the following syntax:
import unittest
from django.test import Client
class Test(unittest.TestCase):
def setUp(self):
# global variable
self.client = Client()
def testUrls(self):
reponse = self.client.get('/countries/all/get')
self.assertEqual(reponse.status_code, 200)

web2py db is not defined

I'm trying to run a script at command line that uses the models with the following command:
c:\web2py>python web2py.py -M -N -S automate -R applications/automate/modules/eventserver.py
but I keep getting the error:
web2py Web Framework
Created by Massimo Di Pierro, Copyright 2007-2011
Version 1.99.7 (2012-03-04 22:12:08) stable
Database drivers available: SQLite3, pymysql, pg8000, IMAP
Traceback (most recent call last):
File "c:\web2py\gluon\shell.py", line 206, in run
execfile(startfile, _env)
File "applications/automate/modules/eventserver.py", line 6, in <module>
deviceHandler = devicehandler.DeviceHandler()
File "applications\automate\modules\devicehandler.py", line 10, in __init__
self.devices = self.getActiveDevices()
File "applications\automate\modules\devicehandler.py", line 18, in getActiveDe
vices
print db
NameError: global name 'db' is not defined
What am I doing wrong?
edit: From my research I have only found the solution "add -M to your command" but I've already done that and it still doesnt work.
edit2: I have db = DAL('sqlite://storage.sqlite') in my db.py so it should get loaded
edit2: I have db = DAL('sqlite://storage.sqlite') in my db.py so it should get loaded
Assuming db.py is in the /models folder, the db object created there will be available in later executed model files as well as in the controller and view, but it will not be available within modules that you import. Instead, you will have to pass the db object to a function or class in the module. Another option is to add the db object to the current thread local object, which can then be imported and accessed within the module:
In /models/db.py:
from gluon import current
db = DAL('sqlite://storage.sqlite')
current.db = db
In /modules/eventserver.py:
from gluon import current
def somefunction():
db = current.db
[do something with db]
Note, if you do define the db object in the module, don't define it at the top level -- define it in a function or class.
For more details, see the book section on modules and current.

Categories