I am working on some repacking of a project that has a variety of different technologies involved. I am trying to move a python module into the project structure for storage in git and have all project files located together.
The python code works in its on folder. I have empty __init__ in each folder and there are no problems with from Documents import *
When I move the folder into my larger project, all of these imports fall apart.
Can anyone please help me to understand what is different? the relative locations seem to be all the same, I wouldn't have thought putting the project in another folder should've affected the ability to see the other modules
Thanks
EDIT:
as mentioned in answers, I try accessing it by nbcu_nes_ingest.Package
which appears to work on windows in my dev box.
When I deploy it to AWS server where things will live, I get the following.
I cannot run my setup.py anymore and It still doesnt see the import
When importing python looks from the root of the structure. In the first example the root is "nbcu_nes_ingest", so when it looks for "Documents" it is directly under the root directory. However, in the second example the root is now "slicer" and there is no "Documents" sub dir directly beneath that. So your import would have to now be "from nbcu_nes_ingest.Documents import NCStoryDocument"
Related
I'm climbing my learning curve in Python and try to understand where to put everything.
I originally have a python module in a folder and then a sub folder src, in this src folder I will then have my main source files say main.py then I will have models folder storing my models codes.
/myproject/src/main.py
/myproject/src/models/a-model.py
/myproject/src/models/b-model.py
So my main will import the model like this:
from models.a-model import a
Then when I package the zip file I just zip the myproject folder with that folder structure and deploy and everything is fine.
Now I have another new module doing something different but need to use the same models.
I can easily duplicate them all and code separately and deploy. But I would like to share the codes to the models, so that when one model changes, I only need to update once, instead of 2 places.
My new module is like
/mynew/src/main-b.py
/mynew/src/models/a-model.py
/mynew/src/models/b-model.py
What is the best practise to do this?
Do I put like this?
/myproject/src/main.py
/mynew/src/main-b.py
/models/a-model.py
/models/b-model.py
And then update the import?
But I have doubt how do I deploy? Do I also have to setup the same folder structures?
One would be adding /myproject/src/models to the environment variable PYTHONPATH. Python adds the directories listed in PYTHONPATH environment variable to sys.path, the list of directories where Python searches when you try to import something. This is bad, because modifying PYTHONPATH has its own side effects, fortunately, virtual environments provide a way to get around those side effects.
Alternatively and much better you could add your modules to site-packages directory, site-packages is added to sys.pathby default, this obviates the need to modifyPYTHONPATH. To locate thesite-packages` directory, refer to this page from Python Documentation: Installing Python Modules (Legacy version).
You could also use LiClipse IDE which comes with Pydev already installed. Create source a folder from the IDE and link your previous project with your newer project. When you link your projects the IDE adds the source folders of your older project to the PYTHONPATH of your newer project and thus Python will be able to locate your modules.
Whether or not this question has been asked before, I apologize. Currently taking a networking class and can't run this RDT python program because modules aren't being imported even though everything is there.
the instructions from the professor are to run the program and record the results. Program won't run because of this error statement.
ImportError: No module named RTD even though you can clearly see everything is organized and right there. I don't understand why I am having an issue with this. Help? Thanks!
EDIT 1 (Added the import statement)
Import Statement: from RDT import *
I have faced this same problem when I renamed some packages on my Python projects on PyCharm. Looking at you file structure, seems like you have several Python projects under the GBN/RDT directory since there are some .idea directories within each folder and if everything was a single project, there should be some __init__.py files on each folder indicating they are Python packages.
If this is the case, try making PyCharm aware that you have several source directories (e.g., PR3R, RDT, etc...)
So proceed with the following steps:
Right-click over one of the sources directories, say, PR3R.
On the dropdown menu, go to Mark directory as
Select Sources roots
Try to execute RDT.py again. I assume you want to execute the script.
Repeat this process to the other projects.
However... If you want to import something from one module to another module (e.g. import function foo from Receiver.py in RDT.py), you have to:
Mark RDT (child of GBN) as Sources roots (as I explained previously)
Add __init__.py (which is an empty file that Python uses to know that a given directoy is a package of modules) within RDT and in each child directory (e.g., PR3R, PR3S, and so on....)
This issue has been driving me insane for the past few days.
So basically, I'm trying to port over a Pure Python project to a proper PyCharm project. This is to basically improve code quality and project structure.
I wish it was as simple as basically creating a virtualenv to house everything, but it isn't. This project will eventually be developed simultaneously by multiple developers with Git as source control, and the default libraries will be modified. I presume this means that the libraries should ideally be tracked by Git in the end. Virtualenv shouldn't help here as far as I know because it's not portable between systems (or at least that's still being tested).
This project will also be, in the future, deployed to a Centos server.
So the only plan I can think of to successfully pull off this would be to simply bring in all of the libraries (which was done using pip install -t Libraries <ExampleLibrary>) into a single folder, with a __init__.py inside, and use them from other python files as a package within the Pycharm project.
Is this possible / recommended? I tried various methods to reference these libraries, but they all don't work during runtime. Somehow when the files in the library import something else from their own package, an ImportError is raised saying that there's no such module.
Will accept any other suggestions too.
Using Pycharm Community Edition.
EDIT: After having a good night's rest I think the crux of the issue is really just project organization. Before I ported it over to Pycharm the project worked as expected, but this had all of the python files in the root directory, and the libraries in a subfolder of the root, with every project file having the same boilerplate code:
import os, sys
absFilePath = os.path.dirname(os.path.abspath(__file__));
sys.path.insert(1, absFilePath + "/lib")
I was hoping that by using Pycharm to help me flesh out the packages, I could avoid having repeated boilerplate code.
Note: Not full solution.
The addition of the template code below forces the file containing the code to be in the same directory as the libs folder.
For Pycharm, all I had to do was mark the libs folder as a source folder. Even with the addition of the template code to the file, the modified libraries still work as expected.
For the Python Shell, this template code is still needed:
import os
import sys
absFilePath = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(1, absFilePath + "/lib")
I'm fairly new to python and especially it's import mechanism. I'm not entirely sure i'm using the terminology correctly so i should apologize for that up front.
firstly, this seems to be a problem i'm having with a 3rd party import so i can't really change the structure of their release.
In the release, all of the packages are in site-packages/[ROOTFOL]/[PACKAGE]
the [ROOTFOL] does not have a __init__.py file, only the package folders have this file.
this folder is placed into site-packages and the site-packages is present in my PYTHONPATH
in the examples they provide, they use it like this:
import ROOTFOL.PACKAGE.WhateverObject as obj
I'm trying to avoid adding every single package to the PYTHONPATH as there are a bunch of them. Everything seems to work fine, however it really inhibits my ability to work with the auto-complete functionality and that is the frustrating part.
Something else i find strange, is that when the packages are installed, there is a EGG-INFO folder placed along side the package. In this there are several .txt files and one of which is namespace_packages.txt which has only the ROOTFOL. Is there some way i should be setting this to PyDev?
So, what you're seeing here is their distribution model. Usually a module will have one root import that everything stems from, but that's not necessarily the case. They're providing a package with (what I assume) is many modules that don't interact with each other; or they can all stand alone.
instead of importing each package individually, you could use the 'from' keyword:
from ROOTFOL.PACKAGE import *
which will grab everything inside that sub-module. You could e-mail the developer and ask why they deployed it this way...or you could add your own __init__.py to the root folder and,
from ROOTFOL import *
which will walk the tree. Good luck!
First off, I'm pretty new so I hope I hadn't missed anything too trivial.
Here's a small preface with lots of info:
I'm using Windows & Python 2.7.
I've been using an open-source module named pybrain, which I need to change pretty much for my own purposes. So far I've been changing it directly from the python site-packages folder, but I guess it's a pretty messy way to work, so I decided to try and re-do thing so as to launch it from a different folder.
I've also decided to start using Aptana (which as far as I can gather is Eclipse-based enough for the same solutions to apply) instead of the messier but simpler "Spyder" I've been using so far.
Pybrain is a pretty layered module with lots of different subfolders, e.g.:
pybrain
--> subfolder1
--> subfolder2
...
So far I've figured these out:
- I've removed the path to the pybrain folder in site-packages from the PYTHONPATH in aptana project.
- I've added the path to the new project folder.
This works for some imports, namely, the ones that only reference relative paths inside the subfolders, e.g. I can import from things in subfolder1 if I write a module in the main folder.
However, whenever I try to import things from the other subfolder - I can't use "pybrain" in the hierarchy:
from pybrain.subfolder2 import *
doesn't work in subfolder1.
And here is my question:
How do I configure "pybrain" to be a usable name in the code, just as it was when I had pybrain in the site-packages folder?
I think you have added the wrong path to your source folder...
I.e.:
if you have a structure
/project
/project/pybrain
/project/pybrain/__init__.py
The source folder set should be '/project' (whereas I think you've set /project/pybrain as the source folder)... if that's not the case, please add more information on your folders and what did you set as a source folder...
Probably Aptana has some way to configure the list of folders which are considered source packages as in pycharm and in eclipse-pydev.
In any case, you could access your module using a .pth file in your site-packages. This file can be named as you want (p.e. pybrain.pth) and should contain only one line with the path to your pybrain folder. See this and this.