Here is my problem, I am trying to make an application which copies data files during its setup. When i am doing pip install the setup copies a few files to a directory.
Now my question is, When inside a virtual environment, what is the behaviour that the customer expects- does he want all the created data files inside the virtual environment directory for each virtualenv or copy all the files into a common directory outside the virtual environment directory.
While running the application there will be new files that will be created and stored along these copied files. What is the ideal behaviour that is expected form a python virtualenv. common or isolated?
virtualenv is more for development, not for deployment. There are many scenarios to deploy Python app. but if you prefer virtualenv usage and you have common files, they can be anywhere IMHO, because virtualenv is not real isolation, it's only Python paths mangling/modification mechanism (not like "chroot"), so you decide where to place your common files, even /usr/share/my-app-1.0/datafiles/. Also, virtualenv is used to isolate binaries/scripts, but if data files are static, you can place them when you prefer.
In my opinion, that depends on application, which you create. Virtualenv is just way of running on same machine multiple applications with different dependencies. Data from applications is another thing.
When I would write web application, that will be single app on server, then I would use one directory.
On the other hand, when I would write GUI app, then things get different. If data is something that must be changed with every version, but end user does not touch it directly (e.g. some dictionaries, translation, etc) I would put it in dist-packages along the application package (see package data in setup.py).
On the other hand, if user can "touch" and use those files, then I would put them in users home directory. See How to find the real user home directory using python?
Related
I'm working on building my first Python package for a client to use. At the most, I am envisioning the user pulling the code from GitHub, then pip installing (pip install .). This package will be used in a Windows environment. What is the convention or where is the easiest place to put log files? Is there a way to tell setup.py to make a log directory that is easily accessible for the user?
For a more specific example, let's say I had the code base locally at C:\Users\iamuser\projects\client_project. I pip install . while in the client_project directory. There is a logs\ directory (C:\Users\iamuser\projects\client_project\logs) that I'd like the log files to be placed into. Is there a way to have my setup.py place log files in that directory? If not, are there any other tools I should try?
I have tried something like this, but any paths acquired while running setup are not where the original setup.py file was located (example: os.path.abspath(__file__) shows some other location than within the client_project directory).
While creating a Python package, I would not make any assumptions about the user's filesystem or permissions therein (ideally not even about the OS). If your Python package creates log files, you could use Python's build-in logging system and logging information would go wherever the user wants it to go (stdout and stderr by default).
If your package generates files the user should have the option to decide where they go, either using a settings or config file or environmental variables.
There are a few places where you could safely store them by default. Such as the home or current working directory (or subfolders of them, see pro and cons in the comments). Important is to use relative paths either in relationship to ~, os.getcwd() or the __file__ attribute of your script. Linux systems have a few places that are usually present and can be used such as /tmp or /var/log but that does not work on Windows.
Sometimes, I store output files in the parent of the current working directory in order not to checkin output files into a Git repo but this relays on additional assumptions.
I need to pack an application written in Python.
I can't use PyInstaller because I can't have a .exe or anything like that, I need pure python scripts being executed (I have Popen in my code calling some of my scripts and passing parameters);
I need it to work cross-plataform (I KNOW that to do so, I need to generate the package in each system because a package generated on Linux won't work in a Mac and vice-versa);
Since I can't really pack it, my idea is to have a folder called modules and have all the dependencies inside it, then, in my code I would just point the imports to this folder. I would zip the entire project and ship it, the user would just unpack and run it.
The problem is:
How to not only download the package in a specific folder but also install it there? (I can't alter my user's environment, this is a MUST);
How to direct my imports to this local folder? I think I can do something like import modules.numpy for example, but then, numpy have dependencies of its own... how to make sure it will look into my custom folder?
My scenario:
I have a requirements.txt for my project; I have several local files that are used inside the project; I have a Popen that calls one of my file.py; I am using Python3;
I can't use virtualenv because I am using wxPython and there's a conflict between it and virtualenv (no idea why - something related with main thread...)
I'm reading headfirst python and have just completed the section where I created a module for printing nested list items, I've created the code and the setup file and placed them in a file labeled "Nester" that is sitting on my desktop. The book is now asking for me to install this module onto my local copy of Python. The thing is, in the example he is using the mac terminal, and I'm on windows. I tried to google it but I'm still a novice and a lot of the explanations just go over my head. Can someone give me clear thorough guide?.
On Windows systems, third-party modules (single files containing one or more functions or classes) and third-party packages (a folder [a.k.a. directory] that contains more than one module (and sometimes other folders/directories) are usually kept in one of two places: c:\\Program Files\\Python\\Lib\\site-packages\\ and c:\\Users\\[you]\\AppData\\Roaming\\Python\\.
The location in Program Files is usually not accessible to normal users, so when PIP installs new modules/packages on Windows it places them in the user-accessible folder in the Users location indicated above. You have direct access to that, though by default the AppData folder is "hidden"--not displayed in the File Explorer list unless you set FE to show hidden items (which is a good thing to do anyway, IMHO). You can put the module you're working on in the AppData\\Roaming\\Python\\ folder.
You still need to make sure the folder you put it in is in the PATH environment variable. PATH is a string that tells Windows (and Python) where to look for needed files, in this case the module you're working on. Google "set windows path" to find how to check and set your path variable, then just go ahead and put your module in a folder that's listed in your path.
Of course, since you can add any folder/directory you want to PATH, you could put your module anywhere you wanted--including leaving it on the Desktop--as long as the location is included in PATH. You could, for instance, have a folder such as Documents\\Programming\\Python\\Lib to put your personal modules in, and use Documents\\Programming\\Python\\Source for your Python programs. You'd just need to include those in the PATH variable.
FYI: Personally, I don't like the way python is (by default) installed on Windows (because I don't have easy access to c:\\Program Files), so I installed Python in a folder off the drive root: c:\Python36. In this way, I have direct access to the \\Lib\\site-packages\\ folder.
I am making the base skeleton of some Django project files so that I can put them on git and whenever I need to make a new Django site I can grab the files from git and start a blank project.
In my fabfile, I'm generating a virtualenv named virutalenv.
I just want to know that if I need to make many sites on single computer then all will have same not but they will be in the project directory.
Is that ok?
It is possible to create multiple virtualenvs with the same name; they must be in different parent directories, however.
Alternately, you could create multiple virtualenvs in the same parent directory, but with different names.
I have a python project developed in eclipse. Independantly there are various data directories in various locations and it is desirable to operate on each with a different main.py that imports the python project. If I store a main.py with my data, how can I open it in eclipse and run the debugger?
I can run ipython in each directory without a problem; but, I am not clear on how to move between different main.py files within eclipse. The run configuration does allow you to go outside the project and I would prefer not having to change this each time anyway. Any insight would be great!
You could do this by setting up each main.py + data directory as its own Eclipse project, each with a custom run/debug configuration. You do have extra projects, but they don't require any ongoing maintenance.
After the initial setup, you continue work on the real python project and debugging any given data directory consists of selecting the appropriate project and invoking the debugger. Not maximally elegant, but not bad.
If you're truly loath to make new projects, you could bring the main.py files into the project and store the location of the associated data directory inside each of them. Each main begins by changing to the data directory, and then runs as usual. In eclipse you set up multiple run configurations, one for each main.py which you can then run without any extra tinkering.
It is inelegant and fragile to keep the data directory information inside each main.py, but maybe better for your purposes.