I guess this is a python path problem (on FreeBSD 8.1).
Im trying to convert a Data.fs to Postgresql using zodbconvert. Ive downloaded RelStorage-1.5.0b2 and is running:
/usr/local/Plone/Python-2.6/bin/python zodbconvert.py fstodb.conf
, to use the version that Plone is running with.
The error I get:
Traceback (most recent call last):
File "zodbconvert.py", line 22, in <module>
from persistent.TimeStamp import TimeStamp
ImportError: No module named persistent.TimeStamp
Versions:
Plone 4.0.5 (python 2.6)
Postgresql 9.0.3
FreeBSD 8.1
python26-2.6.6_1
python27-2.7.1_1
PS by default "python --version" is 2.7.1
Thanks.
Nikolaj G.
If you are using buildout (I do hope you are) the easiest way to get all the zodbconvert dependencies properly included in the python path is to have buildout create the script for you:
[buildout]
...
parts =
...
zodbconvert
[zodbconvert]
recipe = zc.recipe.egg
eggs = ${buildout:eggs}
scripts = zodbconvert
Buildout then will create a new bin/zodbconvert script for you that includes all the buildout eggs in sys.path.
Alternatively, you can create a generic python script runner that includes all eggs in your buildout and can run arbitrary scripts; you can use this instead of the bare-bones python interpreter to run arbitrary python scripts with all the buildout eggs in sys.path:
[buildout]
...
parts =
...
zopepy
[zopepy]
recipe = zc.recipe.egg
eggs = ${buildout:eggs}
interpreter = zopepy
scripts = zopepy
The bin/zopepy script can then be use to run arbitrary python scripts with all your buildout eggs already in sys.path, so bin/zopepy zodbconvert.py fstodb.conf should work.
Note that the Plone unified installer already comes with the zopepy part included, and my choice of partname for this script was deliberately using the same name.
If you are not using buildout (and with Plone 4, that's not a good idea on the whole), you can also list the required packages (ZODB3, zope.interface, RelStorage, psycopg2) in your PYTHONPATH environment variable.
quick fix..
locate persistent
export PYTHONPATH=$PYTHONPATH:/path/to/your/python_persistent_dir
You have not included the ZODB package with your Python installation. Either adjust the PYTHONPATH to include the ZODB package or just easy_install ZODB - depending on what you are trying to do.
Related
Env:
Windows 10
python 3.6.6
cx-Freeze 5.0.2
Git hub example
It contails .msi for installing
Example project structure:
/package_name
/some_packet
/__init.py
/module_name.py # for an example contains valiable in code "module_1"
/main.py
/setup.py
/some_module.py # for an example contains valiable "module_2"
/some_other_module.py # for an example contains valiable "module_3"
Example of setup.py(simplified)
import cx_Freeze
cx_Freeze.setup(
name="example",
options={
"build_exe": {
"packages": ["asyncio"],
"include_files": ["static\some_static_file.png"]
},
"bdist_msi": {
"upgrade_code": "{492de237-1853-4599-a707-c283d567699f}"
}
},
executables=[cx_Freeze.Executable("main.py")]
)
Current behavior
For creating .msi install file -> run command python setup.py bdist_msi. It will generate .msi files for installing application.
After installing this application: directory(where application is installed) will contain:
main.exe
lib\some_packet directory
lib\some_packet\module_name.pyc file
other files
There are following statements:
1) From root directory(where application is installed) i start search(via grep -Rna command under Ubuntu guest system, it's just more convenient for me) and valiable module_1 could be found in directories(in lib\some_packet\module_name.pyc) and module_2/module_3 couldn't be found. Details:
(v_decompile) any#any-pc:/mnt/hgfs/shar/example_installed$ grep -Rna "module_1"
lib/some_packet/module_name.pyc:2:B�!]�#dZdS)module_1N)r�rr�PG:\heroes\installer-with-cx_Freeze\sources_of_project\some_packet\module_name.py<module>s
(v_decompile) any#any-pc:/mnt/hgfs/shar/example_installed$ grep -Rna -a "module_2"
(v_decompile) any#any-pc:/mnt/hgfs/shar/example_installed$ grep -Rna -a "module_3"
2) File lib\some_packet\module_name.pyc could be easily converted to original file(without comments) by e.g. python-uncompyle6.
Details:
(v_decompile) any#any-pc:/mnt/hgfs/shar/example_installed$ uncompyle6 lib/some_packet/module_name.pyc
# uncompyle6 version 3.3.3
# Python bytecode 3.6 (3379)
# Decompiled from: Python 3.6.6 (default, Jul 20 2018, 15:39:05)
# [GCC 4.8.4]
# Embedded file name: G:\heroes\installer-with-cx_Freeze\sources_of_project\some_packet\module_name.py
# Compiled at: 2019-07-07 11:28:50
module_1 = 'module_1'
# okay decompiling lib/some_packet/module_name.pyc
3) (solved with this question) In both points: file contains source path G:\heroes\installer-with-cx_Freeze\sources_of_project\some_packet\module_name.py It confuses me a bit. Application was installed from .msi and (as I understand) should not know about source directories (regarding path) which was used for creating last one.
Questions:
Is there any way to recover some_module.py and some_other_module.py to original files from main.exe?(like it could be done with lib\some_packet\module_name.pyc)
How to "hide" some other files in application into main.exe or somehow avoid converting .pyc to original files.(maybe some properties in cx_Freeze?)
Note:
It should be done with cx_Freeze.
PS: I don't want to create single .exe. I try to find convenient way for specifying which files should be stored in main.exe like it was done with some_module.py and some_other_module.py
PSS: At this moment I see only I way: put all files on main.py level :) But it will look weird for big project.
Quoting How to obfuscate Python source code:
These two methods [using pyobfuscate and distributing bytecode] are really just a deterrent, not a secure way of hiding the code.
If you want something a bit more robust, you should take a look at Nuitka, which compiles Python code to C++, so you can compile that and just distribute the executable. It seems to be broadly compatible with different libraries and different versions of Python.
See also Python Code Obfuscation
This video might help.
It talks about cxfreeze and how you can use cxfreeze to make a excutable, and I know it works for 3.4+ because the video uses python 3.4, but really your method should be fine...
If I understand Your question correctly, this could be answer You want - Python cx_Freeze for two or more python files (modules)
I am trying to build and distribute rpm package of python module for centos. I have followed following steps
created virtualenv and installed requires
in module added setup.py with install_requires.
then using python2.7 from virtualenv build package
../env/bin/python2.7 setup.py bdist_rpm
Now I got src, no-arch and tar-gz files in 'dist' folder.
foo-0.1-1.noarch.rpm, foo-0.1-1.src.rpm, foo-0.1.tar.gz
I tried to install package src-rpm using 'sudo yum install foo-0.1-1.src.rpm',
got error something like wrong architecture
Then I tried to install package no-arch, 'sudo yum install foo-0.1-1.noarch.rpm' it works smoothly.
But after running script, it gave some import error. here I expect to download that module automatically.
The last thing is I am using some third party library which is not on pip.
So I want to whole setup using virtualenv with required modules. So after installing rpm, user can run script directly instead of installing third party libs separately and explicitly.
Some above steps may sounds wrong, as I am new to this stuff.
Following is code in setup.py
from setuptools import setup, find_packages
setup(
name = "foo",
version = "0.1",
packages = find_packages(),
scripts = ['foo/bar.py', ],
# Project uses reStructuredText, so ensure that the docutils get
# installed or upgraded on the target machine
install_requires = ['PyYAML', 'pyOpenSSL', 'pycrypto', 'privatelib1,'privatelib2', 'zope.interface'],
package_data = {
# If any package contains *.txt or *.rst files, include them:
'': ['*.txt', '*.rst'],
# And include any *.msg files found in the 'billing' package, too:
'foo': ['*.msg'],
},
# metadata for upload to PyPI
author = "foo bar",
description = "foo bar",
license = "",
keywords = "foo bar",
# could also include long_description, download_url, classifiers, etc.
)
Also I am using shebang in script as,
#!/usr/bin/env python2.7
Note:
I have multiple python setups. 2.6 and 2.7
By default 'python' commands gives 2.6
while command 'python2.7' gives python2.7
output of `'rpm -qp foo-0.1-1.noarch.rpm --requires' =>
`/usr/bin/python
python(abi) = 2.6
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
When i install pakcage. script's shebang line (which is now '/usr/bin/bar.py') is getting changed to /usr/bin/python' But I exclusively want to run script on python2.7.
Thanks in advance
I use zc.buildout to build my python application environment. I want to run IPython notebook and use my custom packages to further test and develop.
[buildout]
extends = buildout.cfg
parts +=
ipython_part
[versions]
ipython = 0.13.2
pyzmq = 13.0.2
[ipython_part]
#http://ipython.org/ipython-doc/stable/install/install.html#dependencies-for-the-ipython-html-notebook
#https://github.com/bearstech/ipython_notebook/blob/master/buildout.cfg
recipe = zc.recipe.egg
dependent-scripts = true
eggs =
ipython[zmq,notebook,test]
${myapplication:eggs}
Starting the notebook works, but as soon as I create a new notebook the Kernel dies.
[NotebookApp] Kernel started: c7c64caf-c966-4863-b37d-11cf11901882
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named IPython.zmq.ipkernel
Running IPython inside a virtualenv works like a charm. My problem is related to a buildout setup. I am aware of this answer. It works.
Regarding the wide spread use of buildout & IPython notebook this must be a common situation. Are there any buildout recipes to make it work out-of-the-box?
According to this answer in the notebook startup a process is forked and looses the sys.path prepared by buildout. The same solution should apply to your case as well.
Edit: I settled on adding an initialization keyword to my "development" recipe:
[development]
recipe = zc.recipe.egg
eggs = ipython[all]
# ugly work-around needed for ipython nootbook
initialization =
import sys, os
os.environ['PYTHONPATH'] = ':'.join(sys.path)
Much more clean and it mostly work.
I'm trying to run a python script using python 2.6.4. The hosting company has 2.4 installed so I compiled my own 2.6.4 on a similar server and then moved the files over into ~/opt/python. that part seems to be working fine.
anyhow, when I run the script below, I am getting ImportError: No module named _sqlite3 and I'm not sure what to do to fix this.
Most online threads mention that sqlite / sqlite3 is included in python 2.6 - so I'm not sure why this isn't working.
-jailshell-3.2$ ./pyDropboxValues.py
Traceback (most recent call last):
File "./pyDropboxValues.py", line 21, in
import sqlite3
File "/home/myAccount/opt/lib/python2.6/sqlite3/__init__.py", line 24, in
from dbapi2 import *
File "/home/myAccount/opt/lib/python2.6/sqlite3/dbapi2.py", line 27, in
from _sqlite3 import *
ImportError: No module named _sqlite3
I think I have everything set up right as far as the directory structure.
-jailshell-3.2$ find `pwd` -type d
/home/myAccount/opt
/home/myAccount/opt/bin
/home/myAccount/opt/include
/home/myAccount/opt/include/python2.6
/home/myAccount/opt/lib
/home/myAccount/opt/lib/python2.6
/home/myAccount/opt/lib/python2.6/distutils
/home/myAccount/opt/lib/python2.6/distutils/command
/home/myAccount/opt/lib/python2.6/distutils/tests
/home/myAccount/opt/lib/python2.6/compiler
/home/myAccount/opt/lib/python2.6/test
/home/myAccount/opt/lib/python2.6/test/decimaltestdata
/home/myAccount/opt/lib/python2.6/config
/home/myAccount/opt/lib/python2.6/json
/home/myAccount/opt/lib/python2.6/json/tests
/home/myAccount/opt/lib/python2.6/email
/home/myAccount/opt/lib/python2.6/email/test
/home/myAccount/opt/lib/python2.6/email/test/data
/home/myAccount/opt/lib/python2.6/email/mime
/home/myAccount/opt/lib/python2.6/lib2to3
/home/myAccount/opt/lib/python2.6/lib2to3/pgen2
/home/myAccount/opt/lib/python2.6/lib2to3/fixes
/home/myAccount/opt/lib/python2.6/lib2to3/tests
/home/myAccount/opt/lib/python2.6/xml
/home/myAccount/opt/lib/python2.6/xml/parsers
/home/myAccount/opt/lib/python2.6/xml/sax
/home/myAccount/opt/lib/python2.6/xml/etree
/home/myAccount/opt/lib/python2.6/xml/dom
/home/myAccount/opt/lib/python2.6/site-packages
/home/myAccount/opt/lib/python2.6/logging
/home/myAccount/opt/lib/python2.6/lib-dynload
/home/myAccount/opt/lib/python2.6/sqlite3
/home/myAccount/opt/lib/python2.6/sqlite3/test
/home/myAccount/opt/lib/python2.6/encodings
/home/myAccount/opt/lib/python2.6/wsgiref
/home/myAccount/opt/lib/python2.6/multiprocessing
/home/myAccount/opt/lib/python2.6/multiprocessing/dummy
/home/myAccount/opt/lib/python2.6/curses
/home/myAccount/opt/lib/python2.6/bsddb
/home/myAccount/opt/lib/python2.6/bsddb/test
/home/myAccount/opt/lib/python2.6/idlelib
/home/myAccount/opt/lib/python2.6/idlelib/Icons
/home/myAccount/opt/lib/python2.6/tmp
/home/myAccount/opt/lib/python2.6/lib-old
/home/myAccount/opt/lib/python2.6/lib-tk
/home/myAccount/opt/lib/python2.6/hotshot
/home/myAccount/opt/lib/python2.6/plat-linux2
/home/myAccount/opt/lib/python2.6/ctypes
/home/myAccount/opt/lib/python2.6/ctypes/test
/home/myAccount/opt/lib/python2.6/ctypes/macholib
/home/myAccount/opt/share
/home/myAccount/opt/share/man
/home/myAccount/opt/share/man/man1
And finally the contents of the sqlite3 directory:
-jailshell-3.2$ find `pwd`
/home/myAccount/opt/lib/python2.6/sqlite3
/home/myAccount/opt/lib/python2.6/sqlite3/__init__.pyo
/home/myAccount/opt/lib/python2.6/sqlite3/dump.pyc
/home/myAccount/opt/lib/python2.6/sqlite3/__init__.pyc
/home/myAccount/opt/lib/python2.6/sqlite3/dbapi2.pyo
/home/myAccount/opt/lib/python2.6/sqlite3/dbapi2.pyc
/home/myAccount/opt/lib/python2.6/sqlite3/dbapi2.py
/home/myAccount/opt/lib/python2.6/sqlite3/dump.pyo
/home/myAccount/opt/lib/python2.6/sqlite3/__init__.py
/home/myAccount/opt/lib/python2.6/sqlite3/dump.py
I feel like I need to add something into the sqlite3 directory - maybe sqlite3.so? But I don't know where to get that.
What am I doing wrong here? Please remember that I'm using a shared host so that means installing / compiling on another server and then copying the files over. Thanks! :)
Update
Just wanted to confirm that the answer from #samplebias did work out very well. I needed to have the dev package installed on the machine I was compiling from to get it to add in sqlite3.so and related files. Also found the link in the answer very helpful. Thanks #samplebias !
Python's build system uses a setup.py file to compile all of the native extensions, including sqlite3. It searches common operating system paths for the sqlite3 include and library dirs. If the sqlite3 development package is not installed Python will skip compiling the _sqlite3.so extension, but the pure Python portion of the sqlite3 package will still be installed.
You would need to have the operating system's sqlite3 development package installed when you compile Python and at runtime: sqlite3-devel on Centos, both libsqlite3-0 and libsqlite3-dev on Ubuntu.
Here's an example of the _sqlite3.so extension linkage on my Ubuntu system:
% ldd /usr/lib/python2.6/lib-dynload/_sqlite3.so | grep sqlite3
libsqlite3.so.0 => /usr/lib/libsqlite3.so.0 (0x00007f29ef3be000)
% dpkg -S /usr/lib/libsqlite3.so.0
libsqlite3-0: /usr/lib/libsqlite3.so.0
In general, the first thing to do is to ask your host. I seems a bit odd that SQLite is not installed (or installed properly). So they'll likely fix it quite fast if you ask them.
For python 2.4, you need sqlite and bindings, pysqlite 2 or aspw
None of the files listed in the sqlite folder is the _sqlite3.pyd Python shared library. Are you sure you compiled it when compiling Python? What does the build log say? I think there's a configure flag that needs to be passed.
Alternatively, just install pysqlite
I have difficulty especially in installing MySQLdb module (MySQL-python-1.2.3c1), to connect to the MySQL in MAMP stack.
I've done a number of things such as copying the mysql include directory and library (including plugin) from a fresh installation of mysql (version 5.1.47) to the one inside MAMP (version 5.1.37).
Now, the MySQLdb module build and install doesnt give me error.
The error happens when I'm calling 'import MySQLdb' from python shell (version 2.6).
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py", line 19, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so, 2): Symbol not found: _mysql_affected_rows
Referenced from: /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Expected in: flat namespace
in /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Any idea, what else do I need to do to make it works?
Thanks a bunch,
Robert
=========
Add the system response after using virtualenv as suggested by Hank Gay below...
(MyDjangoProject)MyMacPro:MyDjangoProject rhenru$ which python
/Users/rhenru/Workspace/django/MyDjangoProject/bin/python
After I run python in virtualenv, importing MySQLdb:
>>> import MySQLdb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py", line 19, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in <module>
File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so, 2): Symbol not found: _mysql_affected_rows
Referenced from: /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
Expected in: flat namespace
in /Users/rhenru/.python-eggs/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
import sys and sys.path
>>> import sys
>>> print sys.path
['', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages/distribute-0.6.10-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages/pip-0.7.1-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python26.zip', '/Library/Python/2.6/site-packages/PyXML-0.8.4-py2.6-macosx-10.6-universal.egg', '/Library/Python/2.6/site-packages/pydot-1.0.2-py2.6.egg', '/Library/Python/2.6/site-packages/pyparsing-1.5.2-py2.6.egg', '/Library/Python/2.6/site-packages/vobject-0.8.1c-py2.6.egg', '/Library/Python/2.6/site-packages/pytz-2010h-py2.6.egg', '/Library/Python/2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.6-universal.egg', '/Library/Python/2.6/site-packages/distribute-0.6.12-py2.6.egg', '/Library/Python/2.6/site-packages/pip-0.7.1-py2.6.egg', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-darwin', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-mac', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/plat-mac/lib-scriptpackages', '/Users/rhenru/Workspace/django/MyDjangoProject/Extras/lib/python', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-tk', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-old', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/Users/rhenru/Workspace/django/MyDjangoProject/lib/python2.6/site-packages', '/Library/Python/2.6/site-packages', '/Library/Python/2.6/site-packages/PIL', '/Library/Python/2.6/site-packages/setuptools-0.6c11-py2.6.egg-info', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC', '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/wx-2.8-mac-unicode']
How are you installing MySQL-Python? I just tested in a fresh virtualenv and pip install mysql-python seems to have done the trick.
UPDATE:
pip is sort of like a package manager for Python packages.
By default, pip installs to your current site-packages directory, which is on your $PYTHONPATH. This lets other libraries/applications (like Django) access it. pip also works well with virtualenv (it should; Ian Bicking wrote them both), which is a nifty library that lets you sandbox an application. This is nice because it means you can try out new things without polluting (or even needing write access to) the global site-packages directory.
It probably seems like yak-shaving right now, but I'd say it's worth the effort to get up to speed on pip and virtualenv (you may also want to look into virtualenvwrapper, but we'll skip that for now; it's just sugar for virtualenv). It will lead to a slightly more complicated deployment scenario than putting everything in the global site-packages, but for development it's really no harder, and there are lots of good guides to deploying using a virtualenv.
I'd recommend something like the following:
curl -0 http://python-distribute.org/distribute_setup.py
python distribute_setup.py
easy_install pip
pip install virtualenv
virtualenv --distribute MyDjangoProject --no-site-packages
cd MyDjangoProject
source bin/activate (this activates the sandbox that virtualenv created)
pip install django mysql-python
At this point, you should have a totally functional Django+MySQL install (if I missed any steps, just comment and I'll try to add it in). You can start your Django project like this: django-admin.py startproject MyDjangoProject. cd into your project's directory, edit your settings.py file to point to your MySQL database, and run the dev server to test it out like so: ./manage.py runserver (you may need to chmod u+x your manage.py file). Voila! You should be able to access your site on localhost:8000. When you're done working on the project, you can just use deactivate to exit the virtualenv sandbox.
Try not to hold all this against Django: a lot of it is just best practices stuff for working with Python libraries. You could get by with a lot less, but this way it's more reproducible and you're less likely to accidentally mess up one of this project's dependencies when working on a different project.
I had this problem and it turned out to be due to an errant configuration:
export VERSIONER_PYTHON_PREFER_32_BIT=yes
I can't recall what I had this enabled for (some package that required 32-bit), probably related to Google AppEngine. But Setting it to 'no' solved by issues.
Otherwise I just installed everything using homebrew and pip.