We Keep Coding

Data Driven Security - Chapter 2 - Test Script

I am currently going through Data Driven Security by Jacbos and Rudis, and in Chapter 2 of the book they have the following script that is supposed to be ran in the IPython Console and produce a histogram:
ProductName: Mac OS X
ProductVersion: 10.14
BuildVersion: 18A391
Darwin Kernel Version 18.0.0
Pandas - v0.23.1-4
Numpy - v.1.15.4-1
import pandas as pd
import numpy as np
np.random.seed(1492)
test_df = pd.DataFrame({ "var1": np.random.randn(5000) })
test_df.hist()
The setup instructions suggest using Canopy over pip, which I have attempted to setup using both options to download/update appropriate libraries. When I attempt to execute the code in my Canopy environment, it appears to work as I am not receiving an error, but there is no output (expecting a Histogram).
Welcome to Canopy's interactive data-analysis environment!
Kernel running in the 'User' environment.
Pylab is active using TkAgg.
Python 3.5.2 |Enthought, Inc. (x86_64)| (default, Mar 2 2017, 08:29:05)
Type "copyright", "credits" or "license" for more information.
IPython 5.6.0 -- An enhanced Interactive Python.
%run -i "/Users/john/Desktop/test.py"
%run "/Users/john/Desktop/test.py"
%run -i "/Users/john/Desktop/test.py"
I've been trouble shooting for days and can't seem to figure out why I have do not have the appropriate output. I have attempted running both Py3 and 2.7 but no avail. I have installed and uninstalled Python, and still nothing.
If anyone can recommend or suggest any help I would be very appreciative!

If you are working interactively at the Python prompt (in Canopy's default IPython pylab mode), then those commands will work as-is. Otherwise, such as running a script, as you are, you should also include these commands:
import matplotlib.pyplot as plt
plt.show()
On Python 3, I would also recommend installing the PyQt package in the Canopy Package Manager. See this article: "Python 3 in Canopy 2 - Plotting fails, perhaps with 'No module named PyQt4'"

Updated: TENSORFLOW Activating Virtualenv through Terminal is not Working

macOS High Sierra 10.13.2 — MacBook (Retina, 12-inch, Early 2016)
Trying to set up Python TensorFlow NOT because I like Python (hate it) but because that was what was recommended
NOTE: Not very experienced with Terminals and whatnot, but an okay programmer.
NEW PROBLEMS:
My computer seemed to have downloaded TensorFlow, but when following the steps to test it, this showed up:
Macbook:~ rose$ cd Desktop
Macbook:Desktop rose$ Python
Python 2.7.10 (default, Jul 15 2017, 17:16:57)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import tensorflow as tf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named tensorflow
EVERYTHING seemed fine when I finished the steps for installation...
So, I was thinking today that it would be pretty awesome to see if I could figure out how to set up some sort of neural network programming environment on my computer and eventually found TensorFlow. Everything was going pretty well, I was just following the instructions on this page at the official site for instructions. I installed pip, Virtualenv, nose/tornado (said I needed them, created a Virtualenv environment for Python 3.n, which I updated to today, then got to this step:
Activate the Virtualenv environment by issuing one of the following commands:
$ cd targetDirectory
$ source ./bin/activate # If using bash, sh, ksh, or zsh
$ source ./bin/activate.csh # If using csh or tcsh
I used the command it said to use for bash (wow, these names are horrible and nondescript), but I'm not 100% sure I use bash, almost certain because that is supposed to be the default on Mac and got this:
Input:
Macbook:~ rose$ source ./bin/activate
Error Message Output:
-bash: ./bin/activate: No such file or directory
NOTE: There were some other messages, here are screens of the whole shell.
Screenshots on my GitHub Pages

You haven't created a virtual environment. There were errors. Please reread carefully your own screenshots, last but one and the last.
Cannot copy/paste the error message here from the images. That's one of many reasons why images without text are bad.

Unable to import nltk on mac os x

I had successfully installed nltk from this site. And just to validate i am able to import it from the terminal. But when i execute my python script from the Spyder it gives me following error in Spyders terminal
File "/Prateek/Python/RC_ISSUES/algorithm_RC.py", line 9, in <module>
import nltk
ImportError: No module named nltk
Below output is from the terminal
I know there might be similar questions but i thought it is different from rest of the other questions

When you execute a python script, the operating system is looking for the interpreter as specified on the first line of the script, which most of the time is:
#!/usr/bin/python
On Mac OS X, this is the python as distributed with the system when you installed it, which is the one distributed with the system. It is usually very likely to be the one that has the older compilation date:
2.7.10 (default, Jun 1 2015, 09:45:55) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
If you do type python in your shell, you're very likely to see another path to that interpreter, e.g. if you installed the brew version of python:
% type python
python is /usr/local/bin/python
So you have two ways around it, either you explicitly launch your script with python:
python algorithm_RC.py
in doubt, use the full path you found out with type:
/usr/local/bin/python algorithm_RC.py
or, you can change your script first line with:
#!/usr/bin/env python
which will use the same python as the one you're reaching from your shell. You can also use the full path to your manually installed python, by making that line:
#!/usr/local/bin/python
or whatever the type command gave. But I would advise you against that, as the /usr/bin/env solution is more flexible and makes sure you're using in both cases the same python from the shell and within the script.
Finally, you can also use the system's python by calling explicitly easy_install from /usr/bin:
sudo /usr/bin/pip nltk
And if you don't have pip there, then you'll have to install it first:
sudo /usr/bin/easy_install pip
HTH

try 1:
>>>import nltk
and hit enter and now try
>>>nltk.download()

Python: select one of multiple installed module versions

On my system, I have several modules installed multiple times. To give an example, numpy 1.6.1 is installed in the standard path at /usr/lib/python2.7/dist-packages, and I have an updated version of numpy 1.8.0 installed at /local/python/lib/python2.7/site-packages/.
The reason I cannot simply remove the old version is that I do not have permissions to change anything on my work computer. I however need to use the new numpy version.
I have added /local/python/lib/python2.7/site-packages/ to my PYTHONPATH. Unfortunately, this does not help, since /usr/lib/python2.7/dist-packages is inserted into the path first and therefore, numpy 1.6.1 will be loaded. Here's an example:
>>> import os
>>> print os.environ['PYTHONPATH']
/local/python/lib/python2.7/site-packages
>>> import pprint
>>> import sys
>>> pprint.pprint(sys.path)
['',
'/local/python/lib/python2.7/site-packages/matplotlib-1.3.1-py2.7-linux-x86_64.egg',
'/local/python/lib/python2.7/site-packages/pyparsing-2.0.1-py2.7.egg',
'~/.local/lib/python2.7/site-packages/setuptools-3.4.4-py2.7.egg',
'~/.local/lib/python2.7/site-packages/mpldatacursor-0.5_dev-py2.7.egg',
'/usr/lib/python2.7/dist-packages',
'/local/python/lib/python2.7/site-packages',
'/usr/lib/python2.7',
...,
'~/.local/lib/python2.7/dist-packages',
...]
So, it seems that the import order is
current directory
eggs from PYTHONPATH
eggs from local module path (~/.local/lib/python2.7/site-packages/*.egg)
system-wide module path (~/usr/lib/python2.7/dist-packages/)
directories from PYTHONPATH
intermediate paths (omitted for brevity)
userbase directory (~/.local/lib/python2.7/site-packages/)
My problem is that I would need to put item 5. before items 3. and 4. for my code to work properly. Right now, if I import a module that was compiled against numpy 1.8.0 from the /local/* directory, and this module imports numpy, it will still take numpy from the /usr/* directory and fail.
I have circumvented this problem by placing something like this in my scripts:
import sys
sys.path.insert(0, '/local/python/lib/python2.7/site-packages/')
Thereby I can force Python to use the right import order, but of course this is not a solution, since I would have to do this in every single script.

Besides the suggestions already given in the comment section, have you thought about using virtualenv? This would give you fine-grained control over every module that you want to use. If you're not familiar with virtualenv you'll want to read the documentation to get a feel for how it works.
Purely for example, you could install and set it up, like so (virtualenv-1.11.6 looks to be the most recent version currently):
$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.11.6.tar.gz
$ tar xvfz virtualenv-1.11.6.tar.gz
$ cd virtualenv-1.11.6
$ python virtualenv.py ../numpyvenv
$ cd ../numpyvenv
$ source ./bin/activate
(numpyvenv) $ pip install numpy
# downloads, compiles, and installs numpy into the virtual environemnt
(numpyvenv) $ python
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.version.version
'1.9.1'
>>> quit()
(numpyvenv) $ deactivate
$ # the virtual environment has been deactivated
Above, we created a virtual environment named "numpyvenv", activated the environment, installed numpy, printed the numpy version (to show it works), quit python, and deactivated the environment. Next time you activate the environment, numpy will be there along with whatever other modules you install. You may run into hiccups while trying this, but it should get you started.

I had this problem on a Mac I was using without administrator access. My solution was the following:
Find the directory of the numpy version you want to use. For me this was /Library/Python/2.7/site-packages
Create a file ~/.startup.py and point to it with PYTHONSTARTUP=~/.startup.py in your .bashrc file
In .startup.py:
import sys
sys.path.insert(0,'/Library/Python/2.7/site-packages/') <--- imports this BEFORE the standard parts
import numpy
print("Importing numpy version"+numpy.__version__) <---- To remind that we have changed the numpy version
This seems to work fine for me. I hope it helps.

While a virtualenv seems the way to go, as of Force python to use an older version of module (than what I have installed now) you can also use a modification of
import pkg_resources
pkg_resources.require("Twisted==8.2.0")
import twisted

I had the same issue on Debian Wheezy after installing the latest numpy module with easy_install.
The new numpy module was installed in /usr/local/lib/python2.7/dist-packages/numpy while the old module was in /usr/lib/pymodules/python2.7/numpy. When I tried to import the numpy module, the older version was imported.
And as you say, adding to PYTHONPATH the new module path does not help, because is added in the sys.path below the older entry.
The issue seem to be in easy-install, because it creates a file easy-install.pth that imports /usr/lib/pymodules/python2.7 before any local module.
To fix the issue I just edited the file /usr/local/lib/python2.7/dist-packages/easy-install.pth and commented out the line /usr/lib/pymodules/python2.7 so this line will be placed below in the sys.path.

PythonAnywhere + virtualenv: "Could not find platform dependent libraries <exec_prefix>..."

I have a Django (1.5.1) site running on Python 2.7.3 in a virtualenv at PythonAnywhere. As far as I remember, everything used to work fine. Lately, although I haven't changed anything except some Django code, I get the following message when I run pip:
(venv)11:34 ~ $ pip
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
File "/*~*//venv/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "/*~*//venv/lib/python2.7/site-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 16, in <module>
import sys, os, zipimport, time, re, imp, types
ImportError: No module named time
Needless to say, pip does not work at all after producing the above error.
When I run python with the virtualenv activated, I again get the following error:
(venv)11:34 ~ $ python
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python 2.7.3 (default, Apr 29 2013, 15:12:04)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
File "/*~*//.pythonstartup.py", line 1, in <module>
import rlcompleter
ImportError: No module named rlcompleter
>>>
But, thereafter, the Python console appears to be in working order.
Without the virtualenv activated, python runs fine without any errors.
All was well a couple weeks ago when I last ran pip and installed some packages, but now, pip is not working, although all of my Django site's features run just fine. I appreciate any ideas that could get pip to work again.

There's definitely something fishy going on if it can't find module time - as far as I'm aware that module is actually built in to the Python binary itself so it's not as if there's a time.py or time.so file which could have been deleted. It looks as if either Python in your virtualenv has become broken somehow, or there's something about the environment that's messing it up.
Setting a dubious PYTHONHOME variable can cause Python all sorts of trouble as it can't find necessary files, but I think in this case it's the fact that the virtualenv has become broken. In fact, I just tried an old virtualenv on PythonAnywhere myself and got the same issue. It looks as if the symlinks that virtualenv creates have become broken as the result of an upgrade, possibly the recent upgrade from Debian to Ubuntu.
If at all possible I would simply re-create a new virtualenv and run things from there. In principle you could use pip to write a requirements file so you can create a new virtualenv with exactly the same versions of the code, but the problem is that you can't run pip to create this file, QED. There's probably a clever way to use the system Python to run pip and get the dependencies from your virtualenv, but it's going to be tricky - it's not designed to work that way.
Alternatively you could contact the PA devs - I know that some users did have problems with their virtualenvs around the time of the upgrade and they may have cunning scripts which can fix the problem. Even if you just build a new virtualenv and use it, I would contact them and make sure they know about this issue so they're aware of it for future upgrades.

I had this happen to me this afternoon. I upgraded from Mint 14 to Mint 15, and it appears that the system python on the former is 2.7.3 and the latter uses 2.7.4. I fixed this by deleting my virtualenv (which used python 2.7.3) and then recreated it using python 2.7.4.
Check my quick screencast. In it, I show that:
the python environment is broken
recreating the python env makes it work

I get this error this day.Because i use virtualenv,and enter this,but ,my python script still contains this
#!/usr/bin/python2.6
delete this sentence will ok