Updated with information I forgot to include:
Canopy Version: 1.7.4.3348 (64 bit)
Python version: '2.7.11 | 64-bit | (default, Jun 11 2016, 11:33:47) [MSC v.1500 64 bit (AMD64)]'
Also: This problem only started a few days ago, but it's pretty irritating. It could be the network guys are doing something, or it could be I've altered my environment somehow without realizing it.
I have the habitual problem in Canopy that for some reason the prompt in the immediate window acts weird. It lets me type, but doesn't do anything and doesn't seem to advance to the next command.
Welcome to Canopy's interactive data-analysis environment!'t' not in 'tf'
Type '?' for more information.
...:
...:
n [1]: %run "C:\Projects\MyProject\MyProgram.py"
Enter (T)est or (F)ull or (Q)uit: t
How many rows to process: d
't' not in 'tf'
Out[2]: False
'a' not in 'tf'
Out[3]: True
In [4]:
...:
...:
...:
The only way I can get past this is to restart the kernel. I think something in my program might be causing this. Not sure what. Don't want to put the whole thing here. Anyone had this problem and know what kind of thing might cause it?
Also, what is this "mode" called and is there a way to get back to a regular "mode" without restarting the kernel?
Your firewall might be blocking ipython's communication with its kernel. Try disconnecting from the internet, disabling your firewall, and trying again. If this solves the problem then you'll need to configure your firewall not to block localhost, or use a smarter firewall program.
Related
I'm trying to load pykd.pyd in order to be able to use Python during Windbg crash dump analysis. This does not work, as you can see here:
0:006> .load C:\Python27\Lib\site-packages\pykd.pyd
The call to LoadLibrary(C:\Python27\Lib\site-packages\pykd.pyd) failed,
Win32 error 0n126
"The specified module could not be found."
Please check your debugger configuration and/or network access.
For your information, I have started, opening Windbg (version x86) and opening a crash dump file, and I can confirm that the mentioned pykd.pyd file is present.
If I put the filename between double quotes, I get another error message, as you can see here:
0:006> .load "C:\Python27\Lib\site-packages\pykd.pyd"
The call to LoadLibrary(C:Python27Libsite-packagespykd.pyd) failed,
Win32 error 0n2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
(It might be important to mention that both Win32 errors are different!)
Does anybody know what might cause this issue?
Thanks in advance
The Error Codes
The error message corresponds to the error codes. You can check that with the !error command:
0:000> !error 0n2
Error code: (Win32) 0x2 (2) - The system cannot find the file specified.
0:000> !error 0n126
Error code: (Win32) 0x7e (126) - The specified module could not be found.
So: nothing new here. WinDbg already told us.
The Quotation Marks
Since you are already debugging, let's debug this problem as well. Just know the right tool, which is Process Monitor.
Set up a filter like so:
"Process name", "contains", "windbg.exe", then "Include"
"Process name", "contains", "EngHost.exe", then "Include"
Now run the command
0:000> .load "c:\hello"
and you'll see that the quotes will mess up everything:
Now try
0:000> .load c:\hello
and you'll see that it searches in the right place:
Conclusion: .load is without quotation marks.
Loading PyKD
You are probably aware that there is 32 bit and 64 bit. And you can't load 32 bit DLLs in 64 bit processes nor 64 bit DLLs in 32 bit processes.
Same goes for debugging extensions: if you use 64 Bit WinDbg, you need 64 bit extensions. If you use 32 bit WinDbg, you need 32 bit extensions.
So first of all, check if your PyKD DLL (or .pyd here, which is actually .dll, just rename it) has the correct bitness (32 bit if I read that correctly).
Now, the 32 bit PyKD DLL will need a 32 bit installation of Python to run correctly. And likewise, 64 bit PyKD will need a 64 bit installation of Python.
Again you can help yourself with the right debugging tool. Process Monitor clearly shows that pykd.pyd is loaded successfully, but the dependency python38.dll (in my case, maybe Python 2.7 for you) is not:
For the following, I'm not 100% sure, but IMHO:
The PyKd DLL will try to find a Python system installation (as opposed to a virtual environment or venv).
That system installation must be in %PATH%
While you could have a 32 bit and 64 bit Python installation in the %PATH% variable, it will find one of them first. It might be the correct one or not.
Conclusion: put only one Python installation in %PATH% and use the correct bitness. Currently I only know this solution. Maybe the PyKD team posts an answer as well and explains how it can be done without modifying %PATH% all the time.
Don't load python package directly ( C:\Python27\Lib\site-packages\pykd.pyd ). It is a legacy unsupported way.
You need the special python bootstrapper for windbg:
https://githomelab.ru/pykd/pykd-ext
I have an assignment for a Deep Learning class, and they provide a Jupyter notebook as a base code, the thing is that after running the data import and reshape, jupyter notebook through a "Memory Error", after some analysis y tried to compile the same code in a normal .py file, and everything runs well.
The thing is that I'm required (preferably) to use the Jupyter notebook as the base for development, since is more interactive for the kind of task.
<ipython-input-2-846f80a40ce2> in <module>()
2 # Load the raw CIFAR-10 data
3 cifar10_dir = 'datasets\\'
----> 4 X, y = load_CIFAR10(cifar10_dir)
C:\path\data_utils.pyc in load_CIFAR10(ROOT)
18 f = os.path.join(ROOT, 'cifar10_train.p')
19 print('Path: ' + f );
---> 20 Xtr, Ytr = load_CIFAR_batch(f)
21 return Xtr, Ytr
22
C:\path\data_utils.pyc in load_CIFAR_batch(filename)
10 X = np.array(datadict['data'])
11 Y = np.array(datadict['labels'])
---> 12 X = X.reshape(-1, 3, 32, 32).transpose(0,2,3,1).astype("float")
13 return X, Y
14
MemoryError:
The error occurs in the line 12, i know is a memory consuming assignment, but that doesn't mean that 4 GB of RAM wont suffice, and that was confirmed when the code run without problems outside Jupyter.
My Guess is it has something to do with the memory limit either by Jupyter or by Chrome, but I'm not sure and also dont know how to solve it.
By the way:
I have a Windows 10 laptop with 4GB of RAM
and Chrome Version 57.0.2987.133 (64-bit)
I am only a year and 2 months late to this question. The technical answer as to why is really nicely explained here: https://superuser.com/questions/372881/is-there-a-technical-reason-why-32-bit-windows-is-limited-to-4gb-of-ram
It also implies why the conda solution works.
But for a lazy engineer's no-change workaround, close the Chrome tabs not absolutely necessary and restart your kernel so it starts afresh.
Kernel > Restart (& Run All)
Apparently this happens when the python installation is not the best.
As a matter of fact before solving the problem, I had installed on windows manually python 2.7 and the packages that I needed, after messing almost two days trying to figure out what was the problem, I reinstalled everything with Conda and the problem was solved.
I guess Conda is installing better memory management packages and that was the main reason.
Try running with Administrator privileges. Worked for me.
Similar thing happened with me while loading .npy file. Freeing up RAM solved the issue. It didn't have enough memory to load file into variables. Actually, both firefox and chrome was running on my system and closing firefox solved the problem.
Useful Commands:free -h
Note of precaution: before interpreting this command on your own. Its highly recommended to go through this page: https://www.linuxatemyram.com/ .
You can either reduce your dataset for training and testing this can solve your memory error problem.
I'm getting an intermittant segfault in python, which really shouldn't happen. It's a heisenbug, so I haven't figured out exactly what's causing it.
I've done the search and found that there was a known problem with an older version of python, but I'm using 2.7.10 (in a virtualenv, in case that matters)
I'm using pandas (0.18.0) , scipy(0.17.0) and numpy (1.11.0), in case the problem might be in there...
It looks like: How to generate core dumps in Mac OS X?
might be the best way to get stack trace...it appears in ~/Library/Logs/DiagnosticReports I'm not sure if it's USEFUL, and it's not a core per se, to be put into a debugger, but it's something...
I have weird problem that I cannot put my finger on. There is a program that I use (and contribute from time to time) that has colorized console output. Everything worked great until I reinstalled Windows. Now I cannot get colorized output.
This is the script that is used for colorizing.
I have managed to narrow down the problem to, more or less, simple situation, but I have no idea what is wrong.
This is console prompt that works as expected (string test is printed in red):
Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, r'c:\bin\SV\tea\src')
>>> from tea.console.color import cprint, Color
>>> cprint('test\n', Color.red)
test
>>>
But when I run following script with same version of python I get output test but not in red color (there is no color, just default console color):
import sys
sys.path.insert(0, r'c:\bin\SV\tea\src')
from tea.console.color import cprint, Color
cprint('test\n', Color.red)
The same setup worked before I reinstalled my system.
I have checked, environment variables in interactive mode and script are the same.
I have tried this in standard windows command prompt and Console, program that I
usually use.
OS in question is Windows 8 and before reinstall this was also used on Windows 8.
Same code with same setup works at computer at work (Windows 7).
I have Python 2.7 and Python 3.3 installed (as I did before). I have tried to run script
with calling python interpreter directly (c:\Python27\python.exe) or with py -2,
but it does not help.
IPython and mercurial colorizes output as it should.
Any ideas what can I try to make this work?
Edit:
Maybe it was not clear, but script I use to colorize output is given in a link in question. Here it is once again:
https://bitbucket.org/alefnula/tea/src/dc14009a19d66f92463549332a321b29c71d47b8/src/tea/console/color.py?at=default
I have found the problem and solution.
I believe that the problem was the bug in x64 ctypes module. I had Python 2.7 x64 installed and with that version following line (from script that I linked in question):
ctypes.windll.kernel32.SetConsoleTextAttribute(std_out_handle, code)
returns error code 6 with description The handle is invalid. After some investigation, I deduced that problem might be x64 version of python, so I installed 32-bit version and everything works as expected.
Since this solves my problem, and I do not have the time for deeper analysis I will leave it at this, just wanted to give some kind of resolution for question.
This code used to work on Vista (and Windows XP) but after an upgrade to Windows 7 it now fails with the error shown:
Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit (Intel)] on win32
>>> import _winreg
>>> h1 = _winreg.ConnectRegistry(None, _winreg.HKEY_LOCAL_MACHINE)
>>> key = r'SOFTWARE\Python\PythonCore\2.6\InstallPath'
>>> h2 = _winreg.OpenKey(h1, key, 0, _winreg.KEY_ALL_ACCESS)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
WindowsError: [Error 5] Access is denied
I'm fairly sure this is the result of changes in the security model in Windows 7, but various searches I tried have turned up nothing I can use as an answer so far.
(Not that it should be relevant, but to stave off "why would you do that?" responses, this is for a developer's utility which can switch the registry between multiple installations of Python, for use in a multi-project environment where we need more control over which version of Python is in use, and what packages are available, than things like virtualenv can provide.)
Edit: The logged-in user is an Administrator. Also, I've turned off the UAC (User Access Control) stuff as completely as one can (not true... see next edit), as was previously the case before the upgrade from Vista to Windows 7.
Edit 2: As noted in my own answer below, I hadn't rebooted after turning off UAC, so it was still set to the default. Apparently this results in the Access denied error (as I confirmed by testing with UAC set to Default and to Never).
This was a user mistake, compounded or triggered by changes in Windows 7 to how the UAC feature is implemented.
In Vista, the much-detested User Access Control feature was binary, either on or off. On Windows 7 that has been changed to provide four levels of granularity:
Always Notify (when either programs or user tries to change settings)
Default (notify only when programs try to make changes, and dim screen)
Notify without dimming (same as default but don't dim screen when notifying)
Never notify (for either programs or user changes)
My mistake was in not rebooting after dropping the UAC feature down to the Never Notify level. (Vista was aggressive about requesting that you reboot, while Windows 7 seems to be slightly more passive.)
I think you have an access right problem.
Try to open the key with a less demanding access right (e.g. KEY_QUERY_VALUE) and check if it works.
Of course, with that change you will not be able to change the registry, but it would be only for pinpointing the issue.
As an alternative, try to execute the utility from a user with higher privileges - and by the way this would be the only solution I could see for a problem involving access rights.