Is it possible to visualize data in pycharm in the same way that you can do in jupyter?
For instance in jupyter you can run a single line at a time and see the output which can be helpful for working with data sets.
Of course you can just use a function like head() or show() to see what is going on but you have to run the whole file (as opposed to one line at a time in jupyter) and it can make working with data a bit harder to understand.
Does anyone have any recommendations for me in terms of pycharm as that is what I am most familar with, or do you think it is worth me learning something like jupyter?
Why don't you use Atom and install the package Hydrogen? it offers you the same possibilities as Jupyter while working in a script (not a notebook).
You can execute the code line by line like in Jupyter by clicking ctrl+Enter or run it as a script. Here's the documentation of the package.
Atom is a light IDE so it combines both the power of Jupyter and PyCharm. I have used it and it is great and has so many packages like hydrogen, pep8 (helps to write a code that is conform to the pep8) and code beautifiers (for Python, R, JSON,etc), and a lot of great features.
I recommend learning Jupyter. Although PyCharm is an excellent IDE, running Jupyter in it looks clunky and something like this:
Related
My Original Error:
I am new to python and am using anaconda 4.8.3. When I try to autocomplete after math. or sentence. nothing shows up. I have tried installing both pyreadline and jedi, but both are already installed with anaconda apparently. I have not disabled or enabled anything outside of the normal process of learning to use conda, like setting up shells and feeling out how to use ipython/notebooks.
The Fix that I was able to find:
%config Completer.use_jedi = False
Put this line of code literally anywhere, I recommend making a separate text file for this specifically if you are having this issue, and you just need to run it before coding. You will need to do this every time that you open up the notebook but aside from that it is an easy fix. If you know anyone with this problem please share this with them. So far I have not found anything else that works for me, so if there is a more permanent option I would love to see it.
For ipython version 7.19.0 add the following to your ipython config file. default is at where your other profile files are lurking at
.ipython/profile_default/ipython_config.py
c.Completer.use_jedi=False
Looks like some stability issues are still being worked on.
ref IPython core.completer
Running on Mac Sierra, the autocompletion in Spyder (from Anaconda distribution), seems quite erratic. When used from the Ipython console, works as expected. However, when used from the editor (which is my main way of writing), is erratic. The autocompletion works (i.e. when pressing TAB a little box appears showing options) for some modules, such as pandas or matplotlib. So writing 'pd.' and hitting TAB, gets the box with options as expected. However, this does not happen with many other objects: for example, after defining a dataframe named 'df', typing 'df.' TAB shows nothing. In the Ipython console, 'df.' TAB would show the available procedures for that dataframe, such as groupby, and also its columns, etc..
So the question is threefold. First, is there any particular configuration that should be enabled to get this to work? I don't think so, given some time spent googling, but just wanna make sure. Second, could someone state what is the official word on what works and what doesn't in terms of autocompletion (e.g. what particular modules do work from the editor, and which ones doesn't?). Finally, what are the technical aspects of the differences between the editor and the Ipython console in the performance of the autocompletion with Spyder? I read something about Jedi vs. PsychoPy modules, so got curious (however, please keep in mind that although I have scientific experience, I am relatively new to computation, so please keep it reasonably simple for an educated but not expert person).
UPDATE: As a side question, it would be great to know why is the autocompletion better in Rodeo (another IDE). It is more new, has way fewer overall options than Spyder, but the autocompletion works perfectly in the editor.
(Spyder developer here)
My answers:
is there any particular configuration that should be enabled to get this to work?
In Spyder 3.1 we added the numpydoc library to improve completions of some objects (like Matplotlib figures and NumPy arrays). If Dataframe completions are not working for you (they are for me), please open an issue in our issue tracker on Github to track and solve this problem.
could someone state what is the official word on what works and what doesn't in terms of autocompletion (e.g. what particular modules do work from the editor, and which ones doesn't?)
The most difficult part is getting completions of definitions when an object is generated by functions or methods developed in C/C++/Fortran and not in Python. I mean, things like
import numpy as np
a = np.array([])
a.<TAB>
As I said, this should be working now for arrays, figures and dataframes, but it doesn't work for all libraries (and most scientific Python libraries are created in C/C++/Fortran and wrapped in Python for speed).
The problem is that the completion libraries we use (Rope and Jedi) can't deal with this case very well because array (for example) can't be introspected in a static way (i.e. without running code involving it). So we have to resort to tricks like analyzing array's docstring to see its return type and introspect that instead.
what are the technical aspects of the differences between the editor and the Ipython console in the performance of the autocompletion with Spyder?
The most important difference is that in the IPython console you have to run your code before getting completions about it. For example, please run this in a fresh IPython console
In [1]: import pandas as pd
...: df = pd.Da<Tab>
and you will see that it won't return you any completions for Da (when it obviously should return Dataframe).
But, after evaluation, it is quite simple to get completions. You can simply run
dir(pd)
to get them (that's what IPython essentially does internally).
On the other hand, Spyder's Editor doesn't have a console to run code into, so it has to get completions by running static analysis tools in your code (like Jedi and Rope). As I said, they introspect your code without running it. While they work very well for pure Python code, they have the problems I described above for compiled libraries.
And trying to evaluate the code you have in the Editor to get completions is usually not a good idea because:
It is not necessarily valid Python code all the time. For example, suppose you left an unclosed parenthesis somewhere, but you want to get completions at some other point. That should work without problems, right?
It could involve a very costly computation (e.g. loading a huge CSV in a Dataframe), so evaluating it every time to get completions (and that's a must because your code is different every time you ask for completions) could consume all your RAM in a blink.
it would be great to know why is the autocompletion better in Rodeo (another IDE)
Last time I checked (a couple of years ago), Rodeo evaluated your code to get completions. However, we'll take a look at what they are doing now to see if we can improve our completion machinery.
Autocompletion works correctly if there are NO white spaces in the project working directory path.
Autocomplete was not working for me at all.
So, I tried Tools -> Reset Sypder to factory defaults and it worked.
I'm learning some Data Science and for that I'm using Python with Jupyter Notebook. Which I think it's great for data analysis, mainly because it’s super easy to run step-by-step code. You can see everything that is happening.
On the other hand, to do more complex projects, like a web crawler or an Object Oriented program to extract information from an API, I’m using Sublime Text3. IMO it’s simple, clean, light… perfect. Also I think that .py is better than .ipynb for that (I don't even know if it's possible to do OO with Jupyter).
My problem now is integrating these two tools. The best I can do now is convert the dictionnaires in some .csv file and read it manually in Jupyter notebook. Obviously it doesn't sounds very smart and it is like a temporary solution just for experimentation.
This is the first time I'm dealing with a project which I need to integrate more than one environment and not only working with the same languages with all the files in the same folder etc. so I'm not very familiar on how to approach that.
If someone could explain the right way of integrating these two IDEs, how to make all the process more 'automatic', if it's better to use some database and then extract with SQL or something like that I'd appreciate very much.
PS: Also, if you guys have any material on how should a Python Data Science project be organized it would be awesome. Thanks!
I use ipython magic commands to help me switch between a text editor and a ipython notebook.
Specifically, I like to experiment with the code in the Notebook for the reasons you mentioned, and then when I'm ready to integrate it as a class in a bigger system I use the %%writefile filename.py command which will export that cell into a .py file.
You can also use %load filename.py and %run myfile.py to bring .py files into the notebook.
I am new to python. I have a very basic question. Is there a way I can execute part of a python program? ie some thing similar to Matlab where after running the code once, I can execute parts of the program
If you want something similar to Matlab, check out ipython - it comes with the goodness of python with a workflow similar to Matlab.
ipython has the concept of Notebooks which are composed of cells. These cells can be executed individually giving you the behavior you expect.
What you are looking for is called "cell" execution in MATLAB.
The Spyder python editor is in general a good approximation of MATLAB-style IDE for python. It supports executing the full script, the selected lines or a "cell" that is defined by a portion of code stating with a comment like
# %%
or
###
To get Spyder I suggest to install a scientific python distribution such as Anaconda or WinPython.
Alternatively, as pointed out by vikramls, you can embrace a more modern paradigm, convert your script to an ipython notebook and get "cell" execution for free.
PS The ipython notebook is a fantastic environment that allow to mix rich text, code and plots in a single document that is great for some workflows. On the other hand Spyder provides some unique features such as graphical variable inspector (a-la MATLAB), integrated HTML documentation and code error analysis not available in the notebook.
I could not understand the ipython library. This url provide the common feature but I could not core-relate it. http://ipython.org/ipython-doc/stable/interactive/tutorial.html
How to I use IPython to improve my day to day python application experience?
ipython is an improved interactive prompt, not a library. It has features like tab completion and profiles which are not present in the vanilla interactive prompt (which is running python without an input file). All features are listed on the page you cited.
So, it doesn't really improve your day to day python application experience (whatever that means), but it does provide benefits during development.
Also, there is an alternative, called bpython, it has quite great features, too.
do you do any scientific computing? ipython 0.12 has new functionality called ipython notebook that's really useful for data analysis. you can easily print graphs and data inline in your browser and reload your code. You can then print it as a pdf and make a nice report.
it's also useful for learning python due to ipython's functionality. you can quickly test and understand how certain functions operate. A few particularly useful functionality aside from tab completion
object?? gives you more information about the object
%history gives you a list of all your previous commands
%debug if you hit an error, this will put you into the debugger so you can quickly debug