Evaluating IDEs for scientific Python

Updated March 2020.

TL;DR: Spyder is a great IDE to start. For more advanced features, choose PyCharm.

Python is a general purpose scripting language that can be used for statistical analysis, numeric work, machine learning, and much more. If you’re coming from other scientific computing environments like Matlab, Mathematica or R, you might miss having an integrated development environment (IDE). An IDE combines editing, execution, plotting, debugging, etc. into a single coherent package.

Here I evaluate 4 IDEs for scientific Python on my Ubuntu 16 laptop to see how they stack up:

  • Spyder 3.3
  • Atom with Hydrogen 2.14
  • VSCode 1.42
  • PyCharm 2019.3.3

These IDEs combine a text editor, an integrated ipython shell or jupyter kernel integration, support for interactive plotting via matplotlib as well as several other features to tie everything together.

Criteria for evaluation

I’m a computational neuroscientist, and I work on different kinds of scientific workloads in Python, like computing summary statistics, visualization, ETL, machine learning. I was also a data scientist at Google and a BCI engineer at Facebook for a number of years. There, I also used Python on a day-to-day basis. This review is colored by these experiences. What I care about is:

  • A kick-ass text editor with:
    • good autocomplete
    • introspection of variables and Go To Definition integration
    • discoverability of features
    • Support for other formats, e.g. csv, markdown, etc.
  • Seamless integration of the ipython shell
  • Works well for both plain-jane python and interactive cell-mode
  • Bonus points for:
    • seamless debugging
    • git integration
    • smoothness

I am especially interested in support for interactive cell-mode executation. In conventional, plain-jane Python, you write a big script with a .py extension that gets executed all at once – the execution environment only lasts as long as the call to Python. In interactive cell-mode, execution is split into different cells, which can be executed in any order. Jupyter notebooks use this model; many IDEs, including all of the ones I review here, now offer similar features. Interactive execution is often the environment of choice for data science, where your analysis depends on the results you learn as you go along. For this reason, I place a lot of weight on how well this core feature works.

To put these IDEs through the paces, I’ve worked on an analysis that grabs webpages through Selenium, parses and aggregates them as csvs, uses pandas to munge the data, plots many things through plotnine, and learns a model or two with sklearn.

Spyder

Spyder is

a powerful scientific environment written in Python, for Python, and designed by and for scientists, engineers and data analysts.

One of Spyder’s biggest differentiating features is that it features the ipython command line front-and-center. To execute Python code that you create in the editor, you can use any of a number of shortcuts, including:

  • F9 to run a selection
  • F5 to run a whole file
  • Ctrl+Enter to run a code cell
  • Shift+Ctrl+Enter to run a code cell and advance to the next one.

New cells are defined by # %%. Cell execution is nearly instantaneous, and the edit-print-edit loop is very smooth. matplotlib plots show up either in a separate window or inline inside the ipython editor.

Spyder’s editor is excellent. It offers good introspection, highlights errors, gives warnings, and opens up the docstring information upon calling a function. The auto-complete is mostly good but sometimes has inexplicable misses – I haven’t quite figured out why some variables autocomplete inside the ipython command line whereas they won’t inside the editor.

The editor can find the file/line where a function was defined by holding the Ctrl button and clicking the function name. It has support for setting breakpoints within the interface, and feeds that information to the pdb debugger. pdb can then be manipulated via its command line interface inside ipython: c to continue, u to go up a level, q to quit, etc. The editor also integrates help and lists all currently existing variables.

Its editor doesn’t quite stack up to modern infinitely customizable editors in terms of features. Its features are not easily discoverable because it doesn’t have a text search over commands. But honestly, this is all kind of immaterial: Spyder works. It’s polished, it’s fast, it doesn’t try to do too many things. I mean this in the nicest way possible: it’s the closest thing to RStudio or Matlab in the Python world.

Pros

  • Very fast
  • Polished
  • Ipython integration works well
  • Batteries included

Cons

  • Not very discoverable
  • Doesn’t support many other languages apart from Python
  • Dark mode still has a lot of white chrome in the window, hurts my eyes.

Total note: 4.5/5

Atom with Hydrogen

Atom is

a hackable text editor for the 21st Century

It’s first and foremost a powerful text editor that, with extensions, can be used for interactive Python workloads. It supports many plugins – anything from a spell checker, git integration, Markdown rendering and even an embedded color picker (because why not?). You can install the terminal-plus plugin to access a terminal, which will then give you a way to run Python inside the editor. To get a debugger, install python-debugger.

You can install the Hydrogen plugin to access to an integrated interactive cell mode for Python and all the other languages supported by jupyter. You write code separated by # %%, press Ctrl+Alt+Enter to execute a cell, and voila! You see the results (plots, dataframes, text) inline with the code. It’s all very slick. It uses jupyter kernels behind the scene. One very neat feature is that it can use remote jupyter kernels – that means you can interact with a kernel inside the editor even if the kernel is remote. Sweet!

There’s a lot of different features you can use, which can feel a bit overwhelming at first. Thankfully, Atom has an excellent command palette: Ctrl+Shift+P brings up a search window which allows you to search through all available commands. Type hydrogen to see all the Hydrogen-specific commands.

The command palette in Hydrogen

Setting watches will allow you to look at the content of a variable in a separate pane. The hydrogen-python plugin will give you a variables explorer as well. All-in-all, it’s a pretty interesting way to use Python interactively.

The one thing I find missing is a true ipython command line. If I want to see the order of execution of my code, I’m out of luck. Similarly, if I want to see the help for a command, I can’t just type command? in a terminal – I have to create a new code cell, look at the help, and then potentially immediately delete the code cell. It’s a bit inelegant.

Although the text editor in Atom is very good, the autocomplete and the introspection is sub-par compared to three other editors here. Combine this with a fragmented ecosystem, and it’s not exactly the most user-friendly solution. Overall, although I like Atom a lot, and it’s very customizable, it’s not my top choice for interactive Python work.

Pros

  • Very customizable
  • Innovative cell mode
  • Plugins for everything
  • Easily discoverable
  • Remote kernels

Cons

  • Lots of things to install – not very beginner friendly
  • Lackluster autocomplete and definition search
  • No real ipython command line brings me great sadness

Overall score: 3.5/5

VSCode

VSCode is a free text editor from Microsoft. Like Atom, it’s first and foremost a text editor with easy plugins. It has as command palette that can be pulled up with the same shortcut (Ctrl+Shift+P). Like Atom with Hydrogen, it has an interactive Python mode that uses Jupyter kernels in the background.

This interactive mode fixes a lot of the issues in Atom. The results of running cells show up in a different window, and you get access to an actual command line (🙌). The boundaries between the different cells show more prominently. Furthermore, introspection, finding definitions, and autocomplete are all a step above Atom.

However, the interactive Python mode is slow as molasses on my computer. It can take 2 seconds between hitting Ctrl+Enter and the command showing up as executed in the side window display. This is a real bummer, because vanilla VSCode itself is super fast. It’s specifically the interactive Python mode that is slow. I have a 2017 Dell XPS 13 laptop – a perfectly respectable machine, if a little outdated – and the sluggishness really gets in the way of my productivity.

I’m keeping my fingers crossed that six months down the line, the performance issues will have been fixed by MS, because I think it’s a compelling offering. In the meantime, not my top choice.

Pros

  • Very customizable
  • Cool split mode with Jupyter kernels
  • Easily discoverable
  • Plugins for everything
  • Great introspection
  • Emulates ipython command line

Cons

  • Sluggish performance

Overall score: 3.5/5

Pycharm

PyCharm is

The Python IDE for professional developpers

Unlike the three other IDEs I discussed before, this one is not free – it’s 89$US a year. If you have an academic affiliation however, you can get a free license.

PyCharm’s best feature is its autocomplete and introspection. It really is in a class of its own. When you’re learning a new language, having good autocomplete can make the difference between mediocre and acceptable productivity.

It has first-class support for Python, no plugins needed. Debugging, listing variables, git, a dedicated plot window, it’s all here. However, the interface is, in my opinion, anxiety-inducing. It’s so feature-packed that it can be hard to find the feature you want. Let’s say you want to rename a file, for instance. Forget about F2, you need to use Refactor > Rename (Shift+F6). The shortcuts are completely different than every other piece of software I’ve used. One redeeming factor is the Command Palette, Ctrl+Shift+A, in which you can search from hundreds of commands.

The interactive Python mode is slightly different than the last three IDEs. It does not work on .py files. Rather, it pops up when you open an ipython notebook (extension .ipynb). It then gives a split view – on one side, you see the textual representations of cells. On the right-hand side, you see the code as it might appear in Jupyter, as well as its output. It’s an interesting take on the concept. Performance is quite good; however, it sometimes slows down dramatically after a couple of hours. A restart takes care of the issue.

Overall, although PyCharm is too big for its own good, you can’t argue with the excellent introspection – the autocomplete is flawless, the inline help is very good, it will highlight subtle errors as you write your code and offer fixes, etc. The interactive cell mode is quite good – although it could use a command line. All-in-all, a great choice if you’ve started to outgrow Spyder.

Pros

  • Every feature in the book
  • Introspection is top-notch
  • Refactoring, debugging, you name it
  • Cool cell mode

Cons

  • Anxiety-inducing in its complexity
  • Absurd default keyboard shortcuts
  • Sometimes slows down for no good reason

Overall grade: 4/5

45 comments

  1. Hey,

    Thank you for a valuable post on different IDEs. I personally think that Spyder is the best one because I work in both Windows and Linux. It works fine in both OS’. However, on some of the older computers in our lab (i.e., for runnin psychological tasks) Spyder is incredibly slow. Last year I found the IDE Rodeo. It is, sort of, a clone of the R IDE Rstudio.

  2. Thank for your sharing.
    I have a question: why my iep’s code colr not the same with you.Is there any setting?thanks.

  3. Thanks for a great Overview. Im also using python for scientific research, and this has been very helpful. Iv’e tried several python IDEs in the last month or so, and still havent find the best for me. My needs, in importance order:

    1. Very good interactivity and plotting abilities
    2. Seamless debugging – setting breakpoints in the editor, ability to examine variables while in breakpoints, ability to interactively code while in breakpoints.
    3. Good editor, good IDE environment, Good project management capabilities

    These are my impressions, I’ll be glad to here your opinion:

    1. Iv’e started with the IPython notebook. as fun as it was to add notes and explanations while coding, and as beautiful as the outcome appear, it is not suitable for researching projects in general, as the notebook gets larger and not coherent very fast. I sometimes try many methods before I find the right why to go, and I dont need all the trails in my notebook. IPython notebook is great to summarize your work once you figured what to do and want to record the process and explain the research path.

    Also, the notebook is not meeting any of the criterias above reasonably. Interactivity is limited. yes, you can right code and see the output immediately, but that code is staying in your notebook and catches many cells you dont want to be seen. Debugging is a pain also. very very limited and not friendly. Managing the code is also not easy, copy pasting cells or moving cells around is done cell by cell, which can be very annoying when trying to move many cells. Eventually, the IPython notebook is really great for recording the research progress and sumrizations, but not for the project advancements it self.

    2. So then I tried Spyder. For my taste, spyder is a beautiful IDE. I loved the HTML look, the instant info on function and the overall design. The F9 to run a code is very handy, and it meets objective number 1 very good (4/5) I disliked the fact that one changing a function in the editor, I need to mark it and do F9 for the change to take effect in the console, but that was minor. My biggest problem with spyder was the debugger. I dont know how you scored it 4/5, I would score it 2/5. It is a simple PDB debugger. while PDB might be very powerful for users how knows it good, it is not a GUI debugger and for me can not serve as one. Also, the debugger in spyder is not consistent and tends to halt. I also find it difficult to examine local variables while in BP which misses completely the power of debugging in scientific research. To sum it, Spyder is great when I dont make mistakes, but when I do make mistakes, It is much harder to track them in spyder then it is on WING.

    3. So I tried WING IDE next. It was good. very nice overall development environment. very suitable for big research projects. very nice code completion and re factoring capabilities. The debugger is very good too. The only problem with it is that is a little ugly.. the GUI colors selection is awful, the toolbars are ugly, the help documentation is not readable without struggling, fonts are ugly too… and also the interactive capabilities and minimal. No integration with an IPython console, so graphing is less seamless (figures are created in a separate window) and no integration with Pandas. So it is reasonable, but It is not fun. and we want to enjoy the aesthetics, make us more productable, isnt it so? :)

    So Im still looking for the best IDE for me. I going to try pydev with IPython now. I have a feeling it might be the winner for me. Would appriciatie comments. did I miss anything?

  4. Hi there, just wanted to say that the interactive console in PyDev is getting a lot of attention and the latest release improved on that area a lot (there are details at http://pydev.blogspot.com.br/2013/11/pydev-30.html — and it plays well with matplotlib now). This area is also expected to improve even more in the upcoming releases (it may be interesting following DAWN too: http://www.dawnsci.org/, which bundles PyDev and is targeted at scientific data analysis).

  5. thanks for the review. It’s nice to see my favorite IDE scores highest in your comparison (spyder). With respect to the plotting I wish to see something like RStudio, where figures can be docked, but that’s just details. I have been using it for 3 years now, mainly on linux, occasionally on windows.

    The reason I looked for (different) IDE’s is that Spyder keeps crashing on my plots. I use multiprocessing and matplotlib with gtk backend to update a master figure. Other backends do not update plots when used with multiprocessing, and gtk often crashes my IDE after the process is finished and I try to make some edits in the script. Anyway, I didn’t find any better alternative than Spyder and after this blog I don’t think I’ll look further. I’ll look into the iPython integration, I never looked into that (not enabled by default on my installation).

  6. Patrick – I installed spyder on windows 7 and could not find any link or icon to an executable.
    It just disappeared which is about the most pointless install I have ever seen. Where to they put the link to run the app – its not in programs (as it should be). It doesnt install in registry etc. Really frustrating.

    • Hi dantheman, I’m an Spyder developer. May I ask you how you installed Spyder? Did you use a scientific distribution (PythonXY, WinPython), our Windows installers or pip?

    • Since this is a question from a year ago, I think my answer will serve people who have the same question and come across here.

      You can launch it by going to windows DOS prompt mode. Simply click start, type in cmd in the search bar and launch command mode, then type spyder to launch it. Or maybe simply type spyder in the search bar and see if you can launch it directly.

  7. Whatever you do, don’t purchase the KomodoIDE for Mac. It boasts lots of features that do not work. Save your money or purchase PyCharm from Jetbrains. They have excellent support and the product works well. Don’t waste your time and money on Komodo IDE 8. The product is JUNK!

  8. (Disclaimer: Spyder dev here). Hi Patrick, thanks a lot for your evaluation. I’ll take into account your suggestions for our next release (about to be released):

    1. I’ll remove unnecessary toolbars to diminish visual clutter. This is a real problem with IDEs, i.e. they could be cluttered quite easily in their attempt to visually show the user all the cools things they can do. But you’re right: most of those things are not constantly used, so they can hidden by default.

    2. I didn’t know tooltips were so annoying. They are meant to show only signatures because for docstrings there is our Object Inspector (which shows them in rich text). I think the best thing to do would be to join them to the completion widget, as PyDev and Rstudio do. Another option would be to leave them open until the user starts to write some text. What do you think would be the best option?

    • 1. I think by default the toolbars should be arranged by default like so, starting from the left:
      New, open, save, save all (no print button). Run (contextual, so that when a selection is highlighted it runs the selection and not the whole file), debug, config. You really don’t need more than that. All the tooltips on hover should show the corresponding keyboard shortcuts.

      The working directory toolbar should be integrated into the “project” panel. The file explorer panel should not be shown by default, since most of the time it’s redundant with the project panel. The Outline panel should be integrated with the object inspector and variable inspector. Object inspector should be renamed “online help” or something similar. History log should be integrated with the console panel. The Options button in the editor should only show actual viewing options, like tile vertically, new window, and the like. Every panel comes with an options button, yet some have “Options” written next to the button, others not. Choose one.

      2. I like PyDev’s solution, personally. You could show a short version of the docstring and keep the full version for the object inspector panel. You could have object inspection tied to highlighting, or to a keyboard shortcut (say, shift+F1 for contextual help). Also, if you could have an “open definition” option that would open the definition of a function/class on highlight, like Matlab’s Ctrl+D, that would be great.

      • 1. I’ve made most of the changes you suggested to our toolbars in 2.2. But to see them you have to remove your ~/.spyder2 dir and restart Spyder.

        I don’t think making “Run” contextual is a good idea, but I’ll give it more thought when we introduce cells in 2.3.You’re probably right about the run buttons that should be changed by “play” ones, but we are not gonna do it for 2.2, maybe for 2.3.

        It’s not easy to add keyboard shortcuts to our tooltips because all of them are configurable. But it’s something we could try for 2.3.

        The Working directory toolbar controls the working dir for all our consoles (Python and IPython), so I don’t think it’s a good idea to add it to the project plugin. Besides, Matlab also has it as part of its main toolbar, so it’ll give some familiarity to its users.

        Since our project functionality is not too strong, it’s better to leave file explorer where it is now.

        The outiline explorer is designed to let you navigate your code, so I think it’s better to leave it next to the Editor (as it is now).

        The object inspector will be renamed to “Online help” when we add some documentation on how to use Spyder in 2.3. For now it only inspect objects.

        I think you are right about the Editor options menu and the “Options” label in some buttons and not in others. I’ll fix it this week.

        2. I have an idea on how to improve our tooltips so that they work as in other IDEs (but that’s work for 2.3). Although I don’t like PyDev’s tooltips (because when attached to the completion widget they took to much screen space), we could add a config option to made them work that way.

        There is already an “open definition” functionality: Just keep pressed Ctrl and move the mouse to the function/method you want to inspect. You’ll see that it will be highlighted as a browser hyperlink (i.e. its color will change to blue and will be underlined). If you left click on it, it will open the file where it is defined. I know it’s not easy to find :)

    • Also the icon for Run should be a green arrow pointing towards the right (“Play”) rather than a little running man. It’s standard for “Run” to represented by a play button.

    • I tried the latest version. I like it a lot – definitely seems more polished. I noticed little things like the fact that it shows an icon in the Ubuntu menu, whereas it didn’t before, and it’s now the Programming menu rather than Menu. I dig the new toolbar arrangement, very much like what I had in mind. I see that you’ve changed the behaviour of the outline panel too, I like it (although below everything else might not be the best default placement).

      The only thing I didn’t like about the new default configuration is that the iPython console is closed by default and the python console is open. I think it should be the other way around. It might be good to have the ipython console open by default with load_ext autoreload, autoreload 2 running by default to facilitate the transition for Matlab users. Excellent job.

      • First of all, I’m glad you liked it my changes :) Your criticism was very up to the point and helped us to polish much more our interface.

        About IPython: I was just thinking this is the way to go, i.e. show the IPython console and hide the Python one, so you’re basically reading my mind! :) This would help to avoid confusion on having two evaluation interfaces and promote IPython as our preferred one. However we didn’t want to delay 2.2 more time so we’ll do this change in 2.3.

        I’ll see what we can do about autoreloead, seems very interesting.

    • Actually I just tried the project view and I was very confused about the project/workspace distinction. I created a workspace and a project in the same directory and I was expecting the files to show up in the project panel. Rather, I don’t see the files, and when I try to import the directory specifically it says that it’s already open. I have no idea what’s going on.

      • Projects are our weak point right now. They are unintuitive and not well integrated with our other plugins. We’ll work on improving them in 2.3.

        My main idea is to copy the way RStudio defines projects, which seems very well thought.

    • ANNNNND it just showed up after I changed the workspace directory (I think?) so that it was one folder up from the folder I was interested in using.

    • I haven’t, but it doesn’t look like it has a set of features particularly aimed at science. I don’t plan on reviewing other IDEs unless they claim integration with matplotlib, ipython or dynamic interaction with the Python console.

  9. First the disclaimer: I work for Enthought as a developer, although not on Canopy itself at the moment.

    I’m sorry you ran into such profound difficulties with Canopy. Right now Canopy Linux (unlike the OS X and Windows versions) is in public beta and has been out for less than 2 weeks, so it’s not entirely surprising that there may be issues with it. Please help us diagnose and fix them via Enthought’s support system – there’s a fair chance that any common issues already have a known work-around.

      • Apologies for your troubles with Canopy. I work at Enthought and have been trying to reproduce your problem but on a machine with Ubuntu 12.10. I installed the 64 bit version and although I do get the message “QGtkStyle could not resolve GTK. Make sure you have installed the proper libraries.” Canopy does launch and run fine. I have tried this with Unity and also with the Gnome classic interface. A colleague has tested Canopy on 12.04 and has not had problems. If you have any additional debug information on the issues please do let me know. For example, do you have your PYTHONPATH or LD_LIBRARY_PATH env vars set to anything?

        Thanks.

  10. Now I am curious, I wonder if I would be able to make Canopy run… your article left me a bad taste :/ Thank you for the other three anyway! Right now I’m doing well with IEP because it’s smooth and lightweight, it’s what I recommend to everybody.

    • I’ve been using Canopy for a MOOC where it was suggested as the default environment. Sadly, no debugger yet, which is a big downside. Also, the steps to handle abort are pretty clunky.

  11. Yes, I’m surprised there was no mention of iPython. It’s not an IDE, because it has no editor, but it’s so well integrated with scipy and matplotlib that it’s a very nice way to do Matlab-like work in python.

    • I really like ipython. I talk about using ipython + using your favorite editor in the PyDev section. It’s a workable solution, but there’s a lot of advantages in having the interpreter integrated with the editor, as is the case with IEP and Spyder, like the ability to highlight a snippet of code in the editor and run it, and having the editor open the relevant file and highlight the right line after an error/when debugging.

  12. While I know it’s not technically an IDE, you should really consider the IPython notebook. It’s really what converted me from working mainly in Matlab to almost exclusively in Python. Since a lot of what I do is largely interactive work, stumbling my way through ideas and results, having a log of all of that in notebook form is great.

      • Really? you really think is great? I LOVE the notebook idea but i cant see how to use it besides to sum up a project. I find it improper for fast algorithmic develpoment (as I expect from a matlab replacer). Maybe Im using it wrong?

        What im lacking in comparison to matlab:
        1. a “running a selection” option. When im trying something new, I dont want the entire cell to be run. and since the alt+enter not working for me (for some reason) i cant enter the middle of the notebook and easily start trail an error session.

        2. Also, I dont like to “mess up” the notebook with trail and error code. I would happily do that in “aside” interpeter that knows all the workspace variables, but not in the notebook…

        3. No debugger, No variable watch. How can someone live with that? What if i want to run code to do some funcionality, and run it command by command and make sure in each step variables holds what they suppose to hold? in Ipython, that would mean printing a variable after each step, which again, mess up the code. i find this lack of “watch” to really slow me down.

        How do you get over this? Am I using it wrong? because it is a beautiful tool that i cant really seem to use as productivly as matlab. Ill aprriciate any comments.

        BTW, if the notebook could contain another regular Interpretor that knows the workspace variables and could run commands aside of the notebook, that would segnificantly improve the workability for me..

    • I tried it for a day or so. The integration with the Python shell is minimal, that is, you can run a full file, but you can’t highlight a snippet and run it directly. The interface is bloated IMHO. The editor is all right, on par with Spyder. Overall, it’s more for the hardcore developer than for the casual scientific user.

Leave a Reply to ccordoba12 Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s