Data Science with Python in Visual Studio Code


This post was written by Rong Lu, a Principal Program Manager working on Data Science tools for Visual Studio Code

Today we’re very excited to announce the availability of Data Science features in the Python extension for Visual Studio Code! With the addition of these features, you can now work with data interactively in Visual Studio Code, whether it is for exploring data or for incorporating machine learning models into applications, making Visual Studio Code an exciting new option for those who prefer an editor for data science tasks.

These features as currently shipping as experimental. We’re starting our Visual Studio Code investments in the data science space with two main use cases in mind:

  • Exploring data and experimenting with ideas in Visual Studio Code. Just like how you would use Jupyter Notebooks to explore data, with Visual Studio Code you can accomplish the same but using a familiar editor with your favorite settings. You can define and run individual cells using the IPython kernel, visualize data frames, interact with plots, restart kernels, and export to Jupyter Notebooks.
  • Import Jupyter Notebooks into Python code. When it comes time to turn experimentation into reproducible, production-ready Python code, Visual Studio Code can make that transition very easy. Run the “Import Jupyter Notebook” command in the editor and code will be extracted into a Python file, then all the rich features that make you productive are at your fingertips – including AI-powered IntelliSense (IntelliCode), integrated debugger, Visual Studio Live Share, refactoring, multi-file management, and Git source control.

Now, let’s take a closer look at how Visual Studio Code works in these two scenarios.

Exploring data and experimenting with ideas in Visual Studio Code

Above is an example of a Python file that simply loads data from a csv file and generates a plot that outlines the correlation between data columns. With the new Data Science features, now you can visually inspect code results, including data frames and interactive plots.

A few things to note:

  1. Just like how you organize Jupyter Notebooks using cells, you can define code cells in your Python code by using “#%%” and Markdown cells by using “#%% [markdown]”.
  2. “Run cell” links start to light up once “#%%” are detected. The first time the “Run cell” link is clicked (or Shift-enter are pressed), the “Python Interactive” window on the right will show up and a Jupyter server will start in the background. Code in the cells will then be sent to the Jupyter server to execute and results will be rendered in the window.
  3. By using the top toolbar in the Python Interactive window, you can clear results, restart the iPython kernel, and export results to a Jupyter Notebook.
  4. You can navigate back to the source code by clicking on the “Go To Code” button in each cell.

Import Jupyter Notebooks into Python code

If you have existing Jupyter Notebooks that are ready to be turned into production-ready Python modules, simply bring them into Visual Studio Code by running the command “Python: Import Jupyter Notebook”: this will extract Python code as well as Markdown blocks from the notebook, and put everything into a Python file.

Here is an example of a Jupyter Notebook and the generated Python file. Each code cell becomes a code section with annotation “#%%”, and each Markdown cell turns into a comment section with “#%% [markdown]” annotation. Both cell types are runnable in Visual Studio Code, which means you can reproduce the exact same results that you would see in a Jupyter Notebook.

Try it out today

We’re rolling out these features as experimental in the latest Python extension for Visual Studio Code that shipped today, be sure to check out our Jupyter code cells docs for instructions on getting started. Please give it a try and let us know what you think by taking a 2-minute survey to help shape the features for your needs.

A quick side note, this is an evolvement from the Visual Studio Code Neuron extension that we worked along with students from Imperial College London this summer. The Neuron extension received a lot of positive feedback just recently, and now we’re taking their awesome work further by building more of such capabilities into the Python extension.

Have fun playing with data in Visual Studio Code! 😊


Comments (24)
  1. Cronan says:

    How would I access this? No details about it in the links …

    1. RongLu says:

      Thanks for raising this! This might be helpful: https://code.visualstudio.com/docs/python/editing#_jupyter-code-cells. We’ll update the blog post to include this link. Thanks!

  2. Thanks, this is great!

    1. Thanks, it’s great, can i change font or plot or image size in Python interactive?

  3. On Python Interactive window, the code I ran is not syntax-highlighted. It just turns yellow. Can I fix this?

    1. RongLu says:

      Which color theme are you using? We’re aware that the code may not be colorized correctly on certain color themes. Try the default light or default dark theme in VS Code, if that’s an option for you. Also, do you mind opening an issue on https://github.com/Microsoft/vscode-python/issues? We will look into fixing it shortly. Thanks!

      1. I’m using One Dark Pro color theme but then I try default light and dark theme, it doesn’t help.

      2. Updated. It works when reopening with a default theme. Thanks!

        1. RongLu says:

          Good to hear the default worked! Also, thanks for filing the issue on GitHub.

  4. JP Bee says:

    OMG, ahrrg. I haven’t used Microsoft software since I left the company 18 years ago. And right upfront, I tried out the Jupyter VSCode thing. Wow, won’t even start up. I followed a bunch of instructions (virtualenv config, jupyter is installed etc..) but it still errors out saying Jupyter is not installed. Meh.

    1. RongLu says:

      Hi JP,

      Can you check your install following this doc: https://github.com/Microsoft/vscode-python/blob/master/PYTHON_INTERACTIVE_TROUBLESHOOTING.md? If things still don’t work for you, can you log an issue on https://github.com/Microsoft/vscode-python/issues? Thanks!

      Rong

  5. Luis96xd says:

    This is excellent! I have already updated my Visual Studio Code, I have this new functions, Thanks!
    I have imported Jupyter Notebook code to Python Code and I saw the “Run Cell” new functinality.
    Everything working well. 😀

  6. nevenomi says:

    Hello,

    great stuff, we’re loving it in our data science team !
    However, the server starts in the file folder and not in the opened folder.
    Hope it will come soon, it’s frustrating !

    But keep up this good work 🙂

    1. RongLu says:

      Hi nevenomi,
      Just to confirm – “However, the server starts in the file folder and not in the opened folder.” <– your py file is *not* saved (Untitled) in this case, correct? We will be working on making the opened folder the working directory in such situations. However, if the py file is saved somewhere outside of the opened folder, the file location will be used as the working directory. Does that sound good to you?

      Thanks
      Rong

      1. nevenomi says:

        Hello Rong,
        As i stated in https://github.com/Microsoft/vscode-python/issues/3407
        I was merely stating the mismatch between python (workdir is project root) and jupyter (workdir is nb root).

        In Atom, Hydrogen has this choice as a feature, hope it will be picked up here too, since it promotes good practice in complex project folders.

        But apart from this, it’s a damn fine experimental feature 🙂

  7. Very nice work! I’m in love with it, but I didn’t figure out how to recreate a python notebook after editing it in VSCode. Is it possible to export back a python script with Jupyter tags to a Jupyter Notebook?

    1. RongLu says:

      Hi Paulo,

      Thanks for the kind words! 🙂 You can export to Jupyter Notebooks by first running the cells and then clicking on “Export” button in the Interactive window. See https://twitter.com/davorabbit/status/1063310234117001216. Directly exporting from py files without running first is something we will look into adding.

      Thanks
      Rong

      1. Thanks! I didn’t see that the save button would give back a notebook.

        I think I will use a lot these imported notebooks as my main file. I’ll still need to commit the .ipynb files to share the notebooks with others. Why not you suggest people from Azure: https://notebooks.azure.com/ to implement a remote executer for imported notebook files as Google Collab does with original notebooks?

        It would make the notebook format useless and VSCode the main interface for data science in python!

        1. RongLu says:

          Hi Paulo,

          Thanks for the suggestions! We’re looking at two things in the short term: 1) allow executing cells using remote Jupyter servers 2) allow exporting code into Jupyter Notebooks hosted in Azure (aka Azure Notebooks) – for sharing like you said. Does that match what you had in mind? Would love to hear if you have any other ideas! 🙂

  8. Babak Fard says:

    This is very exciting. Thanks. I am using BeakerX in Jupyter. It comes very handy when I want to look into my data in a dataframe. Is there any way that I can have that functionality in Interactive Mode (first option in this writing)?

    1. RongLu says:

      Hi Babak,

      Widgets are not yet supported in this version, but is something we’ll look into supporting in the future. Would you mind opening an issue here https://github.com/Microsoft/vscode-python/issues to help us track this request? Thanks 🙂

      Thanks
      Rong

      1. Babak Fard says:

        Just did. Thank you very much

  9. breathevalue says:

    As a user of different precursors of this behavior (vscode jupyter extension, atom’s hydrogen) — this seems really nice! I like where this is going! I can’t wait for support for remote jupyter kernels and interactive widgets to land!

    I take a little issue with this claim from the article though: “When it comes time to turn experimentation into reproducible, production-ready Python code, Visual Studio Code can make that transition very easy” — I’m not so sure about this. In my experience, code structured to work well inside a notebook doesn’t tend to translate cleanly into a normal python module due to a few issues:

    (1) with normally structured notebook code, import’ing causes side effects: in a notebook most of the code runs in module scope and is often intentionally put into module scope for interactivity wins
    (2) no way to externally parameterize any of the module-scoped functionality of a notebook: if I’ve defined my notebook normally with perhaps some parameters to control the behavior defined in variables on module scope (as seems pretty common) — just turning that notebook into a python module offers me no way to externally control any of the behavior of the module.

    Because of these issues, I find you often end up with some pretty extensive refactoring challenges even when the notebook code is well structured and these refactors are often challenging (because python is really hard to refactor in general). The .ipynb format conversion mentioned above does the work of getting the text out of the notebook — but the hard part of transforming that code into something that can be reasonably reused in another context is still ahead. Furthermore, once you’ve done that transformation, you tend to lose the ability to use/interact with the ‘productionized’ code easily in from a notebook style workflow — you’ll by necessity have undone the code structure that worked well for interactive use as you turned the behaviors that dependend on module scoped vars into functions with parameters.

    I made a module to try and address some of this — https://github.com/breathe/NotebookScripter — that module exposes a run_notebook(notebook_file, **args) method that allows one to treat a notebook as an invocable function into which you can pass arguments. It supports directly executing .ipynb files. I added support for executing .py files with the same execution model — I find it which works great in combination with this plugin!

    1. RongLu says:

      Hi breathevalue,

      Thanks for the detailed notes! Totally agree that refactoring code out of Notebooks isn’t an easy task, and extracting code into a .py file is only the first step. Even with the help of the refactoring features in VS Code, it can be a tedious job and requires much debugging time to get everything right. Therefore, I can totally see a need for wanting to externally parameterize a notebook as is without touching the code in the notebook itself. Your approach to turn Jupyter Notebooks into callable functions that can be parameterized externally is very cool! Great to see that VS Code support is added as well! Thanks and I look forward to seeing how users like the experience 🙂

Comments are closed.

Skip to main content