Jupyter Notebook

By Ryan Wiles

What is Jupyter Notebook

Note: Jupyter Notebook is being replaced by Jupyter Lab, but the Notebooks are compatible and the usage information in this tutorial is still relevant.

In a nutshell, Jupyter Notebook provides an easy-to-use interactive environment. It functions well as both an IDE as well as a presentation tool allowing you to add Markdown and HTML, including images and videos, to improve the presentation of your Notebook. Due to it’s interactive nature and export capabilities, it excels at prototyping and letting you share your work with others.

The description of Jupyter Notebook, according to the Jupyter Project, which is the group that made it is:

"The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more."

While their description is accurate and makes for a nice sales pitch, I find it a bit vague. Let’s try to add a bit more detail to their description so we’ll understand what’s actually going on. Try to think of Jupyter Notebook as somewhat like a browser based Python REPL session that can also some do other really cool things. While it can run on a shared web server, most people are just running the Notebook server locally. By default, when running locally the Notebook server will only accept unauthenticated network requests from the machine it’s running on at http://localhost:8888. Port 8888 is the default port for the Notebook server. It also listens at http://127.0.0.1:8888, but that URL always requires either token or password based authentication.

Jupyter Notebook runs on the iPython kernel. The Notebook browser client sends and receives messages with the Notebook server. The Notebook server relays these messages to the iPython kernel, which maintain the sessions with Python interpreter, executes the python statements, and returns the results to the browser through the server. There’s a bit more going on here than that, but at a high-level it makes for a useful abstraction of the behavior.

We’ll show some of the other cool things that Jupyter Notebook can do a bit later.

Prerequisites

Before installing Jupyter Notebook you need to have Python installed. If you’re one of those terribly impatient people who just have to try it out right now or would just like to see what the fuss is about before committing to installing something, you can demo Jupyter Notebook here: https://try.jupyter.org/

But since I’m going to pretend that you’re committed to this, we’ll forge on ahead with getting you set up for installation. Although, these steps should already be done if you’re somewhat following the Jupyter Tutorial’s outline, you are following it aren’t you? Well, if you’re not and jumped straight here your going to need Python installed, you can follow these Python installation instructions:

  1. Installing Python
  2. The Anaconda Distribution - Although Anaconda is not strictly required, it comes bundled with both Python and Jupyter Notebook and is their recommended way to install it.

Note: Jupyter Notebook requires Python 3.3 or greater , or Python 2.7

Installing Jupyter Notebook

The official install guide can be found here if you care to refer to it: https://jupyter.readthedocs.io/en/latest/install.html

However, both the easiest and their recommended way to install Jupyter Notebook is through installing Anaconda. For details about Anaconda including installation and usage, see The Anaconda Distribution

Alternatively, Jupyter Notebook can be installed manually using pip, though this is only suggested for experienced Python users, so I’m not going to elaborate on that process. The details for manual installation can be found here: https://jupyter.readthedocs.io/en/latest/install.html#id4

Jupyter Notebook Configuration

The Jupyter Notebook Config File

The config file for Jupyter Notebook is located located off your home directory at:

.jupyter/jupyter_notebook_config.py

This file is where Jupyter Notebook stores its non-default configuration settings. If you’re running without any custom configuration options, this file may not exist.

Creating a Missing Config File Directory

If either the directory or file are missing, the following command will create the directory and/or the config file if you really feel like having them around.

jupyter notebook --generate-config

Installing a Self-Signed SSL Certificate

Generally, at this point it would also be a good idea to configure a TLS/SSL Certificate and you can do so by creating a self-signed SSL certificate. I’m going to give the general procedure based on Jupyter’s instructions, but won’t expand on it right now. Modern browsers have started issuing warnings for non-HTTPS traffic as well as throwing alerts like this for untrusted SSL certs.

Browser Untrusted SSL Alert

Personally, I think being confronted with this alert can be a bit intimidating for a typical user. Later, I intend on coming back and doing a more thorough write up on how best to create and install a properly signed certificate that doesn’t cause a scary alert. You’re welcome to do this if you’d like or can wait for a procedure that doesn’t both annoy you and try to scare you away.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout jupyterNotebook.key -out jupyterNotebook.pem

jupyter notebook --certfile=jupyterNotebook.pem --keyfile jupyterNotebook.key

Setting a Notebook Password

At this point I would however recommend going ahead and setting up a password, since you know… that security thing is kind of a big deal these days. You can do this from the command line with the following command which will prompt you for a password, hash it and store it in the config file.:

$ jupyter notebook password
Enter password:
Verify password:
[NotebookPasswordApp] Wrote hashed password to
~/.jupyter/jupyter_notebook_config.json

If you do set a password, you’ll see this when you first start Jupyter Notebook:

Jupyter Notebook Password Prompt: Jupyter Notebook Password Prompt

Other Configuration Settings

I’m deliberately only including the options that I think are most relevant to typical users starting with Jupyter Notebook. The full list of options can be here: https://jupyter-notebook.readthedocs.io/en/stable/config.html

Config Option Description
NotebookApp.password Hashed password to use for web authentication
NotebookApp.password_required Forces users to use a password
NotebookApp.port Override the default port of 8888
NotebookApp.notebook_dir The directory to use for notebooks and kernels
NotebookApp.open_browser Whether to open in a browser after starting

Getting Started with Jupyter Notebook

Starting up Jupyter Notebook

If you installed using Anaconda, you can start-up Jupyter Notebook using either of the two following method. However, if you felt compelled to do the manual install, just follow the Command Line Startup.

Anaconda Navigator Startup:

  1. Open the Anaconda Navigator application Anaconda Navigator Jupyter Launch
  2. Click the “Launch” Button for Jupyter Notebook

Command Line Startup:

$ jupyter notebook
[I 22:54:57.238 NotebookApp] The port 8888 is already in use, trying another port.
[I 22:54:57.270 NotebookApp] JupyterLab extension loaded from ~/anaconda3/lib/python3.6/site-packages/jupyterlab
[I 22:54:57.270 NotebookApp] JupyterLab application directory is ~/anaconda3/share/jupyter/lab
[I 22:54:57.274 NotebookApp] Serving notebooks from local directory: /Users/notebook
[I 22:54:57.274 NotebookApp] The Jupyter Notebook is running at:
[I 22:54:57.274 NotebookApp] http://localhost:8889/
[I 22:54:57.274 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).

Note: The log messages will tell you what port this instance of Jupyter Notebook started on and what its base directory is.

Either of the above two startup options should open Jupyter Notebook in your browser as seen below. Though you if you set a password like suggested above, you should first see the password prompt in your browser.

Jupyter Notebook Main Screen

The Invariant Options

The Files Tab

As mentioned above, the Files tab presents you with a directory explorer that’s rooted in Jupyter Notebook’s base directory. Jupyter Notebook files end with a “.ipynb” file suffix, but you can use the directory explorer to preview and edit most text-based files. You can do most of the standard filesystem operations through this interface. Create/move/rename/delete files and directories in addition to duplicating/uploading/downloading most files.

Note: You can’t rename a Notebook while it has a running kernel, you have to shutdown the kernel, rename the notebook, and then restart the kernel.

Our First Notebook

Use the “New” button with the appropriate Python version 2 or 3, to create a new Notebook. Each Notebook opens in its own tab allowing you to have multiple Notebooks open at once. It should look something like this:

A New Empty Notebook: New Empty Jupyter Notebook

By default, the Notebook opens as a Code based Notebook, as can be seen in the toolbar dropdown, though there are other options such as Markdown.

In general, the interface should feel somewhat familiar to users who have used IDEs before. The keyboard icon to the right of the “Code” dropdown opens a useful command pallet. The “Trusted” icon indicates whether it will allow Javascript to execute.

One key difference and one that helps to set Jupyter Notebook apart are cells. You can see one in the image above indicated by In [ ]: and surrounded by an outline. We’ll describe them in more detail shortly, but I needed to introduce the term.

Edit Mode vs Command Mode

When viewing a Notebook, there’s are two different modes. When in the Edit mode, which is indicated by the pencil icon to the right of the “Trusted” icon, you can edit the contents of a cell. Hitting escape, similar to “vi” places you in Command mode. The currently selected cell will change from a green outline to a blue one and the pencil icon will disappear.

Useful Hot-Keys While in Command Mode:

Hot Key Description
H Show the Hot-Key help modal
Enter Change to Edit mode on the currently selected cell
P Open the Command Palette
S Save and checkpoint
F Open the “Find and Replace” modal
A Create a new empty cell above
B Create a new empty cell below
C Copy selected cells
X Cut selected cells
Shift + V Paste copied cells above
V Paste copied cells below
DD Delete the cell, like delete line in “vi”
Z Undo the Delete the cell operation
Y Change Cell to Code type
M Change Cell to Markdown type
R Change Cell to RawNBConvert type
Select cell above
K Select cell above
J Select cell below
Select cell below
Shift + ↑ Multi-cell select going upwards
Shift + ↓ Multi-cell select going downwards
Shift + M Merge selected cells or current with cell below
Shift + L Toggle line number for all cells
L Toggle line number for current cell


Useful Hot-Keys While in Edit Mode:

Hot Key Description
Tab Code completion or indent
Shift + Tab Show docstring (tooltip); description & params
Alt + Enter Execute current cell and insert cell one below
Shift + Enter Execute current cell and select/insert below
Ctrl + Enter Execute current cell
Ctrl + Shift + Hyphen Split the cell at the cursor
Ctrl + S Save and checkpoint
Command + / Comment
Command + ] Indent
Command + ] Un-indent

Umm… what’s a checkpoint? So, your Notebook has a file, we discussed that above. Jupyter, being the considerate interface that it is, however tries to keep you from losing your work. So even if you don’t save all the time, it basically creates a recovery file, kind of like MS Word tries to do, so that if something crashes or you unexpectedly quit, you don’t lose everything. The default checkpoint save happens about every 2 minutes, but hitting Ctrl + S forces a save to the file and an update of the checkpoint.

Exporting Your Work

After you’ve put all that hard work into creating your amazing widget, it’d be great to be able to share it with the world, or at least sent it to your boss. That way they won’t think that you’re completely slacking off. Luckily, Jupyter Notebook will let you download your results in a variety of different formats.

Download as

Jupyter Notebook Examples

Now, it’s time to show a few examples and hopefully demonstrate some of what make Jupyter Notebooks so special.

Obligatory Hello World

Hello World

Note: Don’t forget to hit “Ctrl + Enter” to run the cell

You’re Out of Order

Here, we’re going to demonstrate that Jupyter, by means of the underlying Python session, only cares about the order of execution of the cells, not their order on the page. If you think about it from an execution order perspective it all makes perfect sense.

Start with Three Empty Cells: Out of Order 1

Set Variable in the Last Cell: Out of Order 2

Show Variable in the First Cell: Out of Order 3

Print Variable in the Middle Cell: Out of Order 4

While this can be really handy when your exploring the data and figuring things out, it’s still generally a good idea to try to keep all the cells in geographically arranged in proper execution order. If you don’t, you’ll encounter errors when you attempt to do “Cells -> Run All”.

Questionable Object Documentation

If you add a question mark at the end of an object and run the cell, it will show you the documentation for that object. This functionality is courtesy of the iPython kernel. A double question mark attempts to show more even documentation and the source code if it’s available.

Showing Object Documentation: Show Object Documentation

Adding Some Eye Candy

Showing Graphics Inline: Inline Graphic Plot

What’s Next

Finding and Sharing Notebooks

You can share your Notebook with other by uploading it to a website or hosting on GitHub like many other people have: https://github.com/search?q=ipynb

The nbviewer website provides a great way to share your notebook and find others that have been shared.

nbviewer website

Jupyter Notebook References

© 2018 Ryan Wiles