Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.
Purdue University Purdue Logo Purdue Libraries

Data Science

Books and resources about Data Science

ArcGIS Notebook in ArcGIS Pro

What is ArcGIS notebook?

ArcGIS notebook is ESRI's Jupyter notebook environment, which provides the option to create python notebooks as part of the ArcGIS project and apply interactive spatial data analysis using python, spatial map, and data visualization. ArcGIS notebook was introduced in ArcGIS Pro in Version 2.5 in 2019 (The latest version by August 2021 is version 2.8). It is also available in ArcGIS online with the on-demand request. 

ArcGIS Notebook Environment Setup

Python environment setup is important in ArcGIS notebook to manage and install required python libraries for data analysis.

1. Go to the main interface of ArcGIS pro and click Settings to access the setting panel.

 

2. Under the setting panel on the left of ArcGIS Pro, click Python Then Manage Environments to create, edit, or remove python environments in ArcGIS Pro.

 

3. The default environment of ArcGIS notebook, "arcgispro-py3", cannot be modified.  To install third-party python libraries, we need to create a cloned environment by clicking the Clone Default button or click the Clone button of the selected environment. Then active the cloned environment by selecting the cloned environment and click the OK button. After changing the active environment, you may need to restart ArcGIS Pro to reinitialize Python and use the new environment.

 

4. With a cloned environment, you can install available third-party libraries listed in Python Package Manager, or check, updated and remove installed libraries. Here is the link for detailed instruction.

Note: Different from Jupyter notebook or some other online python notebook like Google colab, ArcGIS notebook doesn't support library package installation by writing installation code in the notebook (like pip install or conda install). Invalid library installations may disconnect your notebook from the ArcGIS server / disable notebook functions in ArcGIS Pro until the environment is restored. 

Python Libraries for Geospatial Analysis

Libraries for Data Science

Data Analysis:

Pandas ---data wrangling, manipulation, and visualization; https://github.com/pandas-dev/pandas

Numpy ---scientific computing and performing basic and advanced array operations; https://github.com/numpy/numpy

Scipy ---linear algebra, integration, optimization, and statistics; https://github.com/scipy/scipy

Data Visualization:

Matplotlib ---the foundation of Python data visualization, standard data visualization like Matlab; https://github.com/matplotlib/matplotlib

Seaborn --- A wrapper of Matplotlib with modern colors and styles and support complex visualizations; https://github.com/mwaskom/seaborn

Bokeh ---  interactive and scalable visualizations inside browsers using JavaScript widgets; https://github.com/bokeh/bokeh

Machine learning:

Scikit-learn --- Includes the most common machine learning algorithms; https://github.com/scikit-learn/scikit-learn

Keras --- neural networks and modeling; https://github.com/keras-team/keras

PyTorch --- A popular Python framework for neural network and deep learning with GPU acceleration; https://github.com/pytorch/pytorch

Tensorflow --- Another popular Python framework for neural network and deep learning with GPU and TPU; https://github.com/tensorflow/tensorflow

 

Libraries for Geospatial Analysis

ArcGIS Pro:

Arcpy --- geographic data analysis, data conversion, data management, and map automation; https://pro.arcgis.com/en/pro-app/latest/arcpy/get-started/what-is-arcpy-.htm

ArcGIS Python API --- Support web-based geospatial analysis, deep learning and big data analysis: https://github.com/Esri/arcgis-python-api

Independent Geospatial Libraries:

Geopandas --- add support for geographic data to pandas objects;  https://github.com/geopandas/geopandas

GDAL --- translator library for raster and vector geospatial data formats; https://github.com/Esri/gdal

Rasterio --- access and visualize geospatial raster data;  https://github.com/mapbox/rasterio/blob/master/docs/index.rst

Folium --- leverage leaflet (https://github.com/Leaflet/Leaflet) to build interactive web maps; https://github.com/python-visualization/folium

 

 

Create an ArcGIS Notebook

Note: ArcGIS Notebook is available in ArcGIS Pro Version 2.5 or later. 

1. Open ArcGIS Pro (ESRI account login is required to access ArcGIS Pro. Organization login is available to students, staff, and faculty at Purdue University by using BiolerKey). 

For users with a personal ESRI account, select ArcGIS login, then input your username and password to log in.

For users with organization access, select Your ArcGIS organization's URL then input your organizational link to ArcGIS Pro. It will direct the user to the organization account login portal. Then input your organizational username and password to log in. (Purdue's ArcGIS link is "purdueuniversity.maps.arcgis.com", then it will direct the users to BoilerKey login)

 

2. Create an ArcGIS project by clicking the Map icon, then name the project ("Notebook Demo" as the example in the figure below)

 

3. On the project interface, select Insert, then click New Notebook to create a blank ArcGIS notebook.

 

4. Now a new notebook is created in the project. A new folder named Notebooks is also created in the Catalog, where you can access, rename, or remove notebooks in the current project.

ArcGIS Notebook Hands-on Practice

ArcGIS Hands-on practice: Regional climate data wrangling and visualization

Introduction:

In this exercise, we will work on the WHIN station dataset (https://whin.org/about/) for data science practices from data wrangling to data analysis to data visualization. The WHIN stations capture daily weather data in ten counties (including Tippecanoe county) in Northern Indiana. While the daily weather records usually have some missing data from certain stations. In this exercise, we will apply two interpolation algorithms to fill those missing data in the database. After interpolation, we will compare the interpolated temperatures to the observed temperature (‘temperature’ column in the table) using python data visualizations and select the better algorithm to fill the missing weather data. We will also visualize the temperature data at the county level and discuss the temperature variation.

Data: 

Data 1: WHIN_current_02232021.csv

Data 2: WHIN_stations.csv

Data 3: WHIN_weather_0223_IDW_EBK_Counties.csv

Note: Data 1 and 2 are for data wrangling and temperature interpolation practices; Data 3 is for data visualization practice.

Source code:

ArcGIS Notebook Code: ArcGIS_notebook_hands_on_regional_climate_data_analysis_2021.ipynb

Python libraries: 

Arcpy (installed in default environment)

Numpy (installed in default environment)

Pandas (installed in default environment)

Matplotlib (installed in default environment)

Seaborn (installation needed)

 

 

Python in ArcGIS Pro

There are multiple ways to access python programming in ArcGIS Pro. The users can write interactive python code (.ipynb files) and regular python script (.py files) through different platforms and functions. 

 

Ways to access python:

Interactive python code (.ipynb files):

1. ArcGIS Notebook

2. Jupyter Notebook under ArcGIS Pro

Py scripts (.py files):

1. Python command line

2. IDLE (Python GUI)

3. Third-party platforms (Spyder and PyCharm)

 

How to access interactive python code (.ipynb files):

1: ArcGIS notebook as demonstrated in this tutorial

Image source: https://www.esri.com/arcgis-blog/products/arcgis-pro/analytics/introducing-arcgis-notebooks-in-arcgis-pro/

 

2. Jupyter notebook in ArcGIS Pro (Note: If you are using an organizational account for ESRI login,  be sure to access the Jupyter notebook under ArcGIS to access Arcpy and ArcGIS API, Not other jupyter notebook environments like anaconda)

 

How to access python script (.py files)

1. Python command line: users can write python directly using the command line and execute .py files

 

2. IDLE (Python GUI): The editor is to write the py scripts and the shell is to execute the py scripts.

 

 

3. Access py scripts through third-party platforms like Spyder and PyCharm. First, install Spyder / PyCharm using the Python Package Manager, then active Spyder / PyCharm using Python Command Prompt.

 

 

 

ArcGIS Pro vs ArcMap

Python versions: ArcGIS Map vs ArcGIS Pro

ArcGIS Pro uses Python 3.0+, while ArcMap uses Python 2.7. Python 3. and Python 2. are different languages and many functions are not compatible with each other. For ArcMap users, please check out here (list of ArcMap tools not available in ArcGIS pro:  https://pro.arcgis.com/en/pro-app/latest/tool-reference/appendices/unavailable-tools.htm ) to find the functions that are not supported in ArcGIS pro.

 

Learn more about python in ArcGIS Pro: https://pro.arcgis.com/en/pro-app/latest/arcpy/get-started/installing-python-for-arcgis-pro.htm

Learn more about the differences between Python 3 and Python 2: https://www.guru99.com/python-2-vs-python-3.html

Acknowledgements

This online tutorial development is supported by IndianaView Minigrant 2021 (https://www.indianaview.org/). The data used in the hands-on exercise is provided by Wabash Heartland Innovation Network (WHIN) (https://whin.org/). Thanks to supports from Purdue Libraries' GIS services (https://www.lib.purdue.edu/gis) and D-VELoP lab (https://www.lib.purdue.edu/d-velop).