Introducing GEE and Geemap
1. Introducing GEE and Geemap#
Google Earth Engine is a popular cloud-computing platform used by the geospatial community. It has a multi-petabyte catalog of satellite imagery and geospatial datasets, allowing users to visualize, manipulate, edit and create spatial data in an easy and fast way. Built upon the Earth Engine Python API and open-source mapping libraries, geemap makes it much easier to analyze and visualize Earth Engine datasets in a Jupyter environment. Since its initial release in April 2020, geemap has become the most popular Python library for Earth Engine data analysis and visualization.
In this chapter, we will cover the basics of Geospatial Data Science, Google Earth Engine, and geemap. Then, we will walk through the steps to set up a conda environment and install geemap. We will also learn how to use geemap with Google Colab without installing anything on your computer. At the end of this chapter, we will look at some useful resources for learning Earth Engine and geemap beyond this book.
Before introducing geospatial data science, we need to understand what data science is. The term “data science” has gained a lot of attention during the past decade, along with related terms such as big data, data analytics, and machine learning. According to Google Trends, the online search interest over time in “data science” has experienced a rapid increase since 2016 (see Fig. 1.1). When we googled “data science”, 4.1 billion records were returned, compared to 7.6 billion on “big data”, 2.6 billion on “machine learning”, and 1.9 billion on “data analytics”. Interestingly, the interest in “big data” has been decreasing since 2018, while the interests in “data science” and “machine learning” continue to increase.
Okay, so what is data science? Data science is a broad term that encompasses many different areas of interest. From a high-level perspective, data science is the science of data or the study of data [Cao, 2017]. From the disciplinary perspective, data science is an interdisciplinary field that uses scientific methods, processes, algorithms, and systems to extract knowledge and insights from noisy, structured and unstructured data, and apply knowledge and actionable insights from data across a broad range of application domains [Dhar, 2013].
Geospatial data science is the discipline that specifically focuses on the spatial component of data science. It brings forth theories, concepts and applications that are specific to geographic data in the realm of data science [Hassan, 2019]. A good example of geospatial data science is NOAA’s analysis of spatial and temporal datasets (e.g., satellite imagery, weather data, and climate models) to provide actional hurricane forecasts using statistics, machine learning, and mathematical models [Eftelioglu et al., 2017].
Google Earth Engine (GEE) is a cloud computing platform with a multi-petabyte catalog of satellite imagery and geospatial datasets [Gorelick et al., 2017]. During the past few years, GEE has become very popular in the geospatial community and it has empowered numerous environmental applications at local, regional, and global scales [Amani et al., 2020, Boothroyd et al., 2020, Tamiminia et al., 2020, Wu et al., 2019]. Since GEE became publicly available in 2010, there has been an exponential growth in the number of peer-reviewed journal publications empowered by GEE (see Fig. 1.2). Based on the most recent bibliometric analysis, there are 698 peer-reviewed journal publications with the word “Google Earth Engine” in the title and 1,779 publications with the word “Google Earth Engine” in either the title or abstract. In 2021, the number of publications with “Google Earth Engine” in the title or abstract reached almost 800, which is more than a 20-fold increase from the year 2014 with only 4 publications.
The geemap package has some optional dependencies, such as GeoPandas and localtileserver. These optional dependencies can sometimes be a challenge to install, especially on Windows. Therefore, we advise you to closely follow the recommendations below to avoid installation problems.
To install geemap and its dependencies, we recommend you use the conda package manager. This can be obtained by installing the Anaconda Distribution (a free Python distribution for data science), or through Miniconda (minimal distribution only containing Python and the conda package manager). See also the installation docs for more information on how to install Anaconda or Miniconda locally.
Geemap is available on the conda-forge channel, a community effort that provides conda packages for a wide range of software. Creating a new conda environment to install geemap is not strictly necessary, but given that some geemap dependencies might have a version conflict with other geospatial packages in an existing conda environment, it is a good practice to install geemap and its dependencies in a clean environment starting fresh. The following commands create a new conda environment named
gee and install geemap in it:
conda create -n gee python conda activate gee conda install -c conda-forge geemap
First, open the Anaconda Prompt or Terminal and type
conda create -n gee python. Press Enter to create a new conda environment named
Next, activate the new conda environment by typing
conda activate gee and press Enter. Then, install geemap by typing
conda install -c conda-forge geemap and press Enter:
Geemap has several optional dependencies, such as GeoPandas, localtileserver, osmnx, rioxarray and rio-cogeo. It can be a bit cumbersome to install these optional dependencies individually. Luckily, these optional dependencies are available through the pygis Python package, which can be installed with a single command. Since pygis has many dependencies, sometimes it can take a while for conda to resolve dependencies. Therefore, we highly recommend you to install Mamba, a fast, robust, and cross-platform package manager. It runs on Windows, macOS, and Linux, and is fully compatible with conda packages and supports most of conda’s commands. The following commands install Mamba and pygis:
conda install -c conda-forge mamba mamba install -c conda-forge geemap pygis
To install Mamba, type
conda install -c conda-forge mamba and press Enter:
Once Mamba is installed in a conda environment, you can then simply replace any
conda command with
mamba. For example, to install pygis, type
mamba install -c conda-forge geemap pygis and press Enter:
Congratulations! You have successfully installed geemap and its dependencies. We will dive into geemap in the next chapter.
Geemap is also available on PyPI. It can be installed with pip using the following command:
pip install geemap
All optional dependencies of geemap are listed in requirements_dev.txt, which can be installed using one of the following:
pip install geemap[all]: installing all optional dependencies listed in requirements_dev.txt.
pip install geemap[backends]: installing keplergl, pydeck.
pip install geemap[lidar]: installing ipygany, ipyvtklink, laspy, panel, pyntcloud[LAS], pyvista.
pip install geemap[raster]: installing localtileserver, rio-cogeo, rioxarray, netcdf4, xarray_leaflet.
pip install geemap[sql]: installing psycopg2, sqlalchemy.
pip install geemap[streamlit]: installing streamlit-folium.
pip install geemap[vector]: installing geopandas, osmnx.
You may install the latest development version by cloning the GitHub repository with Git and using pip to install from the local directory:
git clone https://github.com/giswqs/geemap cd geemap pip install .
It is also possible to install the latest development version directly from the GitHub repository with:
pip install git+https://github.com/giswqs/geemap
If you have installed geemap before and want to upgrade to the latest version, you can run the following command in your terminal:
pip install -U geemap
If you use conda, you can update geemap to the latest version by running the following command in your terminal:
conda update -c conda-forge geemap
To install the development version from GitHub directly within a Jupyter notebook without using Git, run the following code in a Jupyter notebook and restart the kernel to take effect:
import geemap geemap.update_package()
To use geemap in a Docker container, check out the following docker containers with geemap installed.
Let’s activate the conda environment created in the previous section:
conda activate gee
Next, launch JupyterLab by typing the following commands in the Terminal or Anaconda Prompt:
JupyterLab will open as a new tab in the browser. Click the Python 3 icon in the top left corner of the JupyterLab Launcher window (see Fig. 1.9) or go to File > New > Notebook to create a new notebook. Select the newly created notebook in the JupyterLab File Browser tab and press F2 to rename the notebook, e.g., chapter01.ipynb.
Jupyter notebook has two modes: Edit mode and Command mode. Edit mode allows you to type into the cells like a normal text editor. Command mode allows you to edit the notebook as a whole, but not type into individual cells. Jupyter notebook has many keyboard shortcuts [Yordanov, 2017]. Here are some commonly used shortcuts. Note that the shortcuts are for Windows and Linux users. For the Mac users, they’re different buttons for
Ctrl: command key
Shortcuts in both modes:
Shift + Enter: run the current cell, select below
Ctrl + Enter: run selected cells
Alt + Enter: run the current cell, insert below
Ctrl + S: save and checkpoint
While in command mode (press
Esc to activate):
A: insert cell above
B: insert cell below
X: cut selected cells
C: copy selected cells
V: paste cells below
Y: change the cell type to Code
M: change the cell type to Markdown
P: open the command palette.
While in edit mode (press
Enter to activate):
Esc: take you into command mode
Tab: code completion or indent
Shift + Tab: show tooltip
You need to authenticate Earth Engine before using it. The package for the Earth Engine Python API is called earthengine-api, which should have been automatically installed by the geemap package as described in Section 1.5.1. Enter the following script into a code cell of a Jupyter notebook and press
Shift + Enter to execute:
import ee ee.Authenticate()
After running the above script, a new tab will open in the browser asking you to sign in to your Earth Engine account. After signing in, you will be asked to authorize the Google Earth Engine Authenticator. If this is the first time you are authenticating Earth Engine, click CHOOSE PROJECT to select a Cloud Project to use for Earth Engine:
You can either choose an existing Cloud Project or create a new one. If you choose to create a new Cloud Project, enter a project name, e.g.,
ee-your-username and click the blue SELECT button to create a new Cloud Project. If a red warning message appears at the bottom of the page, click on the Cloud Terms of Service link to accept the terms of service and then click the SELECT button again:
After selecting a Cloud Project, click the GENERATE TOKEN button to generate a new token. You will be asked to choose your Earth Engine account for the Notebook Client:
Click the Allow button to allow the Notebook Client to access your Earth Engine account:
An authentication code will be generated and displayed on the page. Copy the authorization code and paste it into the notebook cell asking for the verification code. Press Enter and the
Successfully saved authorization token message will appear beneath the authorization code you entered.
Congratulations! You have successfully authenticated Earth Engine for use in your Jupyter notebook. In general, authentication for local installations is a one-time step that generates a persistent authorization token stored on a local computer. The token can be found in the following file path depending on your operating system. Note that you might need to show the hidden directories on your computer in order to see the
.config folder under the home directory.
Windows: C:\Users\USERNAME\.config\earthengine\credentials Linux: /home/USERNAME/.config/earthengine/credentials MacOS: /Users/USERNAME/.config/earthengine/credentials
Once Earth Engine is authenticated, you can run the following script to initialize Earth Engine for a new Python session. In general, you will need to initialize Earth Engine for each new Python session, i.e., whenever you open a Jupyter notebook or Python script and want to use Earth Engine. Fortunately, geemap can automatically initialize Earth Engine for you when creating an interactive map, which will be covered in the next chapter. In other words, you rarely need to run
If you have difficulties installing geemap on your computer, you can try out geemap with Google Colab without installing anything on your machine. Google Colab is a free Jupyter notebook environment that runs entirely in the cloud. Most importantly, it does not require a setup and the notebooks that you create can be simultaneously edited by your team members - just the way you edit documents in Google Docs.
Click the Open in Colab button below to open this notebook in Google Colab:
Next, press Ctrl + / to uncomment the following line to install geemap:
# pip install geemap
After geemap has been installed successfully, click the RESTART RUNTIME button appearing at the end of the installation log or click on the menu Runtime > Restart runtime. Then type the following code in a new cell:
import geemap Map = geemap.Map() Map
Follow the on-screen instructions to authenticate Earth Engine. After that, you should be able to see the interactive map displayed beneath the code cell (see Fig. 1.15).
If you are trying to use geemap through a VPN. Use
geemap.set_proxy(port=your-port-number) to connect to Earth Engine servers. Otherwise, you might encounter a connection timeout issue.
import geemap geemap.set_proxy(port='your-port-number') Map = geemap.Map() Map
Display Earth Engine data layers on interactive maps.
Visualize Earth Engine vector and raster data without coding.
Retrieve Earth Engine data interactively using the Inspector tool.
Creating interactive plots from Earth Engine data by simply clicking on the map.
Convert data format between GeoJSON and Earth Engine FeatureCollection.
Use drawing tools to interact with Earth Engine data.
Use shapefiles with Earth Engine without having to upload data to one’s GEE account.
Export Earth Engine FeatureCollection to other formats (i.e., shp, csv, json, kml, kmz).
Export Earth Engine Image and ImageCollection as GeoTIFF.
Extract pixels from an Earth Engine Image into a 3D numpy array.
Calculate zonal statistics by group.
Add a custom legend for Earth Engine data.
Add animated text to GIF images generated from Earth Engine data.
Add colorbar and images to GIF animations generated from Earth Engine data.
Create satellite timelapse animations with animated text using Earth Engine.
Search places and datasets from Earth Engine Data Catalog.
Use timeseries inspector to visualize landscape changes over time.
Export Earth Engine maps as HTML files and PNG images.
Search Earth Engine API documentation within Jupyter notebooks.
Import Earth Engine assets from personal account.
Publish interactive GEE maps directly within Jupyter notebook.
Add local raster datasets (e.g., GeoTIFF) to the map.
Support Cloud Optimized GeoTIFF (COG) and SpatioTemporal Asset Catalog (STAC).
Perform image classification and accuracy assessment.
Extract pixel values interactively and export data as shapefile and csv.
Visualize land cover change with Sankey diagrams.
Load vector data from PostGIS.
Create publication-quality maps with cartoee.
To learn more about Earth Engine and geemap, check out the following resources:
In this chapter, we started by learning the basics of Geospatial Data Science, Google Earth Engine, and geemap. We then set up a conda environment to install geemap and its dependencies. We also walked through the steps to use geemap with Google Colab. By now, you should have a conda environment ready for using Earth Engine and geemap. We’re going to dive more deeply into geemap in our next chapter.