Amazon Web Services

Digital Earth Australia store our data products in Amazon Web Service’s S3 Service, with free public access. In addition to data, there are a number of services including an interactive code environment using Jupyter, a metadata explorer and web services available. This page provides technical documentation for using the DEA data and services on AWS.

Data and services listing

S3 details

Bucket

Purpose

s3://dea-public-data

Stores all the public DEA Data (more than 1PB and 60M objects in August 2023)

s3://dea-public-data-inventory

A public AWS S3 Inventory of DEA Public Data. Provides a faster and more efficient content listing than crawling the data bucket.

Both S3 buckets are in the Sydney AWS region ap-southeast-2. Programmatic access requires the environment variable AWS_NO_SIGN_REQUEST=Yes.

Most of the data that is stored on S3 is Cloud Optimised GeoTIFFs, which can be accessed directly without downloading the files.

Key products that are available on S3 include the following:

DEA Product Name

S3 Link

Web View

Surface Reflectance (Landsat 5)

S3: baseline/ga_ls5t_ard_3

Web: baseline/ga_ls5t_ard_3

Surface Reflectance (Landsat 7)

S3: baseline/ga_ls7e_ard_3

Web: baseline/ga_ls7e_ard_3

Surface Reflectance (Landsat 8)

S3: baseline/ga_ls8c_ard_3

Web: baseline/ga_ls8c_ard_3

Water Observations

S3: derivative/ga_ls_wo_3

Web: derivative/ga_ls_wo_3

Fractional Cover

S3: derivative/ga_ls_fc_3

Web: derivative/ga_ls_fc_3

Coastlines

S3: derivative/dea_coastlines

Web: derivative/dea_coastlines

SNS Notifications

We provide SNS notifications for newly produced DEA Datasets.

SNS Topic

Description

arn:aws:sns:ap-southeast-2:538673716275:DEANewData

This topic sends S3 Event Notifications for every object that lands in the dea-public-data bucket

arn:aws:sns:ap-southeast-2:538673716275:dea-public-data-landsat-3

This is a notification that is triggered for each new DEA Surface Reflectance scene

arn:aws:sns:ap-southeast-2:538673716275:dea-public-data-landsat-3-fc

This is a notification for each DEA Fractional Cover scene produced

arn:aws:sns:ap-southeast-2:538673716275:dea-public-data-landsat-3-wo

This is a notification for each DEA Water Observations scene produced

All the the scene level notifications include a STAC 1.0.0-beta.2 JSON document and have message attributes including the following:

Attribute

Description

action

added or archived, whether the scene was created or deleted

product

the Open Data Cube product name for the scene

datetime

the datetime stamp for the scene, this is the capture time of the scene

maturity

the maturity level of the dataset, either “nrt”, “interim” or “final”

bbox.ll_lon

lower left longitude

bbox.ll_lat

lower left latitude

bbox.ur_lon

upper right longitude

bbox.ur_lat

upper right latitude

For Surface Reflectance products, the following additional attributes are included:

Attribute

Description

cloud_cover

percentage cloud cover, a number between 0 and 100

gqa_iterative_mean_xy

a measure of the scenes geometric accuracy in pixels