DEA Tech Alerts and Changelog
Find out about the latest updates to Digital Earth Australia’s products and services, planned maintenance, and any outages that may occur. To check the current status of DEA’s services, see the DEA monitoring dashboard. And, to stay informed of the latest updates, subscribe to our newsletter.
System status of DEA
Expanded extents data processing is currently paused, with data available to May 2024. See below.
Misclassification issue with Sentinel-2 ‘s2cloudless’ cloud masking from 2022. See below.
Terra-derived DEA Hotspots are unavailable. See below.
NCI Explorer may experience instability. See below.
GeoMAD processing bug. See below.
In progress: Shift in origin point of DEA Summary Product Grid
To accommodate an expanded area of coverage of Australia’s external territories, the DEA Summary Product Grid will soon be shifted. The south-west origin point of the grid will be shifted from -5472000.0, -2688000.0 to -6912000.0, -4416000.0 (EPSG:3577). Therefore, all tile grid references will shift 18 tiles west and 15 tiles south. For instance, a tile reference of x10y10 will change to x28y25. For a preview, see the provisional version of the expanded DEA Summary Product Grid on DEA Maps. Download the new grid from AWS S3
All new versions of our ‘summary derivative products’ will be affected by this change (but existing versions of the products will not be changed).
It has already been applied to the latest versions of the following products: DEA Geometric Median and Median Absolute Deviation (Landsat), DEA Water Observations Statistics (Landsat), DEA Fractional Cover Percentiles (Landsat), DEA Tasseled Cap Percentiles (Landsat), and DEA Mangroves (Landsat).
It will be applied to the new version of the following product when released: DEA Land Cover (Landsat)
Learn more about the DEA Summary Product Grid.
18 Oct 2024: DEA upgrade of several Landsat derivative products complete
Digital Earth Australia (DEA) has released Landsat derivative product upgrades for DEA Water Observations Statistics, DEA Fractional Cover Percentiles, DEA Tasseled Cap Percentiles (provisional product), and DEA Mangroves.
The release triggered downtime for these products as upgrades were implemented.
The upgrades are now complete with all services online and ready to use.
17 Oct 2024: DEA Sandbox has been upgraded
Note: Users may encounter caching issues when logging in to the Sandbox for the first time after this upgrade; please empty cache and hard reload your Sandbox’s browser tab if problems occur.
The DEA Sandbox environment has been successfully upgraded to include new Python packages and software.
Several new packages have also been added, including gcfs (for accessing data on Google Cloud), xarray-spatial (for performing spatial analysis with xarray), and pyspark (unified analytics engine for large-scale data processing).
Most significantly, this included an upgrade to JupyterLab 4.0, which updated the main Sandbox user interface. Please note that some existing Sandbox functionality (code snippet tool and resource monitor graphs) are not currently supported by JupyterLab 4.0 and will be restored as soon as they are available.
17 Oct 2024: NCI Explorer outage
NCI Explorer has experienced an outage since yesterday. You may encounter issues when attempting to use this service. We are currently investigating the issue.
16 Oct 2024: Upcoming DEA Sandbox environment upgrade
On 16 October, DEA will be upgrading the DEA Sandbox environment to include new Python packages and software.
Several new packages have also been added, including gcfs (for accessing data on Google Cloud), xarray-spatial (for performing spatial analysis with xarray), and pyspark (unified analytics engine for large-scale data processing).
Most significantly, this will include an upgrade to JupyterLab 4.0, which will update the main Sandbox user interface. Users may encounter caching issues when logging in to the Sandbox for the first time after this upgrade; please empty cache and hard reload your Sandbox’s browser tab if problems occur. Please note that some existing Sandbox functionality (code snippet tool and resource monitor graphs) are not currently supported by JupyterLab 4.0 and will be restored as soon as they are available.
See the full list of Python package versions available in the new Sandbox environment.
3 Oct 2024: DEA Landsat Derivative Product Upgrade Downtime (9 October to 11 October)
Digital Earth Australia (DEA) is releasing Landsat derivative product upgrades for DEA Water Observations Statistics, DEA Fractional Cover Percentiles, DEA Tasseled Cap Percentiles (provisional product) and DEA Mangrove Canopy Cover.
The upgrades are set to take place between the 9–11 of October 2024. Users should expect all existing services for these products will be unavailable from 9am 9 October until 9am 11 October (AEDT). We recommend not using the above four products during this time. A DEA Tech Alert will be sent out when upgrades are completed.
26 Sep 2024: ‘s2cloudless’ ARD reprocessing underway
We have begun reprocessing the older Sentinel-2 ARD data that was impacted by a bug in the s2cloudless cloud masking layer. The bug is ‘24 May 2024: Misclassification issue with Sentinel-2 ‘s2cloudless’ cloud masking from 2022’.
The addition of an offset factor in ESA’s Sentinel-2 L1C Processing Baseline 4.0.0 on January 25
2022 led to the generation of incorrect s2cloudless cloud
classifications in our systems. This resulted in an over-classification of cloud, particularly over bare and
agricultural regions
Sentinel-2 s2cloudless data from 25 January 2022 to 7 June 2024 is affected, and we expect to replace these datasets
over the next several months.
In the meantime, we advise the users to avoid using s2cloudless for cloud masking on data between these two dates,
and consider using the Fmask cloud mask as a temporary alternative during this period.
19 Sep 2024: GeoMAD processing bug
In the DEA Geometric Median and Median Absolute Deviation (Landsat) product, a bug in the processing code related to multithreading using numexpr (a fast numerical expression evaluator for Numpy) has been identified which may cause a 400x400 pixel data block to be misplaced within a tile. For the full GeoMAD archive, it is possible that there are around 8-12 tiles with incorrect data (misplaced blocks). It is unknown at this stage which tiles are affected. We are investigating the bug and will provide more information in late 2024 to early 2025.
13 Sep 2024: Outage affecting multiple web services (Resolved)
Multiple web services are experiencing an outage: DEA Explorer (Prod), DEA WCS (PROD - EKS), DEA WMS (PROD - EKS), and DEA WMTS (PROD - EKS). We are currently working on resolving the issue. See the real-time status of these services.
2 Sep 2024: Expanded extents data processing is currently paused, with data available to May 2024
This year, we expanded the processing extents for our Surface Reflectance products and derivative products.
Unfortunately, due to unanticipated issues with an external network connection, we have needed to pause the back-processing of this expanded extent data since May 2024. You may have noticed this issue, for example, if you attempted to access ARD from the Heard Island and McDonald Islands and found that there are no images more recent than May 2024.
16 Aug 2024: Hotspots MODIS SRSS missing data (Resolved)
On 21 Aug, we detected that the DEA Hotspots MODIS SRSS sub-product was missing *hotspots.txt and *metadata.txt files in its daily data folders since 16 Aug. This was caused by a software bug. After detecting the issue, we fixed the bug on the same day and uploaded all the missing data files.
8 Aug 2024: NCI Explorer may experience instability
The NCI Explorer has experienced some intermittent outages recently. We are currently investigating the cause of the issue. If the NCI Explorer fails to load, please try again later.
2 Aug 2024: Version 4.0.0 of DEA Geometric Median and Median Absolute Deviation released
The newly released version 4.0.0 of DEA Geometric Median and Median Absolute Deviation contains new features and changes such as the following.
Breaking change: Shift in grid origin point.
Enhanced cloud masking to reduce noise.
Combined Landsat 8 and 9 product.
Prefixed band measurement names.
See the Version 4.0.0 Changelog for more information.
1 Aug 2024: Resolved outage of Hotspots S-NPP VIIRS data
The Suomi-NPP VIIRS Hotspots sub-product has resumed publishing data because the satellite’s issue has been resolved.
25 Jul 2024: Outage of Hotspots S-NPP VIIRS data (Resolved)
This issue concerns DEA Hotspots. The Suomi-NPP satellite has experienced a data outage that has resulted in data unavailability until further notice. Therefore, Suomi-NPP VIIRS Hotspots cannot be published until this issue is resolved. Other DEA Hotspots sub-products are unaffected by this outage.
1 Jul 2024: External data products now featured on the Knowledge Hub
External data products are produced by external providers such as a different program of Geoscience Australia or a different organisation entirely. These external data products may assist with the analysis of DEA data. DEA even provides services relating to some of these external products. Therefore, we have begun to include some external products on our Knowledge Hub so that you can easily find and access this data. So far, we have added three external data products to the Knowledge Hub. But watch this space, as there are more to come!
1 Jul 2024: DEA Sandbox new user signups restored
An issue was preventing new users from signing up to the DEA Sandbox for the past 5 days. We have now fixed the issue and you should now be able to sign up for the DEA Sandbox.
28 Jun 2024: DEA Published Product Currency Report released
We have released a report that tracks the Currency of our data products and if they are on time. Currency is a measure of how consistently DEA’s data products have been published through DEA in line with the stated update frequency on or before the scheduled publish date. External auditors can use this report to verify that DEA is meeting its performance targets, and internally, we use it for our annual reporting practices.
View the DEA Published Product Currency Report.
Learn more about how to understand and analyse this report.
Internal staff can also view the full history of data.
24 Jun 2024: Performance issues with DEA Explorer and STAC API (Resolved)
The recent performance issues with the DEA Explorer and DEA Explorer STAC API have now been resolved. You will notice that the performance and stability of these services has returned to normal. We hope you continue to enjoy using these services.
14 Jun 2024: DEA Sandbox service has been restored
The unplanned outage that affected the DEA Sandbox today has been resolved. The DEA Sandbox is now back online.
14 Jun 2024: DEA Sandbox unplanned outage (Resolved)
The DEA Sandbox is currently experiencing an unplanned outage. We are investigating the issue and hope to have the service back up and running soon. We will post an update when the service is restored.
12 Jun 2024: DEA Intertidal data now available on ELVIS
DEA Intertidal elevation and uncertainty data can now be ordered and downloaded from the ELVIS platform.
For instructions on how to access this data from ELVIS, visit the DEA Intertidal page (Access tab).
30 May 2024: NCI THREDDS data access links updated to point to the new THREDDS server
NCI has released an upgrade to the THREDDS Data Service and will decommission the existing THREDDS server after 30th June 2024.
To prepare for this change, all THREDDS data access links in the Knowledge Hub have been updated to point to the new THREDDS server: all https://dapds00.nci.org.au/thredds/... links have been changed to https://thredds.nci.org.au/thredds/....
24 May 2024: Misclassification issue with Sentinel-2 ‘s2cloudless’ cloud masking from 2022
An issue has been identified that is causing widespread misclassification of clouds in DEA’s Sentinel-2 s2cloudless cloud mask data generated since January 2022.
We recommend that you avoid using s2cloudless cloud mask data from 2022 onwards until this issue is investigated.
6 May 2024: Performance issues with DEA Explorer and STAC API
You may notice slow load times or outages with the DEA Explorer and DEA Explorer STAC API. We apologise for any inconvenience that this may cause and we are working to solve this issue.
1 May 2024: Six of Australia’s offshore territories now covered by Digital Earth Australia baseline satellite data
In support of Indigenous Communities in the Torres Strait, in addition to government agencies reporting on Ocean Ecosystem Accounts and Marine Parks management, DEA is making baseline satellite data available for 6 offshore territories and islands in the Torres Strait.
Learn more about the DEA ARD expanded processing extent.
23 Apr 2024: It’s now easier to cite DEA products
A new ‘Cite this product’ section has been added to each product page to make it easier to cite the product in your academic paper, article, or presentation. A citation may been provided for the data itself, and where relevant, to the published methods paper.
See this feature on the DEA Waterbodies product page.
18 Apr 2024: New Validation reports section
The Knowledge Hub contains a new Validation reports section. This is where you will find reports of Geoscience Australia’s validation data, published periodically. This data can be used to validate Geoscience Australia’s other datasets.
See the latest Daily Validation Summary Report: 2023-11-27: Transect NSW Site 1, Sentinel-2B overpass
10 Apr 2024: DEA Intertidal 1.0.0 released
The DEA Intertidal product suite maps the changing elevation, exposure and tidal characteristics of Australia’s exposed intertidal zone, the complex zone that defines the interface between land and sea.
This new product suite expands upon the DEA Intertidal Elevation (Landsat) product which has now been deprecated.
See DEA Intertidal for more information.
3 Apr 2024: Terra-derived DEA Hotspots have been restored
Direct Broadcast satellite downloads from the Terra satellite have been restored and have been successfully processed into DEA Hotspots.
28 Mar 2024: DEA Sandbox outage (Resolved)
Users may have encountered an unplanned outage on the Digital Earth Australia Sandbox.
DEA has implemented a solution as we continue to investigate the cause of the problem.
DEA Sandbox users are warned to expect intermittent issues over the next 12 hours.
Users may be unable to log in to or access their DEA Sandbox account while the incident is being investigated. Users may also notice reduced performance on other DEA systems during this outage.
You can monitor the status of DEA’s systems on the DEA monitoring dashboard. If you would like to contact DEA, please email earth.observation@ga.gov.au
25 Mar 2024: DEA Waterbodies version 3.0.0 released
Improvements include additional supporting data for the most recent observations made available through web mapping services (WMS) and DEA Maps, more metadata, Landsat 9 data, and pipeline upgrades. Version 3.0.0 of DEA Waterbodies uses the same underlying polygon set as DEA Waterbodies version 2.0.0. Learn more
21 Mar 2024: DEA OWS services restored
Outages affecting DEA Open Web Services (OWS) have been resolved. Work will be continuing over the coming months to further improve performance and reliability of these services.
See the DEA monitoring dashboard to check the current status of DEA’s services.
26 Feb 2024: Sentinel-2 contiguity fix reprocessing complete
Learn more in the Sentinel-2A Surface Reflectance NBART and Sentinel-2B Surface Reflectance NBART changelogs.
20 Feb 2024: DEA Notebooks update
A minor update to the DEA Tools package that includes updates to tide modelling and parallel processing functions. It also adds a new load_reproject function to the datahandling module which allows reading and reprojecting external raster data. For more detail, see the dea-notebooks 0.3.1 release notes.
20 Feb 2024: New external land use and bathymetry datasets added to DEA Sandbox
Two new external datasets have been added to the DEA Sandbox: the 2020 ABARES Catchment Scale Land Use of Australia 50m dataset, and the recently released 2023 Geoscience Australia AusBathyTopo 250m high-resolution depth model for Australia. These datasets provide valuable insights into land use patterns and the physical shape of Australia’s mainland, our coasts and deep ocean regions.
These datasets can be loaded on the DEA Sandbox using the product names abares_clum_2020 and ga_ausbathytopo250m_2023.
15 Feb 2024: Upcoming changes to DEA Geomedian, Median Absolute Deviation and other derivative products
The Geoscience Australia Landsat Geometric Median and Median Absolute Deviation Collection 3 products v3.1.0 are undergoing a major upgrade which will be released shortly as v4.0.0.
Find out the details here.
24 Jan 2024: Water Observations and Fractional Cover Percentiles 2023 annual summaries released
See DEA Water Observations Statistics (Landsat) and DEA Fractional Cover Percentiles (Landsat) for more information.
10 Jan 2024: Sentinel-2 contiguity fix - Reprocessing commenced
Reprocessing to fix the Sentinel-2 contiguity issue has commenced with expected completion in early 2024. The issue was caused by anomalies in ESA Level 1 source data.
A known Sentinel-2 contiguity issue created by ESA Level 1 anomalies (“Striping due to lost source packets”) is impacting approximately 0.5% of our Sentinel-2 Analysis Ready Data (S2 ARD) between 2015 and 2023.
Affected products:
ARD Sentinel-2 products (ga_s2am_ard_3, ga_s2bm_ard_3)
See DEA Tech alert email for more information. Click here to subscribe to DEA Tech alert emails.
Older entries
View the tech alerts and changelogs from previous years.