Tech Talk

Clarify Kusto Delete Commands on ADX

Clarify Kusto Delete Commands on ADX
  • Christoph Thale

  • 4 min

  • September 18, 2024

Deleting data in Azure Data Explorer is more nuanced than a single command. Learn the difference between hard deletes, soft deletes, and retention policies – and when to use each.

Hard Delete vs. Soft Delete vs. Retention Policy in ADX

Azure Data Explorer provides robust mechanisms for data deletion. Understanding the differences between hard and soft deletes, and the role of retention policies, is crucial for effective data lifecycle management.

 

Hard Deletes vs. Soft Deletes

  • Hard Deletes (.purge): Irreversible, data is permanently removed from disk. Must be enabled in cluster settings.
  • Soft Deletes (.drop table, .delete): Data appears removed but remains on disk for 14 days. A background job completes the deletion during this window.

The 14-day recovery window is the key difference. If you accidentally soft-delete data, contact Azure Microsoft Support immediately.

Retention Policies

Retention policies add another layer of control, defined by two attributes:

  • SoftDeletePeriod – how long data is retained before becoming eligible for deletion (e.g. 3650 days = 10 years)
  • Recoverability – if Enabled, a 14-day soft-delete window follows the retention period; if Disabled, data is immediately purged

Example Policy

{   "SoftDeletePeriod": "3650.00:00:00",   "Recoverability": "Enabled" }

With this policy, data is retained for 10 years, then soft-deleted with a 14-day recovery window.