Delete data from an index layer

You can delete data and metadata from an index layer using RSQL queries. Indexed records (the metadata) which meet your deleteQuery criteria will be deleted from your index layer. Deleting the indexed records will also automatically delete the respective data stored in blob after one day.

Deletion of index layer metadata and data is an asynchronous and relatively slow process. For example, a delete request with 1 million records satisfying the deleteQuery could take 1 hour to complete deletion of metadata; The actual data in blobstore will continue to be available for 24 hours after complete deletion of metadata, then they will be completely purged.

If you want to remove all data in an index layer which contain millions of record, it would be faster to Delete a layer and recreate it.

To delete index layer data and metadata:

1. Obtain an authorization token.
2. Use the api-lookup v1 API to get API base URLs.
3. Use the index v1 API to delete the indexed records along with the corresponding data that matches your delete query.

Obtain an authorization token

Obtain an authorization token for your HTTP requests. For instructions, see the Identity & Access Management Guide.

Get API base URLs

Use the API Lookup service to get the API endpoints for the index API of the catalog containing the index layer with data handles you want to delete.

For instructions, see the API Lookup Developer's Guide.

Delete data

In order to delete data, you must perform a Delete action by specifying "deleteQuery" request parameter in the index API.

DELETE /<Base path for the index API from the API Lookup Service>/layers/<Layer ID>?deleteQuery=<RSQL> HTTP/1.1
Host: <Hostname for the index API from the API Lookup Service>
Authorization: Bearer <Authorization Token>
Accept: application/json
Cache-Control: no-cache

The rule for \ in "deleteQuery" request parameter is the same as the "query" request parameter in Get Data from an Index Layer -> Section "Get the Data Handle".

Once you perform the Delete action above, you will receive a deleteId in the response. Example response:

{
    "deleteId": "f054dd41-51f2-4633-9a42-cbcf59cc24e1"
}

Query the delete status

You can use the deleteId obtained above to query the status of the delete action you just submitted:

GET /<Base path for the index API from the API Lookup Service>/layers/<Layer ID>/deleteRequest/<Delete ID> HTTP/1.1
Host: <Hostname for the index API from the API Lookup Service>
Authorization: Bearer <Authorization Token>
Accept: application/json
Cache-Control: no-cache

You will receive one of the 4 possible statuses:

1. Scheduled
2. Processing
3. Succeeded
4. Failed

Examples of responses:

{
    "deleteId": "f054dd41-51f2-4633-9a42-cbcf59cc24e1",
    "state": "Scheduled",
    "message": "The delete request was successfuly scheduled.",
    "count": 0
}

When the state is "Scheduled", the count is always 0, meaning no index record deleted yet.

{
    "deleteId": "f054dd41-51f2-4633-9a42-cbcf59cc24e1",
    "state": "Processing",
    "message": "The delete request is being executed.",
    "count": 7289
}

When the state is "Processing", the count is the number of index records that have been deleted so far.

{
    "deleteId": "f054dd41-51f2-4633-9a42-cbcf59cc24e1",
    "state": "Succeeded",
    "message": "The delete request finished successfuly. The number of deleted records is provided in the `count` field.",
    "count": 13445
}

When the state is "Succeeded", the count is the total number of deleted index records.