Backfill Acceleration Task Data

1. Overview

This guide explains how to backfill data for an acceleration plan through APIs. Backfill lets you regenerate or repair data for a specified acceleration plan over a historical date range.

The process follows three steps: create a task, query progress, and locate details. This helps you programmatically track and manage large batches of backfill tasks.

2. Common Request parameters (HEADERS)

The following parameters apply to all APIs in this guide.

parameter	Type	required	Description
`tenant-id`	String	Yes	Tenant ID used to identify the tenant where the resources reside.
`auth-type`	String	Yes	Authentication method. Supports `UID`, `TOKEN`, `ACCOUNT`, and `APIKEY`.
`auth-value`	String	Yes	Authentication value corresponding to `auth-type`.

How to Obtain Common parameters

In Aloudata CAN, select Metric Applications in the top navigation bar, then select API Integration in the left navigation menu. You can obtain tenant-id on the API Integration page. Set auth-value according to auth-type.

step 1: Create a Backfill Task

This API creates a backfill task. After successful submission, the system returns a globally unique task identifier (dagName) that is used to connect subsequent queries.

1. API URL

POST http://{anymetrics_host}:{anymetrics_port}/anymetrics/api/v1/accelerate/task/create

{anymetrics_host}:{anymetrics_port} For how to obtain this value, see Calling APIs

2. Request parameters

2.1 Common Request parameters (HEADERS)

See Common Request parameters (HEADERS) at the beginning of this guide.

2.2 Request Body parameters (Body)

parameter Name	Type	required	Description
`planId`	String	Yes (either this or `planName` is required)	Unique identifier (UUID) of the acceleration plan.
`startTime`	String	Yes	Start date of the backfill data, in `YYYY-MM-DD` format.
`endTime`	String	Yes	End date of the backfill data, in `YYYY-MM-DD` format.
`dateIntervalOfLot`	Integer	No	Number of days processed in each batch. The system splits the date range into multiple task instances by this value. The default is `1`.

3. Request Example

curl --location --request POST 'http://{anymetrics_host}:{anymetrics_port}/anymetrics/api/v1/accelerate/task/create' \
--header 'tenant-id: {tenant_id}' \
--header 'auth-type: UID' \
--header 'auth-value: {uid}' \
--header 'Content-Type: application/json' \
--data-raw '{
    "planId": "5468b0c0462e40e08e44108d8164c7c3",
    "startTime": "2025-01-01",
    "endTime": "2025-01-01",
    "dateIntervalOfLot": 1
}'

4. Response Example

{
    "data": "e94bc831-f5f9-4604-8717-6742ae61e729",
    "success": true,
    "code": "200",
    "errorMsg": null,
    "detailErrorMsg": null,
    "traceId": "78f6b068f84545aea98460f6f5b0627e.195.17761432013872163"
}

Response Result

Return Value: String
Example: "80d3a5a9-027d-4c82-97e2-9e9ac9da873a"
Description: The return value is the unique identifier of this backfill task, **dagName**

step 2: Query the Task Instance List

Use the dagName returned in step 1 to query the execution status of all child task instances in this batch.

1. API URL

GET http://{anymetrics_host}:{anymetrics_port}/anymetrics/api/v1/accelerate/task/instance/list

2. Request parameters

2.1 Common Request parameters (HEADERS)

See Common Request parameters (HEADERS) at the beginning of this guide.

2.2 Query parameters

parameter Name	Type	required	default	Description
`dagName`	String	Yes	-	Backfill task identifier returned in step 1.
`pageIndex`	Integer	No	1	current page number, starting from `1`.
`pageSize`	Integer	No	20	Number of records returned per page.

3. Request Example

curl --location --request GET 'http://{anymetrics_host}:{anymetrics_port}/anymetrics/api/v1/accelerate/task/instance/list?dagName=80d3a5a9-027d-4c82-97e2-9e9ac9da873a&pageIndex=1&pageSize=20' \
--header 'tenant-id: {tenant_id}' \
--header 'auth-type: UID' \
--header 'auth-value: {uid}' \
--header 'Content-Type: application/json'

4. Response Example

{
    "dagName": "80d3a5a9-027d-4c82-97e2-9e9ac9da873a",
    "pageIndex": 1,
    "pageSize": 20,
    "total": 3,
    "instances": [
        {
            "id": "12876560",
            "status": "SUCCESS",
            "mtGuid": "catalog.schema.table_name",
            "startPartition": "2025-01-01",
            "endPartition": "2025-01-01",
            "startTime": "2025-03-25T10:00:00",
            "endTime": "2025-03-25T10:02:30",
            "resourceGroup": "default",
            "priority": 1,
            "createTime": "2025-03-25T09:59:00",
            "scheduleTime": "2025-03-25T10:00:00"
        }
    ]
}

5. Response Field Description

Field Path	Description
`dagName`	Overall identifier of the backfill task.
`pageIndex` / `pageSize`	Current pagination information.
`total`	Total number of instances split from this backfill task.
`instances`	task instance list.
`instances[ ].id`	Key field. Unique ID of the task instance, used to query details in step 3.
`instances[ ].status`	Instance run status. Possible values: `WAITING`, `RUNNING`, `SUCCESS`, and `FAILED`.
`instances[ ].mtGuid`	Globally unique identifier of the materialized table operated by this instance
`instances[ ].startPartition`	Start date of the data partition refreshed by this instance
`instances[ ].endPartition`	End date of the data partition refreshed by this instance
`instances[ ].startTime`	Actual start time of the instance
`instances[ ].endTime`	Actual end time of the instance
`instances[ ].createTime`	task instance creation time

step 3: Query Details of a Single Instance

When a child task is FAILED or needs further analysis, use the instance id obtained in step 2 to query detailed execution information, including the actual SQL and error logs.

1. API URL

GET http://{anymetrics_host}:{anymetrics_port}/anymetrics/api/v1/scheduler/task/detail

2. Request parameters

2.1 Common Request parameters (HEADERS)

See Common Request parameters (HEADERS) at the beginning of this guide.

2.2 Query parameters

parameter Name	Type	required	Description
`id`	String	Yes	Instance `id` returned in step 2.

3. Request Example

curl --location --request GET 'http://{anymetrics_host}:{anymetrics_port}/anymetrics/api/v1/scheduler/task/detail?id=12876560' \
--header 'tenant-id: {tenant_id}' \
--header 'auth-type: UID' \
--header 'auth-value: {uid}' \
--header 'Content-Type: application/json'

4. Response Example

{
    "taskInstanceId": 12876560,
    "status": "SUCCESS",
    "sql": {
        "originalSql": "INSERT INTO ... SELECT ... WHERE dt >= '2025-01-01' AND dt <= '2025-01-01'",
        "formattedSql": "..."
    },
    "runType": "MANUAL",
    "msg": null,
    "serverInfo": "10.161.0.244",
    "startTime": "2025-03-25T10:00:00",
    "endTime": "2025-03-25T10:02:30",
    "startPartitionTime": "2025-01-01T00:00:00",
    "endPartitionTime": "2025-01-01T00:00:00",
    "retryCnt": 0,
    "mtGuid": [{"catalog": "...", "schema": "...", "table": "..."}],
    "acceleratePlanUuid": ["5468b0c0462e40e08e44108d8164c7c3"],
    "statusTime": {
        "WAITING": "2025-03-25T09:59:00",
        "RUNNING": "2025-03-25T10:00:00",
        "SUCCESS": "2025-03-25T10:02:30"
    },
    "taskType": "RESULT_PLAN"
}

5. Key Response Fields

Field Name	Description
`sql`	Key field. SQL statement actually executed by the task, including substituted date variables.
`msg`	Key field. When the task fails, this field contains the specific error stack and is the primary basis for troubleshooting.
`serverInfo`	IP address of the server that executed the task.
`retryCnt`	Number of retries already attempted by the task.
`statusTime`	Timestamps for each task status, such as waiting, running, and finished, for duration analysis.
`runType`	Trigger type. `MANUAL` indicates manual backfill, and `SCHEDULE` indicates system scheduled execution.

Process Summary

The following table summarizes the three-step call flow:

step	API	Key input	Key output
① Create Task	`POST .../task/create`	`planId`, date range	`dagName`
② Query List	`GET .../task/instance/list`	`dagName`	Paginated instance list, including `id`
③ Query Details	`GET .../scheduler/task/detail`	`id`	SQL, error logs, and other details