AWS DynamoDB Replicating Record Status Test
Amazon Kinesis Data Streams can be used to capture changes to Amazon DynamoDB. Kinesis Data Streams capture item-level modifications in any DynamoDB table and replicate them to a Kinesis data stream. The applications can access this stream and view item-level changes in near-real time. When a Kinesis data stream is enabled for a DynamoDB table, the table sends out a data record that captures any changes to that table’s data. These data records are captured and published in near-real time. After they are written to the Kinesis data stream, they can be read just like any other record. A Kinesis data stream counts its throughput in shards. You will need to provision an appropriate number of shards on your Kinesis data stream to accommodate the change in data capture records from DynamoDB. When the data streams are under-provisioned, that can lead to unsuccessful replications due to excessive throttling. Customer-controlled configurations such as manual updates to your Kinesis data stream’s access policies that prevent DynamoDB from adding data to your data stream can also lead to unsuccessful replication. These unsuccessful replications cause the build-up of unreplicated records, that can eventually lead to critical data loss. However, by continuously monitoring replicating record status, you would be able to determine whether the stream has sufficient capacity or not. The AWS DynamoDB Replicating Record Status Test does exactly the same.
This test monitors the age of the oldest unreplicated records on the target server, which helps administrators to find out the unsuccessful replication attempts due to customer controlled configuration choices. This way, they can proactively increase the provisioning of data streams before it leads to any critical data loss.
Target of the test : An AWS DynamoDB server
Agent deploying the test : A remote agent
Outputs of the test : One set of results for the target AWS DynamoDB server being monitored.
Parameter | Description |
---|---|
Test Period |
How often should the test be executed. |
Host |
The IP address of the AWS DynamoDB server that is being monitored. |
AWS Region |
This test uses AWS SDK to interact with AWS DynamoDB and pull relevant metrics. To enable the test to connect to AWS, you need to configure the test with the name of the region to which all requests for metrics should be routed, by default. Specify the name of this AWS Region in this text box. |
AWS Access Key ID, AWS Secret Access Key and Confirm Password |
To monitor AWS DynamoDB, the eG agent has to be configured with the access key and secret key of a user with a valid AWS account. For this purpose, we recommend that you create a special user on the AWS cloud, obtain the access and secret keys of this user, and configure this test with these keys. The procedure for this has been detailed in the Obtaining an Access key and Secret key topic. Make sure you reconfirm the access and secret keys you provide here by retyping it in the corresponding Confirm Password text box. |
Timeout Seconds |
Specify the maximum duration (in seconds) for which the test will wait for a response from the server. The default is 120 seconds. |
DD Row Count |
By default, the detailed diagnosis of this test, if enabled, will report only the top-10 tables. This is why, the DD Row Count parameter is set to 10 by default. If you want to include more or less tables in detailed diagnosis, then change the value of this parameter accordingly. |
Detailed Diagnosis |
To make diagnosis more efficient and accurate, the eG Enterprise embeds an optional detailed diagnostic capability. With this capability, the eG agents can be configured to run detailed, more elaborate tests as and when specific problems are detected. To enable the detailed diagnosis capability of this test for a particular server, choose the On option. To disable the capability, click on the Off option. The option to selectively enable/disable the detailed diagnosis capability will be available only if the following conditions are fulfilled:
|
Measurement |
Description |
Measurement Unit |
Interpretation |
---|---|---|---|
Maximum age of old unreplicated records |
Indicates the maximum age of oldest unreplicated records on the DynamoDB table in the target server. |
Seconds |
This measure grows based on unsuccessful replication attempts when these are caused by customer-controlled configuration choices. To keep this metric as low as possible, you might need to ensure the right provisioning of your Kinesis data stream capacity and make sure that DynamoDB’s permissions are unchanged. Use detailed diagnosis of this measure to find out the Table name, Delegated operation, Age of old unreplicated records. |
Minimum age of old unreplicated records |
Indicates the minimum age of oldest unreplicated records on the DynamoDB table in the target server. |
Seconds |
|