fix(aws cloudwatch, aws s3): emit component_sent_bytes_total through Driver to ensure component labels - #285
Open
Clee2691 wants to merge 1 commit into
Open
fix(aws cloudwatch, aws s3): emit component_sent_bytes_total through Driver to ensure component labels#285Clee2691 wants to merge 1 commit into
Clee2691 wants to merge 1 commit into
Conversation
📝 WalkthroughSummary by CodeRabbit
WalkthroughAWS sinks now propagate resolved regions into request labels, emit component bytes-sent metrics, and avoid duplicate transport metrics. CloudWatch adds partial-upload byte accounting, while Kinesis and S3 responses report request byte sizes. ChangesShared metrics and AWS transport
CloudWatch partial upload metrics
Kinesis region and byte accounting
S3 region and byte accounting
SNS and SQS region labels
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Sink
participant Driver
participant AWSService
participant BytesSent
Sink->>Driver: configure HTTPS protocol and region label
Driver->>AWSService: send request
AWSService-->>Driver: return request byte size
Driver->>BytesSent: emit component bytes-sent metric
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Clee2691
force-pushed
the
LOG-7893-0.54.0
branch
from
July 16, 2026 22:04
3f4e627 to
61eb96b
Compare
Author
|
/hold |
Author
|
/hold cancel |
Author
|
/hold |
Clee2691
force-pushed
the
LOG-7893-0.54.0
branch
2 times, most recently
from
July 20, 2026 19:44
fdb9f35 to
0007855
Compare
Author
|
/hold cancel |
Clee2691
force-pushed
the
LOG-7893-0.54.0
branch
from
July 22, 2026 00:08
0007855 to
83769a5
Compare
Author
|
/retest |
Author
|
/hold |
Member
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Clee2691, jcantrill The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Clee2691
force-pushed
the
LOG-7893-0.54.0
branch
from
July 27, 2026 18:20
83769a5 to
7e70e0b
Compare
|
New changes are detected. LGTM label has been removed. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/sinks/aws_cloudwatch_logs/service.rs`:
- Around line 250-267: Prevent outer retries from replaying requests that
produce CloudwatchError::PutPartial, so the bytes_sent_handle emission in the
svc oneshot error path cannot be followed by full-request accounting on replay.
Update CloudwatchRetryLogic to classify PutPartial as non-retryable while
preserving existing retry behavior for other errors.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 69f88144-bf47-4663-a95f-ddc9b4f5e713
📒 Files selected for processing (31)
lib/vector-common/src/event_test_util.rslib/vector-common/src/internal_event/bytes_sent.rslib/vector-stream/src/driver.rssrc/aws/mod.rssrc/sinks/aws_cloudwatch_logs/config.rssrc/sinks/aws_cloudwatch_logs/integration_tests.rssrc/sinks/aws_cloudwatch_logs/request.rssrc/sinks/aws_cloudwatch_logs/request_builder.rssrc/sinks/aws_cloudwatch_logs/retry.rssrc/sinks/aws_cloudwatch_logs/service.rssrc/sinks/aws_cloudwatch_logs/sink.rssrc/sinks/aws_kinesis/config.rssrc/sinks/aws_kinesis/firehose/config.rssrc/sinks/aws_kinesis/firehose/integration_tests.rssrc/sinks/aws_kinesis/firehose/record.rssrc/sinks/aws_kinesis/service.rssrc/sinks/aws_kinesis/sink.rssrc/sinks/aws_kinesis/streams/config.rssrc/sinks/aws_kinesis/streams/integration_tests.rssrc/sinks/aws_kinesis/streams/record.rssrc/sinks/aws_s3/config.rssrc/sinks/aws_s3/integration_tests.rssrc/sinks/aws_s_s/service.rssrc/sinks/aws_s_s/sink.rssrc/sinks/aws_s_s/sns/config.rssrc/sinks/aws_s_s/sns/integration_tests.rssrc/sinks/aws_s_s/sqs/config.rssrc/sinks/aws_s_s/sqs/integration_tests.rssrc/sinks/s3_common/config.rssrc/sinks/s3_common/service.rssrc/sinks/s3_common/sink.rs
…Driver to ensure component labels
Clee2691
force-pushed
the
LOG-7893-0.54.0
branch
from
July 27, 2026 20:55
7e70e0b to
dc3e599
Compare
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/vector-common/src/internal_event/bytes_sent.rs`:
- Around line 9-16: Update the CloudWatch bytes-sent instrumentation around
CloudwatchResponse::bytes_sent, the service-level BytesSent registration, and
Driver emission so each successful request records component_sent_bytes_total
exactly once, preferably through Driver with component labels. Remove or exclude
the overlapping service-side emission, preserve the intended labels, and add a
regression assertion preventing duplicate or unlabeled series.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 16b6dc9e-d0bc-43ae-9cad-c29ce41375c6
📒 Files selected for processing (30)
lib/vector-common/src/event_test_util.rslib/vector-common/src/internal_event/bytes_sent.rslib/vector-stream/src/driver.rssrc/aws/mod.rssrc/sinks/aws_cloudwatch_logs/config.rssrc/sinks/aws_cloudwatch_logs/integration_tests.rssrc/sinks/aws_cloudwatch_logs/request.rssrc/sinks/aws_cloudwatch_logs/request_builder.rssrc/sinks/aws_cloudwatch_logs/service.rssrc/sinks/aws_cloudwatch_logs/sink.rssrc/sinks/aws_kinesis/config.rssrc/sinks/aws_kinesis/firehose/config.rssrc/sinks/aws_kinesis/firehose/integration_tests.rssrc/sinks/aws_kinesis/firehose/record.rssrc/sinks/aws_kinesis/service.rssrc/sinks/aws_kinesis/sink.rssrc/sinks/aws_kinesis/streams/config.rssrc/sinks/aws_kinesis/streams/integration_tests.rssrc/sinks/aws_kinesis/streams/record.rssrc/sinks/aws_s3/config.rssrc/sinks/aws_s3/integration_tests.rssrc/sinks/aws_s_s/service.rssrc/sinks/aws_s_s/sink.rssrc/sinks/aws_s_s/sns/config.rssrc/sinks/aws_s_s/sns/integration_tests.rssrc/sinks/aws_s_s/sqs/config.rssrc/sinks/aws_s_s/sqs/integration_tests.rssrc/sinks/s3_common/config.rssrc/sinks/s3_common/service.rssrc/sinks/s3_common/sink.rs
🚧 Files skipped from review as they are similar to previous changes (25)
- src/sinks/aws_s_s/sink.rs
- src/sinks/aws_cloudwatch_logs/sink.rs
- src/sinks/s3_common/sink.rs
- src/sinks/aws_kinesis/sink.rs
- src/sinks/aws_kinesis/firehose/integration_tests.rs
- src/sinks/s3_common/config.rs
- src/sinks/aws_kinesis/streams/record.rs
- src/sinks/aws_s_s/service.rs
- src/sinks/aws_kinesis/streams/integration_tests.rs
- src/sinks/aws_s_s/sqs/config.rs
- src/sinks/aws_kinesis/firehose/record.rs
- src/sinks/aws_s_s/sqs/integration_tests.rs
- lib/vector-stream/src/driver.rs
- src/sinks/aws_s_s/sns/config.rs
- src/sinks/s3_common/service.rs
- src/sinks/aws_kinesis/service.rs
- src/sinks/aws_s3/config.rs
- lib/vector-common/src/event_test_util.rs
- src/sinks/aws_kinesis/streams/config.rs
- src/sinks/aws_cloudwatch_logs/config.rs
- src/sinks/aws_s3/integration_tests.rs
- src/sinks/aws_cloudwatch_logs/request.rs
- src/sinks/aws_cloudwatch_logs/integration_tests.rs
- src/aws/mod.rs
- src/sinks/aws_cloudwatch_logs/service.rs
Author
|
/retest |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is a forward port of #284 that also includes the fix for S3 for
v0.54.0-rhbranch.This is a bug for all AWS sinks.
/cc @vparfonov
/assign @jcantrill
Links
Related: #284
JIRA: https://redhat.atlassian.net/browse/LOG-9671