Feat/buffer travel future#2668
Conversation
Add roboflow_core/time_travel@v1, a fusion block that returns a workflow value (detections, numbers, strings, images, ...) from a different frame of the same video stream at a relative offset (N-k past, N+k future), keyed per video_identifier. - Past offsets resolve immediately from a bounded per-video ring buffer and work in every execution context. - Future look-ahead delays the whole workflow output via the InferencePipeline stream-pipeline emission mechanism. It is opt-in via TIME_TRAVEL_STREAM_LOOKAHEAD (default off) so synchronous consumers (WebRTC/webexec, HTTP, preview) keep receiving an immediate per-frame result and are never turned into a PipelinedWorkflowRunner unexpectedly. - Generalize the InferencePipeline delayed-emission gate so any handler that opts into stream pipelining activates the drain/close path, not just RF-DETR.
|
👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely. 🚧 Right now this is a draft, so automated Claude review is paused — nothing is being spent yet. Mark it Ready for review to trigger it. Warning 💸 The Claude reviewer bills in credits, not vibesAutomated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared. Draft when unsure, Ready when you mean it:
However you get there, arrive prepared:
Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.
|
|
Nice! I like how the feature is encapsulated to just a single block! Well done @rafel-roboflow ! |
…ence into feat/buffer-travel-future
What does this PR do?
Adds a new workflow block Frame Delay (
roboflow_core/frame_delay@v1) that returns a value produced on an earlier frame of the same video stream. You wire any workflow output (detections, numbers, strings, images, …) into it, set a negativeoffset, and it returns that value as it was|offset|frames ago — enabling cross-frame comparisons and temporal alignment on video streams.Key points:
video_identifierring buffer keyed byframe_number; memory is bounded via eviction (|offset| + marginframes retained).output(the delayed value),is_available(whether the target frame was buffered yet), andreference_frame_number.offsetmust be<= 0; positive offsets are rejected both at manifest-validation time (literal ints) and at runtime (selector-provided values). True future look-ahead would require delaying the whole workflow output, which isn't possible on synchronous runtimes (WebRTC/webexec, single-image HTTP), so it is intentionally not offered.roboflow_core/time_travel@v1/TimeTravelare kept as accepted aliases so any workflow saved under the earlier name keeps resolving to this block.Related Issue(s): N/A
Type of Change
Testing
Test details:
tests/workflows/unit_tests/core_steps/fusion/test_frame_delay.py): past-offset lookup returning the previous frame's value,default_valuefallback before the buffer fills, zero-offset returning the current value, positive-offset rejection (both runtimerun()and manifest validation), per-video_identifierstate isolation, bounded-buffer eviction, and state persistence acrossrun()calls.tests/workflows/integration_tests/execution/test_workflow_with_frame_delay.py): a multi-frame workflow with two past-offset steps (-1and-2) driven byproperty_definitionframe numbers (model-free), asserting the correct delayed values,is_available, andreference_frame_numberacross a frame sequence.Checklist
SHORT_DESCRIPTION/LONG_DESCRIPTION; no separate docs site changes.Additional Context
main):frame_delay/__init__.py,frame_delay/v1.py,loader.py, and the two test files (5 files, +479).-1,-5), aligning a slow/delayed signal to the frame it belongs to, or remembering a value N frames ago (e.g. the dominant color 10 frames earlier).video_identifier, so it degrades on stateless/multi-replica remote HTTP runtimes and resets on restart.