Skip to content

fix(workflows): reject a non-string prompt in prompt-step validate()#3582

Open
Noor-ul-ain001 wants to merge 1 commit into
github:mainfrom
Noor-ul-ain001:fix/prompt-nonstring-prompt-parity
Open

fix(workflows): reject a non-string prompt in prompt-step validate()#3582
Noor-ul-ain001 wants to merge 1 commit into
github:mainfrom
Noor-ul-ain001:fix/prompt-nonstring-prompt-parity

Conversation

@Noor-ul-ain001

Copy link
Copy Markdown
Contributor

Summary

PromptStep.execute str()-coerces config['prompt'] and dispatches the result to the integration CLI as the model's instructions:

prompt = evaluate_expression(prompt_template, context)
if not isinstance(prompt, str):
    prompt = str(prompt)   # <-- a list/None/int becomes its Python repr

But PromptStep.validate only checked that prompt was present, not that it was a string — the exact parity gap the sibling ShellStep already closes for its run field (which cites "the command-step input/options and gate options type checks" as precedent).

So a YAML authoring slip like prompt: [review, this] or a bare prompt: (YAML null) passed validation, then execute sent the Python repr ("['review', 'this']", "None") to the LLM verbatim — silently wrong instructions, no error, and a COMPLETED status. The engine does not auto-validate step config (load_workflow explicitly defers validation to validate_workflow()/engine.validate()), so validation is the only place this surfaces before dispatch.

Fix

Extend validate to reject any non-string prompt, mirroring the shell-step run and command-step input/options type checks:

elif not isinstance(config["prompt"], str):
    errors.append(
        f"Prompt step {config.get('id', '?')!r}: 'prompt' must be a "
        f"string, got {type(config['prompt']).__name__}."
    )

A {{ ... }} expression is still a str, so it stays valid.

Testing

  • Adds test_validate_rejects_non_string_prompt (parametrized over null/list/int/dict), mirroring TestShellStep::test_validate_rejects_non_string_run.
  • Adds test_validate_accepts_expression_prompt to confirm an expression prompt still validates.
  • Full TestPromptStep + validation suite passes (85 selected, 0 failures).

🤖 Generated with Claude Code

`PromptStep.execute` str()-coerces `config['prompt']` and dispatches the
result to the integration CLI as the model's instructions. But its `validate`
only checked that `prompt` was *present*, not that it was a string — the exact
parity gap the sibling `ShellStep` closes for `run`.

So a YAML authoring slip like `prompt: [review, this]` or `prompt:` (null)
passed validation, then `execute` sent the Python repr (`"['review', 'this']"`,
`"None"`) to the LLM verbatim — silently wrong instructions with no error and a
COMPLETED status. The engine does not auto-validate step config
(`load_workflow` explicitly defers validation), so validation is the only place
this surfaces before dispatch.

Extend `validate` to reject any non-string `prompt` with the shell-step's
phrasing ("'prompt' must be a string, got <type>"), mirroring the shell `run`
and command `input`/`options` type checks. A `{{ ... }}` expression is still a
str, so it stays valid. Adds regression coverage for non-string prompts
(null/list/int/dict) and confirms an expression prompt still validates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Noor-ul-ain001
Noor-ul-ain001 requested a review from mnriem as a code owner July 17, 2026 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant