Skip to content

FEAT Migrate consumers to TargetConfiguration capability checks#1645

Open
hannahwestra25 wants to merge 15 commits intomicrosoft:mainfrom
hannahwestra25:hawestra/move_to_target_config
Open

FEAT Migrate consumers to TargetConfiguration capability checks#1645
hannahwestra25 wants to merge 15 commits intomicrosoft:mainfrom
hannahwestra25:hawestra/move_to_target_config

Conversation

@hannahwestra25
Copy link
Copy Markdown
Contributor

Summary

Replaces scattered isinstance(target, PromptChatTarget) checks with explicit, declarative capability requirements validated against TargetConfiguration. Consumers now care about capabilities, not concrete target types.

The bigger picture

Next step in the migration from type-based gating to the TargetConfiguration / TargetCapabilities / CapabilityHandlingPolicy model. With this change, scorers, converters, and attacks can accept any PromptTarget whose configuration satisfies their declared requirements — natively or via an ADAPT normalization pipeline — opening non-chat targets to flows previously locked to chat-shaped APIs.

What changed

  • set_system_prompt moved from PromptChatTarget onto PromptTarget (capability gates behavior, not subclass identity).
  • TargetRequirements gains:
    • required — satisfied by native support or ADAPT policy.
    • native_required — must be natively supported; ADAPT is rejected.
    • validate(*, target) — enforces both tiers, called automatically by base classes.
  • CHAT_CONSUMER_REQUIREMENTS introduced as the shared constant for scorers/converters.
  • Scorers and converters declare TARGET_REQUIREMENTS = CHAT_CONSUMER_REQUIREMENTS and validate in __init__; chat_target / converter_target params typed as PromptTarget.
  • AttackStrategy exposes TARGET_REQUIREMENTS and validates the objective_target in its __init__. CrescendoAttack, ChunkedRequestAttack, and MultiPromptSendingAttack declare their MULTI_TURN need as native_required — adaptation would silently collapse their multi-turn semantics, so it's rejected at construction time.
  • ConversationManager.set_system_prompt accepts PromptTarget and calls ensure_can_handle(SYSTEM_PROMPT). Prepended-conversation routing switched from isinstance(PromptChatTarget) to a capability check.
  • Tests cover validate() across native support, ADAPT, RAISE, and empty-required cases.

Out of scope

No behavior change for targets that already satisfied the old PromptChatTarget contract. Custom adaptation still plugs in through CapabilityHandlingPolicy / normalizer overrides.

Tests and Documentation

Added/Updated tests

if self._objective_target is None:
raise ValueError("objective_target must be set before creating attacks")
if not isinstance(self._objective_target, PromptChatTarget):
if not self._objective_target.configuration.includes(capability=CapabilityName.MULTI_TURN):
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be using TargetRequirements for scenarios like we do with attacks?

# conversation into a single prompt and silently break the attack's
# semantics. Declare EDITABLE_HISTORY as ``native_required`` so adaptation is
# rejected at construction time.
TARGET_REQUIREMENTS = TargetRequirements(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because of backtracking right? Maybe add the reason in the docstring above (NIT)

# semantics. Declare EDITABLE_HISTORY as ``native_required`` so adaptation is
# rejected at construction time.
TARGET_REQUIREMENTS = TargetRequirements(
native_required=frozenset({CapabilityName.EDITABLE_HISTORY}),
Copy link
Copy Markdown
Contributor

@jsong468 jsong468 Apr 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, if I'm understanding correctly, doesn't this mean _GPT_4O and many other targets won't support crescendo? based on the known capability profiles section of target_capabilities.py? maybe just that section needs to change to align with OpenAIChatTargets supporting editable history.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants