Persona Coherence

Persona coherence evaluation tests whether an agent maintains a consistent identity across all dimensions simultaneously. Rather than testing individual aspects, this evaluates the agent as a whole person.

What it tests

Does the agent maintain a coherent identity across interactions?
Are skill usage, personality, communication, and thinking patterns internally consistent?
Does the agent's behavior match its declared persona?
Is the identity stable under pressure or unusual inputs?

Scorer: `persona_coherence`

The persona_coherence scorer evaluates end-to-end identity consistency across all available dimension scores:

testcase.ScorerConfig{
    Name: "persona_coherence",
    Config: map[string]any{
        "persona_ref":     "senior-engineer",
        "min_consistency": 0.75,
        "dimensions":      []string{"skill", "trait", "communication", "cognition"},
    },
}

Scenario: `persona_coherence`

The persona_coherence scenario generator creates test cases that exercise multiple dimensions simultaneously:

Case{
    ScenarioType: testcase.ScenarioPersonaCoherence,
    Input:        "A junior developer asks you to review their first pull request. It has a security issue and some style problems.",
    Context: map[string]any{
        "persona":          "senior-engineer",
        "expected_traits":  []string{"thoroughness", "patience"},
        "expected_skills":  []string{"code-review"},
        "expected_tone":    "professional but supportive",
    },
}

This scenario tests that the agent:

Uses code review skills correctly (skill dimension)
Shows thoroughness and patience (trait dimension)
Communicates professionally but supportively (communication dimension)
Follows an analytical then reflective cognitive pattern (cognition dimension)

Dimension score

result.DimensionScores["persona"] // 0.0 to 1.0

The persona score is typically derived from the consistency across all other dimension scores. A high persona coherence score means the agent performs well across all dimensions in a way that forms a coherent identity.

PersonaRef

Suites can declare a PersonaRef field that references an agent persona by name. When set, persona-aware scorers use this reference to look up expected traits, skills, and behaviors:

s := &suite.Suite{
    Name:       "senior-engineer-eval",
    PersonaRef: "senior-engineer", // references an agent persona
}

Use cases

End-to-end evaluation of a fully configured AI agent
Regression testing after persona configuration changes
Comparing two different persona configurations for the same role
Validating that a persona behaves consistently across diverse scenarios