1---2name: ai-answer-panel3description: >-4 Measures a B2B SaaS brand's visibility in AI answers like a poll: builds a fixed prompt panel from the company's5 buying moments, runs each prompt 5 or more times in clean sessions across ChatGPT, Perplexity, Gemini and Google AI6 Overviews, and reports how often the brand appears and how often it is recommended, per engine, with confidence7 intervals and never a rank. Returns PANEL READY, VISIBLE AND TRUSTED, VISIBLE NOT TRUSTED, TRUSTED NOT VISIBLE,8 NOT VISIBLE, NOT MEASURABLE or BLOCKED. Use it monthly, or before and after AI visibility work. Not for choosing9 which cited pages to pursue (use cited-pages) or checking crawler access (use agent-readiness).10license: MIT11compatibility: >-12 Works in Claude, ChatGPT, Codex, Cursor and any agent that reads SKILL.md. Uses an AI visibility tool's run-level13 export if the reader has one, otherwise runs the reader pastes. Python 3 is optional, for the rate math.14metadata:15 author: SHAKEUP16 version: "1.0.0"17 stage: check18 plugin: seo-ai-search19 homepage: https://www.shakeuphq.com/skills/ai-answer-panel20---21 22# AI answer panel23 24Tells a company how often AI answers name it for the questions its buyers ask, and whether they recommend it, with enough runs to trust the figure.25 26## Purpose27 28- One job: build or run a fixed prompt panel and report appearance and recommendation rates per engine, with a diagnosis.29- Stage: check. It ends in a verdict with the rule that fired, and a severity on each gap.30- It measures rates, never ranks: answer lists change on almost every run, so a position in one list means nothing.31- It never reports a rate from fewer runs than the panel rules allow, and never blends engines into one score without showing each.32 33## Use when / not34 35| Situation | Use |36| --- | --- |37| "Does ChatGPT recommend us?" or a monthly AI visibility report | this skill |38| Before and after AI visibility work, to see if it moved | this skill, same panel both times |39| Deciding which third-party pages to get onto | `cited-pages`, on this skill's cited sources |40| AI crawlers may not reach the site at all | `agent-readiness` first; a blocked site can't be cited |41| Checking a tactic for spam risk | `ai-search-risk` |42 43## Inputs44 45| Input | Required | Ways to provide, in order | If missing |46| --- | --- | --- | --- |47| Brand name and aliases | yes | `## Company` brand terms; the reader's request | ask once; else `BLOCKED` |48| Category, buyers, buying moments, competitors | yes, to build the panel | the profile; the reader's answers | ask for them in one message; a panel without buying moments covers category and comparison only, coverage `partial` |49| The panel: fixed prompts with ids | no | `## AI answer panel` in the profile; a saved panel file from a past run; built in step 2 | build it; verdict `PANEL READY` until runs exist |50| Runs: per run or per prompt and engine | no | a connected AI visibility tool's run-level export; runs the reader pastes; runs the agent makes in clean sessions, if it can | `PANEL READY`: hand over the panel and the protocol |51| Last period's runs on the same panel | no | the saved results table | trend `not available` |52 53## Profile read54 55From `company-profile.md`: `## Company` (name, brand terms, category, pricing), `## Product`, `## Buyers` and `### Buying group`, `## Buying moments`, `## Competitors`, `## Accounts and data` (connected tools), `## AI answer panel` (the saved panel, engines, runs per prompt, last run), and every `## Learnings` entry for `ai-answer-panel` with status `noted`, `fact` or `accepted` (including the saved panel version).56 57No profile: ask for the brand and its aliases, the category in the buyer's words, the buyer, 3 competitors and the moments that start a search, in one message. Say that `company-profile` and `buying-moments` save this next time.58 59## Steps60 611. **Apply learnings.** Read them first: the panel version in use, prompts the reader retired, and any accepted change to the thresholds.62 632. **Build or load the panel.** If `## AI answer panel` in the profile (or a saved panel file) has one, use it unchanged (a reworded prompt is a new prompt). Otherwise build one with `references/panel.md`: 20 to 40 prompts across the 6 families (category, comparison, budget, problem, use case, integration), each written as the buyer types it, seeded from `## Buying moments`, with the reader's brand in none of them. Give each an id and a family.64 653. **Collect the runs** with the protocol in `references/panel.md`: 5 or more runs per prompt per engine, clean sessions, at least 2 engines, all within 7 days. Without runs, stop here: verdict `PANEL READY`, hand over the panel, the protocol and the results table format.66 674. **Record each run:** every brand named, whether the answer recommends the reader's brand for the buyer described, and every cited URL. A Google AI Overview that didn't appear is `NO_ANSWER`, left out of the appearance rate and counted as a trigger rate.68 695. **Compute the rates.** With Python:70 71 ```72 python3 scripts/panel_rates.py --csv runs.csv --brand "<brand>" --alias "<alias>" --competitor "<competitor>" --prior last_month.csv73 ```74 75 Without Python, per engine: appearance rate is answered runs that name the brand divided by answered runs; recommendation rate is runs that recommend it divided by runs that name it. For a 95% interval, use the Wilson formula, or as a rough guide take the rate plus and minus 1 divided by the square root of the runs. Say which way you ran it.76 77 | Rule | Why |78 | --- | --- |79 | A prompt counts on an engine only with 5 or more answered runs | below 5, one odd answer swings the figure |80 | An engine gets a rate only with 10 or more counted prompts | fewer prompts measure a few phrasings |81 | The panel needs 2 or more engines with a rate | one engine's view is not the market's |82 | The headline rate is the mean of per-engine rates | an engine with more runs shouldn't dominate |83 | A change against last period is real only when the 2 intervals don't overlap | otherwise it is the sample moving, not the brand |84 856. **Diagnose.** Visible: appearance 50% or more, the point where a buyer asking is more likely than not to see the brand. Trusted: recommended in 50% or more of the answers that name it. Apply per engine and to the headline.86 877. **List the gaps** with severities: `critical` when the brand is named in 0 runs of a high-intent family (category, comparison, budget) on an engine with a rate; `warning` when a family's rate is under 20%; `suggestion` when one engine's interval sits wholly below another's (an engine gap). Add competitors' appearance rates on the same runs.88 898. **Pass on the cited sources.** List the domains cited most, by how many prompts cite them, for `cited-pages`.90 91## Check and verdict92 93| Verdict | Condition (first match wins) |94| --- | --- |95| `BLOCKED` | No brand name after asking |96| `PANEL READY` | A panel exists or was built, and there are no runs yet |97| `NOT MEASURABLE` | Fewer than 2 engines with 10 or more prompts at 5 or more runs each |98| `VISIBLE AND TRUSTED` | Appearance 50% or more, recommendation 50% or more |99| `VISIBLE NOT TRUSTED` | Appearance 50% or more, recommendation under 50%: work on independent proof for your claims |100| `TRUSTED NOT VISIBLE` | Appearance under 50%, recommendation 50% or more: work on the sources answers draw from |101| `NOT VISIBLE` | Both under 50%: fix how clearly the brand is described first, then the sources |102 103Named stops: `PANEL READY` waits for the reader to run the prompts (or connect a tool) and paste the results. `NOT MEASURABLE` reports the raw counts, labelled as counts and not rates, and asks for the missing runs. No other stops.104 105## Learn106 107Append a learning to the profile when:108 109- the panel is built or changed: a `fact` with the panel version, the prompt count per family and where the file is kept (the panel must not drift);110- the reader overrides a recommended or not-recommended call on a run (a `proposal` to tighten what counts as recommended);111- a period's rates are measured: a `fact` with the headline rates, per-engine rates and the date, so the next run reads the trend;112- a threshold keeps misclassifying for this market (a `proposal`, applied only once accepted).113 114Example: `- 2026-10-24 | ai-answer-panel | fact | Panel v1: 24 prompts (category 5, comparison 5, budget 3, problem 4, use case 4, integration 3), kept in the profile's AI answer panel section | Why: built from buying moments | Status: noted`115 116After a run, update `## AI answer panel`: panel version, engines, runs per prompt, last run date and each prompt's appearance rate. Without file access, end with that section as a paste-ready block.117 118## Output119 120```121Verdict: <verdict> · Coverage: <complete | partial (what's missing)> · Rates: <script | by hand>122Panel: v<n>, <n> prompts · Engines: <list> · Runs: <n> answered, <n> with no AI answer · Window: <dates>123 124Headline: appearance <n>% · recommendation <n>% (mean of per-engine rates)125 126| Engine | Prompts counted | Runs | Appearance (95% interval) | Recommendation (95% interval) | Diagnosis | Change since last period |127 128| Family | <engine 1> appearance | <engine 2> appearance | ... |129 130Gaps:131| Severity | Engine | Family | Finding |132 133Competitors on the same runs: <name: appearance rate per engine>134Cited most (for cited-pages): <domain: prompts, engines>135Not counted: <thin cells, prompts with fewer than 5 runs>136Learnings applied: <list, or none>137```138 139For `PANEL READY`, the output is the panel (id, family, buying moment, persona, prompt), the protocol and the results table format.140 141Then the handoff block.142 143## Hand off144 145```handoff146skill: ai-answer-panel147version: 1.0.0148verdict: VISIBLE NOT TRUSTED149coverage: complete150for: <brand>, panel v1, <month>151outputs:152 appearance_rate: 0.63153 recommendation_rate: 0.19154 engines: {chatgpt: 0.60, perplexity: 0.66}155 critical_gaps: [comparison on chatgpt, comparison on perplexity]156 cited_domains: [g2.com, review-site.example, vendor-a.example]157next: [cited-pages, claim-check]158open: [add Gemini and AI Overviews runs next month]159```160 161`next` by verdict: `VISIBLE NOT TRUSTED` goes to `[claim-check, cited-pages]` (proof first); `TRUSTED NOT VISIBLE` and `NOT VISIBLE` go to `[cited-pages, agent-readiness]`; `PANEL READY` waits on the reader.162 163## Failure modes164 165| Problem | Do this |166| --- | --- |167| The reader's tool gives one visibility score, no runs | Don't use the score as a rate; ask for its run-level export |168| A prompt was reworded mid-month | Treat it as a new prompt; the old one has fewer runs and drops out |169| The brand name is a common word | Match it only as the brand (capitalized, or beside the category); spot-check 10 runs by reading them |170| Answers name the brand with the wrong category or price | Count the appearance, and log the error as a finding for `company-profile` and the brand's own pages |171| The reader wants a rank ("we're #2 in ChatGPT") | Explain that lists rarely repeat; report the appearance rate and the share of runs where it came first |172| AI Overviews rarely trigger for the panel | Report the trigger rate; don't fill the gap with another engine's figure |173 174## Influenced by175 176- SparkToro's study of AI recommendation consistency, which found lists almost never repeat across runs and argued for appearance rate across many runs instead of rank: https://sparktoro.com/blog/new-research-ais-are-highly-inconsistent-when-recommending-brands-or-products-marketers-should-take-care-when-tracking-ai-visibility/177- Kevin Indig's analysis of citations across ChatGPT, Perplexity and AI Overviews, which found most cited pages appear in only one engine and argued for per-engine measurement: https://www.growth-memo.com/p/the-consensus-gap178- SHAKEUP's AI visibility playbook: the 6 prompt families, clean-session runs, and the split between being named and being recommended.