About this prototype
A scorecard that can be run on any mental model. Give it the framework an investor already uses to size up a company — the business-quality and pharma-pipeline lenses here, but just as easily pricing power, moat durability, or management quality — and it scores every name in a universe straight from public filings, leaving a number, the reasoning, and the exact line it rested on. That judgment normally stays locked in prose, out of reach of any systematic process. The real advantage is that a scorecard like this can be put on a schedule: run it monthly and you see how each company's standing shifts over time, not just where it sits today.
Two frameworks are run here. The first applies a Business Quality Framework to S&P 500 companies — the size and growth of the market (Market growth), the company's share and how defended it is (Competitive position), whether it can set and keep price (Pricing power), and how well revenue becomes profit (Profitability) — as twelve sub-questions, each scored on its own evidence. The second applies a Pharma Pipeline Framework to US large-cap pharma, reading the R&D pipeline the way an analyst would with a clinical scientist beside them — how many credible bets it holds (Depth), how likely they are to pay off (Quality), what they are worth if they work (Commercial), and whether they arrive in time to replace revenue going off patent (Durability) — again as twelve sub-questions. Two rules hold throughout: every score cites the filing excerpt or web quote behind it, and where the evidence is thin the attribute is left near zero rather than guessed — the model works only from what it retrieves, never from what it already knows about the company.
Business Quality: data as of 2026-07-21; universe of 46 companies — the 36 S&P 500 Consumer Staples names plus one company from each of the other ten GICS sectors. Pharma Pipeline: data as of 2026-09-30; universe of 10 US large pharma — LLY, JNJ, MRK, PFE, BMY, ABBV, AMGN, GILD, REGN and VRTX. Head back to the ranking and click any company to drill down to its pillars, sub-question justifications, and source links.
How a score is built
Unit of analysis. Each (company, sub-question) pair is
scored once, on its own — twelve scores per company, each seeing only its own
sub-question's rubric, sign convention, and its own retrieved evidence, so attributes
never bleed into one another. Twelve sub-questions means twelve separate model calls per
company.
Two point-in-time evidence sources are gathered and stacked into one numbered set per sub-question, filings first:
- SEC filings (the backbone). 10-K and 10-Q filings, pulled by hybrid dense + lexical search over a section-aware chunk index, restricted to documents dated on or before the as-of date — the point-in-time guarantee — within an 18-month window, and narrowed to the passage most relevant to the sub-question.
- Web (corroboration). Dated findings from financial press, trade and industry sources, and company disclosures, via the model's built-in web search — each a verbatim quote with publisher, date and URL, kept only if dated on or before the as-of date. Filings are deliberately out of scope for this leg, so it adds what the filings do not carry.
Scoring. One structured-output call per sub-question to the scoring
model (gpt-5.6-terra; web search runs on gpt-5.4-mini). Given
an analyst persona, evidence-reading rules, the scale, and the sub-question's rubric, the
model returns — in this order — a one-line reading of what the attribute means for
this business, the points for a positive score, the points against, then the
integer score, a confidence level, a justification, and the citation indices. Forcing it
to commit to the reading and to both sides before the number is what keeps the
score from being a story told to justify a figure already picked.
Sign and magnitude. The sign follows the sub-question's stated convention — does the evidence help or hurt the attribute — while the magnitude tracks how firmly the evidence establishes it: a decisive, quantified, corroborated finding earns an extreme, a lone qualitative mention earns a marginal score, and absent or off-topic evidence lands at 0. The model is shown no distributional prior — it never sees how other companies scored — so it cannot drift toward a centre it cannot see.
Confidence describes the evidence, not the model's mood: high takes several independent blocks that agree, at least one of them quantified; low means the score leans on a single block, on an unquantified assertion, or on sources that merely echo one another.
Reproducible & point-in-time. A whole run is defined by one declarative spec — models, scale, universe, retrieval and web parameters, and the pillars with their sub-questions and rubrics. Prompts, scale and model IDs are frozen per dated vintage and hashed into a manifest, so editing a prompt or rubric spawns a new vintage rather than quietly mixing revisions; the whole dataset regenerates from the spec.
The scoring rubric
Scores are integers from −5 to +5 (−5 strongly undermines the attribute, +5 strongly supports it, 0 neutral or insufficient evidence). Each sub-question brings its own sign convention and a magnitude ladder — what a decisive score looks like versus a marginal one, and what to discount — while the framework supplies the discipline a spec author cannot loosen: the analyst persona, the evidence-reading rules (boilerplate is not adverse, self-description is promotional, repetition is not corroboration, measured beats asserted), the ordered procedure, and the scale. The Business Quality Framework's four pillars and their twelve sub-questions:
- Market growth — market growth; demand durability; market headroom (growth runway).
- Competitive position — market position (current share); share trajectory; barrier durability.
- Pricing power — pricing power (structural ability); contract structure (escalators / pass-throughs); realized pricing (capture gap).
- Profitability — margin level and trend; operating leverage and cost discipline; capital intensity and returns.
The Pharma Pipeline Framework ranks large-cap US pharma on the strength of the R&D pipeline, with four pillars that follow the order an analyst asks the questions in — how many credible bets, how likely they are to pay off, what they are worth if they work, and whether they arrive in time to replace revenue going off patent. Read the pillars alongside the composite: a deep pipeline facing a near-term cliff and an average one with no cliff can share a composite and be opposite investment cases. Its four pillars and twelve sub-questions:
- Depth — late-stage depth; next wave (early pipeline); indication optionality.
- Quality — evidence strength; biology de-risking; R&D track record and return on R&D.
- Commercial — size of need; competitive position; pricing and access.
- Durability — patent cliff exposure and concentration; catalysts (next 12–18 months); bridge coverage.
Anchoring each magnitude on the evidence — rather than on how other names scored — keeps scoring about as steady as the model allows, so the same evidence tends to land on the same score when a run is repeated.
Where it falls short
Three things this prototype does not do well, and what a production build would do about each.
- The twelve sub-scores are not all equally solid. The ones anchored in structured disclosure — margins and returns, realised price contribution, market position — are the most dependable and the best candidates for systematic use. The narrative ones — barrier durability, demand durability, and other judgment calls — draw on prose and news and carry more low-confidence reads. In production the structured attributes could run on their own each month, while the softer ones sit behind an analyst's review and refresh less often.
- The evidence base is narrow. The scores read two sources — SEC filings and a general web search — and nothing else. A lot of what actually moves these judgments is discussed first, and most candidly, in places the model never sees: earnings-call and investor-day transcripts, sell-side and industry research, expert-network interviews and channel checks, patent and regulatory filings. The narrative attributes especially — barrier durability, demand durability, competitive dynamics — would sharpen with that material in the retrieval set. Production would widen the corpus (transcripts first, as the cheapest high-signal addition), each source held to the same point-in-time and citation discipline as the filings.
- Point-in-time is mechanical on filings, best-effort on the web. The filings leg is genuinely point-in-time — documents are cut off at the as-of date. Web publication dates and quotes are asserted by the model against the source and the pages are not snapshotted, so a link can rot or a quote drift. And because the scoring model's own knowledge cannot be rolled back to a past date, a real history is built forward by running each month, not backfilled. Production would snapshot every web source at collection and start the monthly cadence straight away.
Not investment advice. These are systematic, evidence-graded attribute scores meant as a screen and an audit trail for a fundamental or quant process — a demonstration of method, not a recommendation.