Backtest Intent Data on the Deals You Already Won

The renewal call for your intent data subscription is next month, and the account manager has a slide ready. It shows three of your biggest wins from this year, each with a topic surge in the weeks before the deal closed. It is a persuasive slide. It is also an easy slide to make, because once you know which accounts bought, you can usually find a spike somewhere in their history.

The trap has a name, the Texas sharpshooter fallacy: fire at the side of a barn, then paint the target around the tightest cluster of holes. Before you renew, expand or cancel, run a test that cannot be painted after the fact. Freeze the signals as they looked before you ever talked to each account, and check whether they would have pointed you at the right ones. It takes a spreadsheet, your CRM and an afternoon. Building a full scoring model is what our free B2B intent data playbook is for. The backtest comes first.

The one question a backtest answers

If you had only the intent data, and none of the hindsight, would it have told your team where to look? A vendor dashboard cannot answer that for you, because it shows scores as they are now. You need scores as they were on a date that came before the first meeting.

Build the sample from your own CRM

1. Closed-won deals from roughly the last year. Take all of them if the list is short. Note the date of first meaningful contact for each, meaning the first reply, meeting or inbound form, and not the date someone added the account.
2. Closed-lost deals from the same period. These matter as much as the wins. If the data flags winners and losers alike, it is telling you who is shopping, which is useful but a different thing.
3. A random set of target accounts you never engaged. Pull them from your ideal-customer list at random, not by hand. Choosing them yourself lets hindsight back in through the side door.

Keep the groups roughly the same size so the comparison is fair. If your wins are few, say so in the write-up and treat the result as a direction rather than a verdict.

Freeze the signals before first contact

For each account, set a cutoff date some weeks before first contact, and pull what the intent data said as of that cutoff: which topics surged, how strongly and how recently. Ask the vendor for historical exports with the original timestamps. Scores that were recalculated later are no use here, because the recalculation may already reflect the deal.

If the vendor cannot give you the data as it looked at the time, write that down. It does not mean the product is bad. It means you cannot check it against your own outcomes, and that is worth knowing before you sign for another year.

Do the same with your first-party signals: pricing page visits, repeat visits from the same company, content downloads, trial sign-ups. You want to know whether the paid data adds anything on top of what your own site already told you.

One row per account is enough. Give the sheet these columns: account name, group (won, lost or never engaged), date of first contact, cutoff date, flagged by the paid data at the cutoff (yes or no), the topics behind the flag, flagged by your own site at the cutoff (yes or no), and final outcome. Fill it in before anyone looks at the totals, so nobody is tempted to nudge a borderline row.

Questions to send the vendor before the call

While you wait for the export, send the account manager a short list in writing. Good vendors answer these readily, and the answers make the backtest easier to read.

• How is each topic defined, and can we see the keywords or content categories behind the topics we track?
• How is activity matched to a company, and what happens with remote workers and shared networks?
• What baseline is a surge measured against: the company's own history, its industry or the whole dataset?
• How often are scores refreshed, and can we export them with the original timestamps?
• When a score is recalculated, does history get rewritten or kept as it was?

Read the grid, cell by cell

Sort every account into a simple grid: flagged or not flagged at the cutoff, and won or not won. Each of the four cells tells you something different.

• Flagged and won: the data pointed at a buyer early. Check how early. A signal that appears a few days before an inbound form arrives would not have changed anything your team did.
• Flagged and not won: false alarms. Some are unavoidable. The question is how much of your team's week they would have eaten.
• Not flagged and won: buyers the data missed. If this cell is crowded, the topics you track may not match how your buyers actually research.
• Not flagged and not won: quiet accounts that stayed quiet. This cell shows the data is not simply flagging everyone.

Two ratios fall out of the grid, and they are the ones data scientists call precision and recall. Precision asks: of the accounts the data flagged, how many bought? Recall asks: of the accounts that bought, how many did the data flag before first contact? A source can look good on one and poor on the other. Which one you care about depends on how many accounts your team can realistically work in a week.

What to do with the answer

If flagged accounts won noticeably more often than the random set, and the flags showed up well before first contact, the data is earning its place, and the next job is prioritizing it with a tiered model and a different play for each tier. If the paid signals added little beyond your own first-party data, you may be better served by capturing more of your own signals and trimming the subscription.

If the backtest could not be run at all because historical data was unavailable, negotiate for that data in the renewal, or run the test forward instead: freeze this month's scores, wait a quarter and grade them against what actually happened.

In fairness to the vendors, intent data was never sold as a crystal ball. A topic surge describes research behavior, and research is a noisy stand-in for buying. What you want from the backtest is a result you can defend in the renewal meeting, even if it is a modest one.

Take it further

The backtest tells you whether your signals deserve a model. The free B2B Intent Data + AI: The Account Targeting Playbook is the next step: a ranked taxonomy of first-, second- and third-party signals, a scoring framework with weights and decay functions, a five-tier prioritization system with activation plays, the stack architecture connecting Bombora, Clay, HubSpot and your own data, the first-party signals worth capturing on your own site, precision and recall in more depth, and a six-week rollout. Run the backtest this month and bring the grid with you when you open it.

← Back to all articles