How to run a sales tech evaluation

Start before the vendor list. The buying committee needs one page that says what business outcome the new tool must move, which metric proves it moved, and what the current baseline is. The problem statement forces the sponsor to articulate why the status quo is unacceptable, which separates real evaluations from category window-shopping. Pick one primary metric (meetings booked per rep, pipeline sourced per SDR, forecast accuracy, time to first deal) plus two guardrails that must not regress (CAC, rep ramp time, customer data accuracy). Share the document with every stakeholder before any demo is scheduled. Vendors can read it, too; the ones who refuse to sell against your metric are the ones you can short-list out early.

• Name the business outcome in one sentence, not a feature set or a category
• Pick one primary metric with a current baseline and a target delta the tool must deliver within 90 days of go-live
• Add two guardrail metrics that must not regress, so the evaluation does not optimize one number at the expense of another
• Circulate the one-pager to the full buying committee and get written sign-off before building the vendor list

Vendors that email the loudest are rarely the right short-list. Build the long-list from analyst reports, peer referrals in RevOps communities, and the stack audit you ran earlier, then cut it down. A healthy short-list is three to five vendors: enough to force comparison, few enough to run a serious evaluation inside eight weeks. Each vendor on the short-list should clear three gates before they get an invite: category fit against the problem statement, native integrations with your CRM and identity provider, and a reference customer in your segment who is willing to take a call. Document the gates so disqualifications are evidence-based, not personality-based.

• Start the long-list from analyst coverage, RevOps community referrals, and peers at non-competing companies, not from vendor outbound
• Score every long-list entry against three gates: category fit, native integration coverage, and a willing reference customer in your segment
• Cut the long-list to three to five short-list vendors, no more, so the evaluation stays inside the 6 to 8 week window
• Write a one-sentence rationale for every long-list drop so late-stage lobbying cannot resurrect a disqualified vendor

A good RFP is a scorecard with questions, not a questionnaire with a scorecard bolted on. Build both documents in the same week, mapped line for line. The RFP covers five sections at minimum: problem fit, product capability against named use cases, integrations and data model, security and compliance, and commercial terms. The scorecard assigns a weight to each section, with product capability and integrations typically carrying the most. Keep the weights visible to the full committee and resist weighting changes once scoring starts. Send the RFP with a two-week response window and a named point of contact so vendor questions do not fragment across inboxes.

• Structure the RFP in five sections: problem fit, product capability, integrations and data model, security and compliance, commercial terms
• Weight the scorecard before any vendor answers arrive, so weights are not back-fitted to a favorite
• Attach three to five named use cases to the capability section and ask vendors to answer against those, not against a feature matrix
• Set a two-week response window with a single point of contact for vendor questions, and publish answers to every short-listed vendor at the same time

An unstructured demo is a sales pitch, and a sales pitch does not evaluate anything. Send every short-listed vendor the same three to five demo scenarios in writing a week before their session, built from the use cases in the RFP. Ask them to demo on their own instance against your scenarios, not on a canned walkthrough. The scorecard carries over from the RFP so each committee member rates the same capabilities on the same scale across every vendor. Keep every demo to the same length, record them with consent, and debrief inside 24 hours while the memory is fresh. Scenario-based demos surface integration gaps and workflow friction that polished pitches hide.

• Write three to five demo scenarios in business language and send them to every short-listed vendor one week ahead
• Keep every demo to the same length (90 to 120 minutes is standard) and the same audience, so comparisons are fair
• Record the sessions with vendor consent and tag each capability score to a specific moment in the recording
• Hold a 30 minute committee debrief within 24 hours of every demo so impressions are captured before the next vendor

The deepest cuts in a sales tech evaluation happen in security review and integration review, not in the demo room. Pull the vendor's SOC 2 Type II report, their data processing agreement, and their penetration testing summary. Walk IT and security through authentication, SCIM provisioning, data residency, and incident response SLAs before the pilot starts. On the integration side, test the CRM integration against your actual object model (custom fields, record types, sharing rules) and confirm that identity, SSO, and audit logging work end to end. A beautiful product with a brittle CRM sync will cost more to maintain than the license saves, so this review has veto power equal to the capability review.

• Collect SOC 2 Type II, pen test summary, DPA, and a sub-processor list from every short-listed vendor and route them to security review inside the pilot window
• Walk through authentication, SCIM provisioning, data residency, and incident response SLAs with IT, with written answers against your security questionnaire
• Test the CRM integration against your actual object model, including custom fields and sharing rules, not a generic sandbox
• Confirm the vendor supports the API, webhook, and event volumes your workflow will drive at full rollout, with explicit rate limits in writing

The pilot is where the scorecard meets reality. Carve out a 3 to 4 week pilot window with the top one or two vendors from the scorecard, on real data, with the pilot cohort you lined up as a prerequisite. Reps need enough volume to form a judgment: at least 20 to 30 workflows each, inside the time box. Set three or four quantitative milestones the pilot must hit (adoption rate, workflow completion time, data accuracy, meetings booked) plus a short qualitative survey. Keep the manager of each pilot rep accountable for the attention budget, because a pilot that gets squeezed by quarter-end is a pilot that generates no signal. Debrief the pilot in a committee meeting, not in email.

• Set the pilot window to 3 to 4 weeks on a real production or production-like instance, not a sandbox
• Define three or four quantitative milestones in advance (adoption, time to complete a workflow, data accuracy, outcome metric) plus a qualitative rep survey
• Protect the pilot cohort's attention with manager-level air cover; a pilot squeezed by quarter-end produces no signal and should be rescheduled, not reinterpreted
• Close the pilot with a committee debrief that scores each vendor against the pilot milestones before any renewal or discount conversation

Reference calls are where polished evaluations become honest ones. Ask every finalist for three references, and insist that at least two be customers in your segment and at least one be a customer who went live inside the last 12 months. Prepare a short structured script and reuse it on every call so answers are comparable. Ask what the vendor got wrong during onboarding, where the integration broke, how support behaves at renewal, and what the reference would change if they could rerun the purchase. Treat marketing-approved reference customers as a floor, not a ceiling: ask your RevOps community and LinkedIn network for off-list references too, because those calls are where the real risk shows up.

• Ask every finalist for three references and insist at least one went live in the last 12 months, so onboarding memory is fresh
• Reuse the same structured script on every call so answers are comparable and bias is minimized
• Ask what onboarding got wrong, where integrations broke, how renewal conversations felt, and what they would change if they could rerun the purchase
• Supplement vendor-provided references with at least one off-list customer sourced through your network, so the view is not curated

By the time the scorecard, pilot, and references point to a winner, you have the leverage of a credible willingness to leave. Use it. Negotiate commercial terms before the executive signs the business case, not after. Rank the terms you need: total annual price, ramp pricing on seats, multi-year renewal escalator cap, implementation and professional services credits, SLA remedies, data export clauses, and termination for convenience in the event of a merger or acquisition. Share the runner-up vendor name honestly if you are asked; procurement teams respect that transparency and often give better terms when the competitive picture is clear. Document every term in the master agreement, not in email.

• Rank the terms you need in writing before the first commercial call: price, ramp, escalator cap, services credits, SLA, data export, termination rights
• Negotiate against the runner-up vendor explicitly, with their name on the table if asked, because a credible alternative is the only real source of leverage
• Trade a signing concession for structural terms: a capped multi-year escalator or an SLA remedy is worth more than a one-time discount
• Push every verbal commitment into the master agreement or an order form amendment before signature, because contract silence is vendor silence

An evaluation ends in a one-page decision memo, not a signed order form. The memo captures the problem statement, the short-list, the scorecard outcome, the pilot results, the reference themes, and the commercial terms, and it goes into the knowledge base for the next evaluation. Pair the memo with a rollout plan that names the implementation owner, the integration milestones, the training schedule, the adoption target for 30, 60, and 90 days, and the review cadence with the executive sponsor. The decision memo compounds across future evaluations: by the second or third one, the committee moves faster because the pattern is already in writing.

• Write a one-page decision memo covering problem, short-list, scorecard outcome, pilot results, references, and final commercial terms
• Attach a 30, 60, 90 day rollout plan with named owners for implementation, integration, training, and adoption measurement
• Store the memo in a RevOps or sales operations knowledge base so the next evaluation starts from a template, not a blank page
• Book a 60 and 90 day review with the executive sponsor against the primary metric the evaluation was framed around in step 1