baselineRFIHow many pre-built integrations/connectors does the platform ship with for common security tools (SIEM, EDR, firewall, ticketing, identity), and how is a custom integration built for a tool without an out-of-box connector?weight 3
Answer key — what a strong answer shows
Look for a specific integration count and a documented path (API/SDK) for custom connectors, not just 'we integrate with everything.'
baselineRFPDescribe playbook authoring — is it a visual/no-code builder, code-based (Python), or both — and provide a customer reference for time-to-build and time-saved on a specific common playbook (e.g., phishing triage).weight 3
Answer key — what a strong answer shows
Strong answers state a concrete time-to-build figure and time-saved metric from a real customer, not generic 'faster response' claims.
baselineRFIWhat percentage of alerts can realistically be fully auto-closed without analyst involvement for common use cases (e.g., known-benign phishing reports), and what safeguards exist to prevent an incorrect auto-close of a true positive?weight 3
Answer key — what a strong answer shows
Look for an honest percentage (not '100% automation') and a described safeguard, e.g. confidence thresholds or human-in-the-loop for ambiguous cases.
baselineRFIHow does the platform handle playbook version control, testing, and rollback — can a playbook be tested in a dry-run/simulation mode before being enabled in production?weight 3
Answer key — what a strong answer shows
Dry-run/simulation testing before production enablement is a meaningful safety feature; a vendor with no answer likely expects customers to test in production.
baselineRFPDetail case management capabilities — does the platform provide native incident case management (timeline, evidence, collaboration) or does it only orchestrate actions across other tools that retain the case record?weight 3
Answer key — what a strong answer shows
Native case management with a full timeline/audit trail is stronger than a pure orchestration engine with no persistent case record of its own.
baselineRFIWhat metrics/reporting does the platform provide on automation ROI — mean-time-to-respond (MTTR) reduction, analyst hours saved — and is this measured automatically or does it require manual tracking?weight 3
Answer key — what a strong answer shows
Look for automatically-tracked MTTR/hours-saved metrics tied to specific playbooks, not a one-time consulting estimate.
baselineRFPExplain how the platform handles multi-tenant or MSSP use cases — can playbooks and integrations be shared across tenants while keeping tenant data and credentials isolated?weight 3
Answer key — what a strong answer shows
Relevant for MSSPs specifically; look for genuine tenant isolation of credentials/data alongside playbook reuse, not a single shared workspace.
baselineRFIHow are third-party API credentials and secrets used by playbooks stored and rotated — is there a built-in secrets vault, or does it rely on the customer's own secrets management?weight 3
Answer key — what a strong answer shows
A built-in, audited secrets vault with rotation support is stronger than plaintext credential storage or reliance on the customer's own layer.
baselineRFIWhat is the pricing model — per playbook, per analyst seat, per event/alert volume processed — and how does cost scale as alert volume grows significantly (e.g., after onboarding a new high-volume data source)?weight 3
Answer key — what a strong answer shows
Look for transparent, predictable scaling economics; event-volume-based pricing that spikes unpredictably when a new noisy data source is onboarded creates real budget risk.
baselineRFPDoes the platform offer AI/LLM-assisted playbook generation (describing a desired workflow in natural language and having a draft playbook generated), and how much manual refinement is typically needed before that draft is production-ready?weight 3
Answer key — what a strong answer shows
Strong answers are honest about how much manual work remains after AI-assisted generation — a vendor claiming 'fully automated playbook creation' with no refinement needed should be tested directly, not taken at face value.
baselineRFIHow does the platform support cross-team workflows beyond the security team itself — e.g., automatically routing a data-privacy incident to legal, or a compliance finding to the GRC team — with appropriate access scoping for non-security stakeholders?weight 3
Answer key — what a strong answer shows
Look for genuine cross-team routing with proper access controls for non-security users; a SOAR platform that only knows how to talk to other security tools misses real incident-response workflows that cross departments.
baselineRFIWhat happens to in-flight automated incident response if the SOAR platform itself has an outage mid-incident — is there a documented fallback/manual-continuation process, and has this actually been tested?weight 3
Answer key — what a strong answer shows
A vendor without a tested fallback plan for their own platform's downtime during an active incident is a meaningful operational risk — ask if this has been tested, not just theorized.
baselineRFPDetail the availability of a staging/testing environment separate from production for validating new playbooks and integrations before they run against live security data and can take real automated actions.weight 3
Answer key — what a strong answer shows
A genuine staging environment (not just a 'dry run' toggle within production) is materially safer for testing new automation before it can take real, potentially destructive actions.
baselineRFIIs there a community or marketplace for sharing pre-built playbooks and integrations across customers, and how vetted/maintained is that content versus being unmaintained community contributions?weight 3
Answer key — what a strong answer shows
Look for an actively maintained, vendor-curated marketplace rather than an abandoned community forum — unmaintained shared playbooks can be a real security risk if they embed outdated assumptions.
baselineRFPExplain the audit trail for automated actions — is every action a playbook takes (e.g., disabling a user account, blocking an IP) logged with full context for compliance/forensic review, and can this audit trail be exported to an external SIEM or GRC system?weight 3
Answer key — what a strong answer shows
A complete, exportable audit trail of automated actions is essential for both incident forensics and compliance — a platform with only partial or non-exportable logging creates real accountability gaps.
baselineRFIWhat is the typical learning curve for a new analyst to become proficient building and maintaining playbooks — is formal training included, and what is a customer-referenced timeline from onboarding to a team member independently authoring production playbooks?weight 3
Answer key — what a strong answer shows
Strong answers give a concrete, customer-validated timeline; a 'no-code' claim doesn't guarantee genuine ease of use for complex real-world playbook logic.
baselineRFPIntegrate with key components of the existing IT ecosystem, including Heimdal EDR, Cisco Meraki, Microsoft 365, Azure AD, and on-prem Active Directory.weight 3
baselineRFPProvide robust correlation, investigation, and automation workflows to reduce response time and improve analyst efficiency.weight 3
baselineRFPDevelop and deploy automated response playbooks using Logic Apps or other Microsoft-native tools, aligned with pre-approved containment actions and escalation paths (Playbook and Automation Setup).weight 3
baselineRFPShould support automated response actions (e.g., isolating compromised endpoints, triggering firewall rules, revoking user access).weight 3
baselineRFPVendors should provide pricing for different deployment models and include cost breakdowns for licensing (per user, per endpoint, per GB, etc.), support and maintenance, and additional features (e.g., SOAR, extended retention, custom integrations).weight 3
baselineRFPAlerting and Incident Integration: configure alert notifications, integrate with ITSM/SOAR platforms (e.g., in-house ticketing, Jira, Zoho), and configure incident workflows and escalation.weight 3
baselineRFPBidder to provide workflow automation so that applications and infrastructure are integrated automatically, with minimal manual intervention.weight 3
baselineRFPThe vendor should bring workflows and solutions that can automate the majority of incident response activities such as false positive management, managing whitelists, escalation workflow, and SLA management; alerts should be notified to the financial institution only after a proper triage process and enriched with context data, environmental data, vulnerability data, historical data, and threat intelligence.weight 3
baselineRFPSolution should support centralized incident management and triaging of alerts from multiple security products (SIEM, DLP, IPS, WAF, Anti-APT, ETDR); investigation module should integrate with log sources (SIEM, ETDR, EPP, Data Lake) on demand to pull data related to the investigated alert with charting and graphing to analyse data.weight 3
baselineRFPSolution should support full workflow for incident classification, coordination, escalation, exception approvals, and tracking of security exception approvals; SP to provide a ticketing tool with SOC operations access to the financial institution.weight 3
baselineRFPProposal must clearly describe authority boundaries for three tiers of response action: (1) non-disruptive playbook actions authorized without prior agency approval, (2) actions requiring standing agency approval (e.g. account disable/reset, endpoint isolation, blocking IOCs at firewall/proxy/email), and (3) actions requiring explicit agency authorization (changes outside Sentinel, destructive actions, external communications) — including an emergency exception process for confirmed active incidents threatening public safety or critical agency services.weight 3
baselineRFPZero Trust Phase 1 planning: Contractor shall develop technical requirements/implementation plans for Defender for Servers & Cloud Apps (CASB) integration, migration from traditional VPNs to a SASE model, and Security Orchestration, Automation and Response (SOAR) with Conditional Access — engineering automated response playbooks and identity-centric access policies.weight 3
These questions and weights are the reviewed standard for this category — start scoring straight away. To change them, or to import your own questions from a spreadsheet and reuse them on every evaluation, create a free buyer account.
