SA-11 Developer Testing and Evaluation

Guidance: draftPolicy clause: draftWhat these mean

Baselines	Implementation level	Enhancements
Moderate, High, Privacy	Organization	9 (0 in a baseline)

Related controls: CA-2, CA-7, CM-4, SA-3, SA-4, SA-5, SA-8, SA-15, SA-17, SI-2, SR-5, SR-6, SR-7

Control statement

Require the developer of the system, system component, or system service, at all post-design stages of the system development life cycle, to:

• a. Develop and implement a plan for ongoing security and privacy control assessments;
• b. Perform [Selection (one or more): unit; integration; system; regression] testing/evaluation [Assignment: organization-defined frequency to conduct] at [Assignment: organization-defined depth and coverage];
• c. Produce evidence of the execution of the assessment plan and the results of the testing and evaluation;
• d. Implement a verifiable flaw remediation process; and
• e. Correct flaws identified during testing and evaluation.

NIST discussion

Developmental testing and evaluation confirms that the required controls are implemented correctly, operating as intended, enforcing the desired security and privacy policies, and meeting established security and privacy requirements. Security properties of systems and the privacy of individuals may be affected by the interconnection of system components or changes to those components. The interconnections or changes—including upgrading or replacing applications, operating systems, and firmware—may adversely affect previously implemented controls. Ongoing assessment during development allows for additional types of testing and evaluation that developers can conduct to reduce or eliminate potential flaws. Testing custom software applications may require approaches such as manual code review, security architecture review, and penetration testing, as well as and static analysis, dynamic analysis, binary analysis, or a hybrid of the three analysis approaches.

Developers can use the analysis approaches, along with security instrumentation and fuzzing, in a variety of tools and in source code reviews. The security and privacy assessment plans include the specific activities that developers plan to carry out, including the types of analyses, testing, evaluation, and reviews of software and firmware components; the degree of rigor to be applied; the frequency of the ongoing testing and evaluation; and the types of artifacts produced during those processes. The depth of testing and evaluation refers to the rigor and level of detail associated with the assessment process. The coverage of testing and evaluation refers to the scope (i.e., number and type) of the artifacts included in the assessment process. Contracts specify the acceptance criteria for security and privacy assessment plans, flaw remediation processes, and the evidence that the plans and processes have been diligently applied. Methods for reviewing and protecting assessment plans, evidence, and documentation are commensurate with the security category or classification level of the system. Contracts may specify protection requirements for documentation.

Control enhancements

SA-11(1) Static Code Analysis

Baselines: Not in a baseline

Require the developer of the system, system component, or system service to employ static code analysis tools to identify common flaws and document the results of the analysis.

Discussion and assessment objectives for SA-11(1)

Static code analysis provides a technology and methodology for security reviews and includes checking for weaknesses in the code as well as for the incorporation of libraries or other included code with known vulnerabilities or that are out-of-date and not supported. Static code analysis can be used to identify vulnerabilities and enforce secure coding practices. It is most effective when used early in the development process, when each code change can automatically be scanned for potential weaknesses. Static code analysis can provide clear remediation guidance and identify defects for developers to fix. Evidence of the correct implementation of static analysis can include aggregate defect density for critical defect types, evidence that defects were inspected by developers or security professionals, and evidence that defects were remediated. A high density of ignored findings, commonly referred to as false positives, indicates a potential problem with the analysis process or the analysis tool. In such cases, organizations weigh the validity of the evidence against evidence from other sources.

Determine if:

• SA-11(01)[01] the developer of the system, system component, or system service is required to employ static code analysis tools to identify common flaws;
• SA-11(01)[02] the developer of the system, system component, or system service is required to employ static code analysis tools to document the results of the analysis.

Examine: System and services acquisition policy; system and services acquisition procedures; procedures addressing system developer security testing; procedures addressing flaw remediation; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; security and privacy architecture; system design documentation; system developer security and privacy assessment plans; results of system developer security and privacy assessments; security flaw and remediation tracking records; system security plan; privacy plan; privacy impact assessment; privacy risk assessment documentation; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security responsibilities; organizational personnel with developer security and privacy testing responsibilities; organizational personnel with configuration management responsibilities; system developers.

Test: Organizational processes for monitoring developer security testing and evaluation; mechanisms supporting and/or implementing the monitoring of developer security testing and evaluation; static code analysis tools.

SA-11(2) Threat Modeling and Vulnerability Analyses

Baselines: Not in a baseline

Require the developer of the system, system component, or system service to perform threat modeling and vulnerability analyses during development and the subsequent testing and evaluation of the system, component, or service that:

• (a) Uses the following contextual information: [Assignment: organization-defined information];
• (b) Employs the following tools and methods: [Assignment: organization-defined tools and methods];
• (c) Conducts the modeling and analyses at the following level of rigor: [Assignment: organization-defined breadth and depth of modeling and analyses] ; and
• (d) Produces evidence that meets the following acceptance criteria: [Assignment: organization-defined acceptance criteria].

Discussion and assessment objectives for SA-11(2)

Systems, system components, and system services may deviate significantly from the functional and design specifications created during the requirements and design stages of the system development life cycle. Therefore, updates to threat modeling and vulnerability analyses of those systems, system components, and system services during development and prior to delivery are critical to the effective operation of those systems, components, and services. Threat modeling and vulnerability analyses at this stage of the system development life cycle ensure that design and implementation changes have been accounted for and that vulnerabilities created because of those changes have been reviewed and mitigated.

Determine if:

• SA-11(02)(a)
  • SA-11(02)(a)[01] the developer of the system, system component, or system service is required to perform threat modeling during development of the system, component, or service that uses [Assignment: organization-defined information];
  • SA-11(02)(a)[02] the developer of the system, system component, or system service is required to perform vulnerability analyses during development of the system, component, or service that uses [Assignment: organization-defined information];
  • SA-11(02)(a)[03] the developer of the system, system component, or system service is required to perform threat modeling during the subsequent testing and evaluation of the system, component, or service that uses [Assignment: organization-defined information];
  • SA-11(02)(a)[04] the developer of the system, system component, or system service is required to perform vulnerability analyses during the subsequent testing and evaluation of the system, component, or service that uses [Assignment: organization-defined information];
• SA-11(02)(b)
  • SA-11(02)(b)[01] the developer of the system, system component, or system service is required to perform threat modeling during development of the system, component, or service that employs [Assignment: organization-defined tools and methods];
  • SA-11(02)(b)[02] the developer of the system, system component, or system service is required to perform threat modeling during the subsequent testing and evaluation of the system, component, or service that employs [Assignment: organization-defined tools and methods];
  • SA-11(02)(b)[03] the developer of the system, system component, or system service is required to perform vulnerability analyses during development of the system, component, or service that employs [Assignment: organization-defined tools and methods];
  • SA-11(02)(b)[04] the developer of the system, system component, or system service is required to perform vulnerability analyses during the subsequent testing and evaluation of the system, component, or service that employs [Assignment: organization-defined tools and methods];
• SA-11(02)(c)
  • SA-11(02)(c)[01] the developer of the system, system component, or system service is required to perform threat modeling at [Assignment: organization-defined breadth and depth] during development of the system, component, or service;
  • SA-11(02)(c)[02] the developer of the system, system component, or system service is required to perform vulnerability analyses during the subsequent testing and evaluation of the system, component, or service that conducts modeling and analyses at [Assignment: organization-defined breadth and depth];
• SA-11(02)(d)
  • SA-11(02)(d)[01] the developer of the system, system component, or system service is required to perform threat modeling during development of the system, component, or service that produces evidence that meets [Assignment: organization-defined acceptance criteria];
  • SA-11(02)(d)[02] the developer of the system, system component, or system service is required to perform threat modeling during the subsequent testing and evaluation of the system, component, or service that produces evidence that meets [Assignment: organization-defined acceptance criteria];
  • SA-11(02)(d)[03] the developer of the system, system component, or system service is required to perform vulnerability analyses during development of the system, component, or service that produces evidence that meets [Assignment: organization-defined acceptance criteria];
  • SA-11(02)(d)[04] the developer of the system, system component, or system service is required to perform vulnerability analyses during the subsequent testing and evaluation of the system, component, or service that produces evidence that meets [Assignment: organization-defined acceptance criteria].

Examine: System and services acquisition policy; procedures addressing system developer security testing; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; system developer security test plans; records of developer security testing results for the system, system component, or system service; vulnerability scanning results; system risk assessment reports; threat and vulnerability analysis reports; system security plan; supply chain risk management plan; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security responsibilities; organizational personnel with developer security testing responsibilities; system developers; organizational personnel with supply chain risk management responsibilities.

Test: Organizational processes for monitoring developer security testing and evaluation; mechanisms supporting and/or implementing the monitoring of developer security testing and evaluation.

SA-11(3) Independent Verification of Assessment Plans and Evidence

Baselines: Not in a baseline

• (a) Require an independent agent satisfying [Assignment: organization-defined independence criteria] to verify the correct implementation of the developer security and privacy assessment plans and the evidence produced during testing and evaluation; and
• (b) Verify that the independent agent is provided with sufficient information to complete the verification process or granted the authority to obtain such information.

Discussion and assessment objectives for SA-11(3)

Independent agents have the qualifications—including the expertise, skills, training, certifications, and experience—to verify the correct implementation of developer security and privacy assessment plans.

Determine if:

• SA-11(03)(a)
  • SA-11(03)(a)[01] an independent agent is required to satisfy [Assignment: organization-defined independence criteria] to verify the correct implementation of the developer security assessment plan and the evidence produced during testing and evaluation;
  • SA-11(03)(a)[02] an independent agent is required to satisfy [Assignment: organization-defined independence criteria] to verify the correct implementation of the developer privacy assessment plan and the evidence produced during testing and evaluation;
• SA-11(03)(b) the independent agent is provided with sufficient information to complete the verification process or granted the authority to obtain such information.

Examine: System and services acquisition policy; system and services acquisition procedures; procedures addressing system developer security testing; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; independent verification and validation reports; security and privacy assessment plans; results of security and privacy assessments for the system, system component, or system service; system security plan; privacy plan; privacy program plan; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security and privacy responsibilities; organizational personnel with developer security testing responsibilities; system developers; independent verification agent.

Test: Organizational processes for monitoring developer security testing and evaluation; mechanisms supporting and/or implementing the monitoring of developer security testing and evaluation.

SA-11(4) Manual Code Reviews

Baselines: Not in a baseline

Require the developer of the system, system component, or system service to perform a manual code review of [Assignment: organization-defined specific code] using the following processes, procedures, and/or techniques: [Assignment: organization-defined processes, procedures, and/or techniques].

Discussion and assessment objectives for SA-11(4)

Manual code reviews are usually reserved for the critical software and firmware components of systems. Manual code reviews are effective at identifying weaknesses that require knowledge of the application’s requirements or context that, in most cases, is unavailable to automated analytic tools and techniques, such as static and dynamic analysis. The benefits of manual code review include the ability to verify access control matrices against application controls and review detailed aspects of cryptographic implementations and controls.

Determine if the developer of the system, system component, or system service is required to perform a manual code review of [Assignment: organization-defined specific code] using [Assignment: organization-defined processes, procedures, and/or techniques].

Examine: System and services acquisition policy; procedures addressing system developer security testing; processes, procedures, and/or techniques for performing manual code reviews; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; system developer security testing and evaluation plans; system developer security testing and evaluation results; list of code requiring manual reviews; records of manual code reviews; system security plan; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security responsibilities; organizational personnel with developer security testing responsibilities; system developers; independent verification agent.

Test: Organizational processes for monitoring developer security testing and evaluation; mechanisms supporting and/or implementing the monitoring of developer testing and evaluation.

SA-11(5) Penetration Testing

Baselines: Not in a baseline

Require the developer of the system, system component, or system service to perform penetration testing:

• (a) At the following level of rigor: [Assignment: organization-defined breadth and depth of testing] ; and
• (b) Under the following constraints: [Assignment: organization-defined constraints].

Discussion and assessment objectives for SA-11(5)

Penetration testing is an assessment methodology in which assessors, using all available information technology product or system documentation and working under specific constraints, attempt to circumvent the implemented security and privacy features of information technology products and systems. Useful information for assessors who conduct penetration testing includes product and system design specifications, source code, and administrator and operator manuals. Penetration testing can include white-box, gray-box, or black-box testing with analyses performed by skilled professionals who simulate adversary actions. The objective of penetration testing is to discover vulnerabilities in systems, system components, and services that result from implementation errors, configuration faults, or other operational weaknesses or deficiencies. Penetration tests can be performed in conjunction with automated and manual code reviews to provide a greater level of analysis than would ordinarily be possible. When user session information and other personally identifiable information is captured or recorded during penetration testing, such information is handled appropriately to protect privacy.

Determine if:

• SA-11(05)(a)
  • SA-11(05)(a)[01] the developer of the system, system component, or system service is required to perform penetration testing at the following level of rigor: [Assignment: organization-defined breadth];
  • SA-11(05)(a)[02] the developer of the system, system component, or system service is required to perform penetration testing at the following level of rigor: [Assignment: organization-defined depth];
• SA-11(05)(b) the developer of the system, system component, or system service is required to perform penetration testing under [Assignment: organization-defined constraints].

Examine: System and services acquisition policy; system and services acquisition procedures; procedures addressing system developer security testing; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; system developer penetration testing and evaluation plans; system developer penetration testing and evaluation results; system security plan; privacy plan; personally identifiable information processing policy; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security and privacy responsibilities; organizational personnel with developer security testing responsibilities; system developers; independent verification agent.

Test: Organizational processes for monitoring developer security and privacy assessments; mechanisms supporting and/or implementing the monitoring of developer security and privacy assessments.

SA-11(6) Attack Surface Reviews

Baselines: Not in a baseline

Require the developer of the system, system component, or system service to perform attack surface reviews.

Discussion and assessment objectives for SA-11(6)

Attack surfaces of systems and system components are exposed areas that make those systems more vulnerable to attacks. Attack surfaces include any accessible areas where weaknesses or deficiencies in the hardware, software, and firmware components provide opportunities for adversaries to exploit vulnerabilities. Attack surface reviews ensure that developers analyze the design and implementation changes to systems and mitigate attack vectors generated as a result of the changes. The correction of identified flaws includes deprecation of unsafe functions.

Determine if the developer of the system, system component, or system service is required to perform attack surface reviews.

Examine: System and services acquisition policy; procedures addressing system developer security testing; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; system developer security testing and evaluation plans; system developer security testing and evaluation results; records of attack surface reviews; system security plan; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security responsibilities; organizational personnel with developer security testing responsibilities; organizational personnel with configuration management responsibilities; system developers.

Test: Organizational processes for monitoring developer security testing and evaluation; mechanisms supporting and/or implementing the monitoring of developer security testing and evaluation.

SA-11(7) Verify Scope of Testing and Evaluation

Baselines: Not in a baseline

Require the developer of the system, system component, or system service to verify that the scope of testing and evaluation provides complete coverage of the required controls at the following level of rigor: [Assignment: organization-defined breadth and depth of testing and evaluation].

Discussion and assessment objectives for SA-11(7)

Verifying that testing and evaluation provides complete coverage of required controls can be accomplished by a variety of analytic techniques ranging from informal to formal. Each of these techniques provides an increasing level of assurance that corresponds to the degree of formality of the analysis. Rigorously demonstrating control coverage at the highest levels of assurance can be achieved using formal modeling and analysis techniques, including correlation between control implementation and corresponding test cases.

Determine if:

• SA-11(07)[01] the developer of the system, system component, or system service is required to verify that the scope of testing and evaluation provides complete coverage of the required controls at [Assignment: organization-defined breadth];
• SA-11(07)[02] the developer of the system, system component, or system service is required to verify that the scope of testing and evaluation provides complete coverage of the required controls at [Assignment: organization-defined depth].

Examine: System and services acquisition policy; procedures addressing system developer security testing; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; system developer security testing and evaluation plans; system developer security testing and evaluation results; system security plan; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security responsibilities; organizational personnel with developer security testing responsibilities; system developers; independent verification agent.

Test: Organizational processes for monitoring developer security testing and evaluation; mechanisms supporting and/or implementing the monitoring of developer security testing and evaluation.

SA-11(8) Dynamic Code Analysis

Baselines: Not in a baseline

Require the developer of the system, system component, or system service to employ dynamic code analysis tools to identify common flaws and document the results of the analysis.

Discussion and assessment objectives for SA-11(8)

Dynamic code analysis provides runtime verification of software programs using tools capable of monitoring programs for memory corruption, user privilege issues, and other potential security problems. Dynamic code analysis employs runtime tools to ensure that security functionality performs in the way it was designed. A type of dynamic analysis, known as fuzz testing, induces program failures by deliberately introducing malformed or random data into software programs. Fuzz testing strategies are derived from the intended use of applications and the functional and design specifications for the applications. To understand the scope of dynamic code analysis and the assurance provided, organizations may also consider conducting code coverage analysis (i.e., checking the degree to which the code has been tested using metrics such as percent of subroutines tested or percent of program statements called during execution of the test suite) and/or concordance analysis (i.e., checking for words that are out of place in software code, such as non-English language words or derogatory terms).

Determine if:

• SA-11(08)[01] the developer of the system, system component, or system service is required to employ dynamic code analysis tools to identify common flaws;
• SA-11(08)[02] the developer of the system, system component, or system service is required to document the results of the analysis.

Examine: System and services acquisition policy; procedures addressing system developer security testing; procedures addressing flaw remediation; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; system developer security test and evaluation plans; security test and evaluation results; security flaw and remediation tracking reports; system security plan; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security responsibilities; organizational personnel with developer security testing responsibilities; organizational personnel with configuration management responsibilities; system developers.

Test: Organizational processes for monitoring developer security testing and evaluation; mechanisms supporting and/or implementing the monitoring of developer security testing and evaluation.

SA-11(9) Interactive Application Security Testing

Baselines: Not in a baseline

Require the developer of the system, system component, or system service to employ interactive application security testing tools to identify flaws and document the results.

Discussion and assessment objectives for SA-11(9)

Interactive (also known as instrumentation-based) application security testing is a method of detecting vulnerabilities by observing applications as they run during testing. The use of instrumentation relies on direct measurements of the actual running applications and uses access to the code, user interaction, libraries, frameworks, backend connections, and configurations to directly measure control effectiveness. When combined with analysis techniques, interactive application security testing can identify a broad range of potential vulnerabilities and confirm control effectiveness. Instrumentation-based testing works in real time and can be used continuously throughout the system development life cycle.

Determine if:

• SA-11(09)[01] the developer of the system, system component, or system service is required to employ interactive application security testing tools to identify flaws;
• SA-11(09)[02] the developer of the system, system component, or system service is required to document the results of flaw identification.

Examine: System and services acquisition policy; procedures addressing system developer security testing; procedures addressing interactive application security testing; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; system developer security test and evaluation plans; security test and evaluation results; security flaw and remediation tracking reports; system security plan; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security responsibilities; organizational personnel with developer security testing responsibilities; organizational personnel with configuration management responsibilities; system developers.

Test: Organizational processes for interactive application security testing; mechanisms supporting and/or implementing interactive application security testing.

Assessment objectives (SP 800-53A)

Objectives and methods for SA-11

Determine if:

• SA-11a.
  • SA-11a.[01] the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to develop a plan for ongoing security assessments;
  • SA-11a.[02] the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to implement a plan for ongoing security assessments;
  • SA-11a.[03] the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to develop a plan for privacy assessments;
  • SA-11a.[04] the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to implement a plan for ongoing privacy assessments;
• SA-11b. the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to perform [Selection (one or more): unit; integration; system; regression] testing/evaluation [Assignment: organization-defined frequency to conduct] at [Assignment: organization-defined depth and coverage];
• SA-11c.
  • SA-11c.[01] the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to produce evidence of the execution of the assessment plan;
  • SA-11c.[02] the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to produce the results of the testing and evaluation;
• SA-11d. the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to implement a verifiable flaw remediation process;
• SA-11e. the developer of the system, system component, or system service is required at all post-design stages of the system development life cycle to correct flaws identified during testing and evaluation.

Examine: System and services acquisition policy; system and services acquisition procedures; procedures addressing system developer security and privacy testing; procedures addressing flaw remediation; solicitation documentation; acquisition documentation; service level agreements; acquisition contracts for the system, system component, or system service; security and privacy architecture; system design documentation; system developer security and privacy assessment plans; results of developer security and privacy assessments for the system, system component, or system service; security and privacy flaw and remediation tracking records; system security plan; privacy plan; privacy impact assessment; privacy risk assessment documentation; other relevant documents or records.

Interview: Organizational personnel with system and service acquisition responsibilities; organizational personnel with information security and privacy responsibilities; organizational personnel with developer security and privacy testing responsibilities; system developers.

Test: Organizational processes for monitoring developer security testing and evaluation; mechanisms supporting and/or implementing the monitoring of developer security and privacy testing and evaluation.

Policy statement

• The System owner shall require the developer of the system, system component or system service, at all post-design stages of the system development life cycle, to develop and implement a plan for ongoing security and privacy control assessments. (SA-11a)
• The System owner shall require the developer to perform Select one or more: unit; integration; system; regression Typical: unit, integration, system and regression testing and evaluation Fill in: frequency at which to conduct the selected unit, integration, system or regression testing/evaluation Typical: with each build for automated tests, and before each release to production for system and regression testing at Fill in: depth and coverage of the selected unit, integration, system or regression testing/evaluation Typical: static analysis of all custom code, analysis of third-party and open-source components for known vulnerabilities, dynamic testing of every externally reachable interface, and a test for each security and privacy requirement. (SA-11b)
• The System owner shall require the developer to produce evidence of the execution of the assessment plan and of the results of the testing and evaluation. (SA-11c)
• The System owner shall require the developer to implement a verifiable flaw remediation process. (SA-11d)
• The System owner shall require the developer to correct flaws identified during testing and evaluation, and shall not accept a release with an unresolved flaw unless the risk is accepted (RA-7). (SA-11e)
• For a system that processes personally identifiable information, the System owner shall ensure the developer’s tests use test data prepared under SI-12(2), not live personally identifiable information, unless the Senior privacy official approves otherwise. (SA-11b)

From the System and Services Acquisition Policy template (draft). Highlighted fields are values your organization sets.

How to apply it

SA-11 asks you to require the developer to plan and run security and privacy testing after design, keep evidence of it, and fix what it finds through a verifiable flaw remediation process. NIST’s SA-11 discussion names the approaches: manual code review, security architecture review, penetration testing, and static, dynamic and binary analysis, with instrumentation and fuzzing. It defines depth as the rigor and detail of testing, and coverage as the number and type of artifacts tested. Contracts set acceptance criteria for the plans, the flaw remediation process and the evidence.

NIST’s Secure Software Development Framework, SP 800-218 (February 2022, version 1.1; as of September 2026), cites SA-11 for practices PW.7, reviewing and analyzing human-readable code, and PW.8, testing executable code.

Common implementations. Tests in the build pipeline: static analysis of custom code, analysis of third-party and open-source components for known vulnerabilities, and unit and integration tests, including a test for each security and privacy requirement. Dynamic testing of every externally reachable interface before each release. Findings go into a tracker, are fixed and then retested, and a release gate blocks releases with unresolved high findings unless the risk is accepted (RA-7). Test data prepared under SI-12(2) rather than copies of production personally identifiable information. For a commercial product, the contract asks for a summary of the supplier’s testing and its flaw remediation process.

Organization-defined parameters. Typical values, which your organization may set differently:

Parameter	Typical value
Type of testing (b)	Unit, integration, system and regression
Frequency (b)	With each build for automated tests, and before each release to production for system and regression testing
Depth and coverage (b)	Static analysis of all custom code, analysis of third-party and open-source components for known vulnerabilities, dynamic testing of every externally reachable interface, and a test for each security and privacy requirement

In the System and Services Acquisition policy, the system owner requires the plan, the testing and the evidence, and does not accept a release with an unresolved flaw unless the risk is accepted.

Evidence assessors ask for.

• The developer’s security and privacy assessment plan
• Test results for a sample of recent releases, including scan reports and the tests of security requirements
• Flaw remediation records showing findings fixed and retested, or accepted as risk
• Pipeline or release gate settings that enforce the tests
• For a system that processes personally identifiable information, how test data is prepared

Inheritance. A shared pipeline and scanning service may be a common control for in-house teams. For a commercial product, the supplier performs SA-11 and the organization relies on the contract and the supplier’s evidence. The system owns requiring it, reviewing the results and accepting each release.

Common findings.

• Scanners running with nobody reviewing the results.
• Tests that skip the APIs or the administrative interfaces.
• Copies of production data, with personally identifiable information, in test environments.
• Tool output but no assessment plan saying what is tested, how deeply and how often.
• High findings released to production without a recorded risk acceptance.

Enhancements in the Moderate baseline. None. SA-11(1) to SA-11(9), including static code analysis, threat modeling, penetration testing and dynamic code analysis, are in no baseline. SA-11 itself is also in the Privacy baseline, so it applies to any system that processes personally identifiable information.

Artifacts

• Acquisition Security Requirements
• System and Services Acquisition Decision Worksheet
• System and Services Acquisition Policy