These four scenarios outline diverse approaches to SAML SP testing, each tailored to its respective context and purpose and requiring a different type of deployment.
Self-testing by SP for production readiness
Summary description
Fully internal.
Deploy a test ISP and configure the tested SP for it.
Relational characteristics
Policy/
Deployment or configuration
!!
Arrangement and execution of tests
!!
Presentation and analysis of test results
!!
Relational or contractual arrangements
!!
Testing of SP deployment by FedOps during onboarding
Summary description
Options
- Initiated upon SP's request
- Potentially automated (the SP has to register anyway)
It probably needs to be integrated into the federation's policy and operational guidelines. However, it can be easily communicated among other requirements after the SP requests onboarding.
Deployment or configuration
!!
Arrangement and execution of tests
!!
Presentation and analysis of test results
!!
Relational or contractual arrangements
!!
Periodic testing of SP deployments by FedOps
Summary description
Options
- Triggered by SPs themselves, with each SP required to invoke it in regular intervals within policy-defined periods.
- Or it could be automatically invoked by FedOps in line with predefined rules.
Must be aligned with the federation's policy and operational A part of the federation's policy and operational rules.
Deployment or configuration
!!
Arrangement and execution of tests
!!
Presentation and analysis of test results
!!
Relational or contractual arrangements
!!
Client institution testing for compliance
Summary description
Conducted by a client institution for contracted services, possibly as part of its internal compliance reviews (e.g., GDPR audits, ISO 27001 security controls).
How is the practical arrangement of the test coordinated between the client institution and the SP?
Practical differences and requirements in comparison to self-testing:
- This form of testing may involve specific compliance criteria dictated by the client institution.
- It often integrates with broader compliance assessments, introducing additional requirements.
- The use of the test by the client institution may necessitate distinct procedures and reporting.
- Could it be done without direct approval or involvement of the SP?
It probably needs to be included in the SLA.
Deployment or configuration
!!
Arrangement and execution of tests
!!
Presentation and analysis of test results
!!
Relational or contractual arrangements
!!
Things/tests to look at
https://release-check.edugain.org/
https://access-check.edugain.org/step1
https://medium.com/the-new-control-plane/i-need-a-saml-idp-to-test-now-477761595b60
https://samltest.id/start-sp-test/
https://jumpcloud.com/blog/how-to-test-saml-and-configure-sso-for-free