...
These four scenarios outline diverse approaches to SAML SP testing, each tailored to its respective context and purpose and requiring a different type of deployment.
SELF - Self-testing by SP for production readiness
...
Summary description
This scenario enables individual Service Providers (SPs) to internally validate their SAML service configuration internally, focusing on signature usage. This scenario is the simplest one in terms of technical requirements and legal considerations. However, its chances for a meaningful level of adoption are quite low.
Deployment or configuration
...
The SP deploys a test IdP, preferably as an easily configurable VM image, container image, or appliance. Alternative (preferred by Niels): The the tool is deployed at the federation.Configuration of the tested SP for it includes.., in which case a web interface is required.
Arrangement and execution of tests
...
No formal arrangements are required as both the tester and SP belong to the same organisation.
...
This scenario is applicable during SP onboarding and may involve manual or automated testing. It is initiated upon the SP's request and integrated into the onboarding procedure of the federation. Its benefits include a wider outreach without significant legal issues, easy enforcement and a single deployment of testing software per identity federation. It requires availability of a web user interface.
Deployment or configuration
...
Periodic testing is conducted by federation operators in predefined intervals aligned with the federation's policy and operational rules, ensuring ongoing compliance. This is an extension of the testing of SPs during onboarding. Ir requires additional SP selection and scheduling functionalities.
Deployment or configuration
Similar to ONBOARDING.
Please state the key differences!!
the deployment at the FedOp testing of SPs during onboarding.
Arrangement and execution of tests
Testing execution must be aligned with the federation's policy and operating rules such as...
Tests across SPs may be spread in time and conducted during predefined high-load or low-load periods.
Presentation and analysis of test results
!!It requires both overviews for several or all SPs and, search/filtering a detailed view for a single one.
Relational or contractual arrangements
The testing process should be allowed or mandated by the federation's policy and operating guidelines should allow or mandate the testing process.
COMPLIANCE - Client institution testing for compliance
...
To best simulate the regular service usage, the testing platform can be deployed by the client organisation. However, it may also be provided by a third party specialised in compliance audits.
More!!In the latter case, which is more comfortable for clients, additional legal issues may arise.
Arrangement and execution of tests
...
The use of the test by the client institution may necessitate specialised procedures and reporting. Please suggest The producer report may retire some SLA-related metrics that would need to be supported in the generated reports!!styled longitudinal metrics.
In a more advanced usage, report signing or 'certificate' issuance may Does report signing need to be supported!!?.
Relational or contractual arrangements
...