One of the core objectives of SPRING is to promote an evidence-based culture in integration policies and practices. With that in mind, partners have been working to identify tools and incentives that allow communities of practice (CoP) to self-evaluate their practices for better outcomes & better data. 

A Diagnostic Framework (DF) was developed to serve this objective and is expected, in general, to support and guide the evaluation (also intended as diagnosis) of practices of migrants’ integration or, alternatively, the identification of research/studies that are based on sound and methodologically consistent evaluation design.

The Diagnostic Framework, developed by ISMU over the past year, adopts the so-called Theory-Based Evaluation (TBE) approach to impact evaluation. According to this approach, three are the key-concepts:

1. Impact, that is defined – according to a widely used definition – as any “positive and negative, primary and secondary long-term effects” produced by an intervention “directly or indirectly, intended or unintended” (OECD-DAC Lexicon, 2002).
2. Impact evaluation, that identifies the evaluation focused on ex-post effects and their linkage to a specific intervention. It is “an evaluation focused on outcomes or payoff rather than process, delivery, or implementation evaluation” (Scriven, 1991), an evaluation which addresses the so-called causal issue.
3. The logic informing impact evaluation. This logic considers impact evaluation in terms of contribution, assuming that the intervention is one of the causes of the changes/modifications that may occur after the intervention. Such a logic is characteristic of the Theory-Based Approach, which adopts a generative causation model whose specificity is “identifying the mechanisms that explain effects – the inference basis for theory-based and realist approaches to impact evaluation”.

The Diagnostic Framework has been developed in three versions, each one corresponding to a different Theory-Based Impact Evaluation sub-approach. The three sub-approaches are Contribution Analysis (CA), Theory of Change (ToC) and Realistic Evaluation (RE). For each of them, a specific operational tool has been produced in order to guide – step by step – its use for identifying and evaluating successful integration practices or checking whether a study or research has been conducted using that sub-approach in a sound and rigorous way. Therefore, six different Diagnostic Framework schemes are available. In addition, a unified version of the Diagnostic Framework has been developed for integration practices only. This version is based on the most relevant elements common to the three sub-approaches and is therefore consistent with the Theory-based perspective on evaluation.

With reference to the sub-approaches, Contribution Analysis (CA) mainly focuses on “causal process design” which means that the intervention “is regarded as a conjunction of causes that follows a sequence (…) whose process is built around a theory – a set of assumptions about how the intervention achieves its goals under what conditions” (DFID, 2012). The Theory-of-Change (ToC) sub-approach is based on “the set of assumptions that explain both the mini-steps that lead to the long-term goal and the connections between policy or programme activities and outcomes that occur each step of the way”. Articulating and eliciting the mini-steps leading to a long-term goal to be reached allows to give evidence to the factors, mechanisms and passages which must be analysed and assessed in order to link the final goal with the intermediate outcomes (changes) and the policy/programme’s implementation process or steps. Finally, Realistic (impact) Evaluation focuses on the interventions’ mechanisms and their implementation context(s). The idea that “causal outcomes follow from mechanisms acting in contexts is the axiomatic base upon which all realist explanation builds” (Pawson and Tilley, 1997).

See the dignostic framework here to understand the approach taken!