Realising Ambition Standards of Evidence

Decisions about which interventions will be supported by Realising Ambition will be guided, in part, by a set of standards of evidence developed by the Social Research Unit at Dartington, and supported by an international expert group.

These standards have been adopted by the Greater London Authority as part of their Project Oracle and were used as a guide by the Allen Review on Early Intervention. They are being used to underpin the UK’s first reliable source of information on evidence-based programmes called Evidence2Success, which will be made publicly available by the Social Research Unit early in 2012.

Any intervention that has the intention to improve children’s health and development can be assessed against the Standards. The Standards have four dimensions.

1. Intervention specificity

This is an estimation of whether the intervention might logically be expected to have an impact on children’s health and development.

Interventions that meet this test typically are clear about:

• who is being served
• what impact on which aspects of children’s health and development will be achieved
• the reason - the logic behind - why the intervention will achieve the outcome.

Interventions that do well against the intervention specificity criterion generally are supported by good documentation, training and coaching.
 

2. Evaluation quality

Many interventions have been evaluated, but the quality of evaluation varies considerably. The Standards value evaluations that give a reliable indication of impact on child outcomes. Other types of evaluation, such as consumer satisfaction and implementation quality are valued but fall outside the focus of the Standards.

Interventions that meet this test typically:

• are supported by people who have a genuine interest in finding out whether the intervention is effective
• have been subjected to an evaluation that compares outcomes for children receiving the intervention with another with the same needs who do not receive the intervention
• ideally, have been independently evaluated using a well-executed randomised controlled trial.
   

3. Intervention impact

The Standards value interventions that can be clear about how much impact will typically be achieved on specific dimensions of children’s health and development.

The Standards emphasise two dimensions of impact:

• a positive effect size, a standard measure of impact that provides comparable data regardless of the outcomes assessed
• no harmful effects or negative side-effects of the intervention
   

4. System readiness

Many of the most effective interventions are not ready for the real world. Realising Ambition is interested in replication of effective interventions, which means they have to be ready for the mainstream public systems responsible for the £55 billion of annual expenditure on children, or making implementation outside of these public systems sustainable.

Meeting this test typically involves:

• having a clear indication of unit cost and staffing requirements
• explicit processes to measure the fidelity of implementation and to address common implementation problems.