The Evaluation report on Youth Diversion Projects (YDPs) was launched on 13 June 2023 by Minister of State James Browne TD.1,2 This was the first evaluation report on YDPs1 and was carried out by Research Matters Ltd between December 2021 and November 2022. The central aim was to create policy-relevant information relating to the structure, conduct, and impact of YDPs.

Context of report

YDPs target young people aged 12 to 17 years who are part of communities with a detected need or where youths are at risk of continuing to be involved with the criminal justice system.

The projects create an opportunity to divert youths from criminality but also to allow for preventative work within the community and at-risk families.3

This work is aligned with objectives outlined in the Youth Justice Strategy 2021–2027.4 There are currently 105 YDPs in Ireland, which are funded by the Irish Government and the European Social Fund (ESF) Plus 2021–2027.5

Research questions

There were five research questions to be addressed in the evaluation.

1. How are the YDPs structured and what are the key inputs into the projects?
2. How are these projects implemented; what are the main processes, actions and activities associated with their implementation?
3. What changes for young people, their families, and the broader community because of being involved in the YDPs?
4. What works, for whom, in which circumstances; what are the mechanisms that operate; and how does the context influence outcomes?
5. What recommendations arise from the evaluation in terms of funding, governance, supports, synergies, coherence, and balance between interventions and wider engagement?

Literature review

The literature review involved a scoping review of grey and peer-review literature nationally and internationally. It brings together contextual material on YDPs and provides current insights into the type of offending carried out by youths; diversion in response to offending behaviour; protective and risk factors; limitations and effectiveness of diversion; interventions, including risk assessment, youth work, and youth justice work. The importance of relationships between young people and services has been identified as an essential feature to getting youths involved and has contributed to successful outcomes.

Finally, previous YDP studies are outlined along with policy guidelines regarding the purpose and scope of projects and the roles and responsibilities of youth justice workers (YJWs) and juvenile liaison officers (JLOs).

Methodology

Theories applied

Two theories were applied in this evaluation: Weiss’s 1997 theory of change and Pawson and Tilly’s 1997 realist evaluation.1 The theory of change developed after key stakeholder interviews is outlined in Figure 1. The realist evaluation described, appraised, and quantified inputs, processes, and outcomes of the projects.

Source: Research Matters, 2023, p. 331

Figure 1: Preliminary theory of change

Source: Research Matters, 2023, p. 341

Figure 2: Data sources and methods

Methods

To address the research questions, a multimethod design was taken to gather data. Figure 2 shows a breakdown of the main data sources along with the number of stakeholders who participated. Case studies and surveys were the main methods used to collect data. Case studies included interviews (12 groups [n=72]; individuals [n=13]), observations, and administration data (ESF database, annual project plans and reports (APR), budget files, and surveys). The surveys involved the project survey (n=64), young people (n=75), JLOs (n=53), YJWs (n=75), managers (n=48), and project committee members (n=54).

Findings

The findings were broken down into two categories: (i) structure, governance, and resource analysis, and (ii) implementation of service analysis.

Structure, governance, and resource analysis

Good coherence was shown between the stakeholders in their understanding of the purpose of YDPs being centred on a belief that crime reduction can be achieved via increased life opportunities for young people.

The support provided by national governance structures was viewed positively, while governance of projects at a local level was deemed to work well in practice.

At the local level, there was a feeling of being supported by national structures. This was evident through good partner cooperation and interaction along with a commitment and willingness to engage. However, at project level, managers had reservations with regards to whether project committees should be involved in overall governance, risk, and financial management.

Budget analysis showed differences in how budgets were allocated, with three-quarters of the budgets being allocated to pay costs. This necessitates further exploration. Additionally, there is no single pay scale in operation across projects. Managers, YJWs, and JLOs were experienced and well-educated; however, less than one-fifth of those employed in this area had achieved a qualification in youth justice. While YJWs reported high job satisfaction in relation to training opportunities, manager support, and quality of service they provided, there were issues around promotion and professional supervision. The MA degree under the Research Evidence into Policy Programmes and Practice (REPPP) programme at the University of Limerick was welcomed.

Implementation of service analysis

A detailed analysis of the interventions implemented in YDPs was presented by the report authors. Interventions that were applied targeted issues that arose in the YDP plan. There was mainly considerable diversity between the interventions. In more than 80% of plans, only one intervention in relation to education or employment was implemented. Only 16 plans implemented interventions related to criminal behaviour. Due to the focus of the projects, this was viewed as a surprising outcome. However, it does lend support to another YDP focus, namely, the building of better life chances in young people to reduce criminality.1 On a positive note, the analysis of YDP plans indicates that planning and implementation of projects are aligned and show that the nature and structure of projects target specific local needs.

Limitations

As acknowledged by the authors, all approaches used to collect data have their own limitations. For example, using online surveys is efficient and convenient. However, they do not capture the complexity of the contexts in which the YDPs operate. There is also a potential for bias by those that complete them. The response rate for surveys was low; hence, the results should be interpreted with caution. The qualitative interviews overcame the survey limitations; however, participants were only from the eight case study sites. Within each site, young people, family, and project committee members were identified by the site manager or coordinator. Only young people who engaged with the projects were interviewed; those that disengaged were not. How project effectiveness was measured was another limitation due to difficulties accessing relevant crime data. While Youth Level of Service (YLS) data provided some measures of effectiveness, this was based on professional judgements on assessments recorded in the annual performance report file.

However, the multiple methods used in the study mitigated some of these limitations to a certain degree.

Recommendations

Several recommendations (n=52) arose from the evaluation. These focused on:

• Improvements in governance (n=11)
• Resourcing of YDPs (n=5)
• Personnel (n=7)
• Implementation of YDPs (n=7)
• Outcomes and impacts (n=6)
• Further research and data development (n=16).

Conclusion

The authors were unable to determine whether projects were effective in reducing crime; however, they are performing well in several locations, which does impact positively on crime reduction. While recommendations for change have been made, project governance is considered good, cooperation is high, and experience, qualifications, and satisfaction of staff are high. There is evidence of best practice in implementation, while feedback from those participating is positive.

The evaluation report was welcomed by Minister Browne. He believes the report is an opportunity to appreciate the impact that youth diversion has had and that the recommendations and insights provided will inform future progress and direction of YDPs. He thanked all those who participated in the evaluation and stated that the findings of the evaluation illustrate the ‘commitment and willingness among all participants to work towards a shared purpose, centred on the needs of vulnerable children and young people’ (p. 2).2

1    Research Matters Ltd (2023) Evaluation of youth diversion projects. Dublin: Department of Justice. Available from: https://www.drugsandalcohol.ie/38981/

2    Department of Justice (2023) Minister of State with responsibility for Youth Justice, James Browne, launches evaluation report on youth diversion projects [Press release]. 13 June 2023. Dublin: Department of Justice. Available from: https://www.gov.ie/en/press-release/dff76-minister-of-state-with-responsibility-for-youth-justice-james-browne-launches-evaluation-report-on-youth-diversion-projects/#

3    Department of Justice (2022) Youth justice projects operational requirements 2022. Dublin: Department of Justice. Available from: https://assets.gov.ie/240584/95edb188-4889-421b-8555-9d62eaf4780c.pdf

4    Department of Justice (2021) Youth Justice Strategy 2021–2027. Dublin: Department of Justice. Available from: https://www.drugsandalcohol.ie/34061/

5    McEntee H (2022) Parliamentary Debates Dáil Éireann – Youth Services. Question 6. 5 July 2022. Vol. 1024, No. 7. Available from: https://www.oireachtas.ie/en/debates/debate/dail/2022-07-05/22/