Home > Evaluating national drugs strategies.

Pike, Brigid (2010) Evaluating national drugs strategies. Drugnet Ireland , Issue 34, Summer 2010 , pp. 12-13.

[img]
Preview
PDF (Drugnet Ireland 34) - Published Version
1MB

While evaluation is an accepted stage in the implementation of drug-related programmes and services, there has been slower progress in making it a routine part of the national policy or strategy cycle. In this regard, the EU Drugs Action Plan 2009–2012 includes two actions:  

68: to develop analytical instruments to better assess the effectiveness and impact of drug policy (e.g. model evaluation tools, policy effectiveness indices, public expenditure analysis, etc.), and
70: member states to evaluate and fine-tune national drug policies on a regular or ongoing basis.
 
In its work programme for 2010, the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA) includes four activities ‘to support Member States in evaluating their national strategies and action plans’:
 
o    develop European guidelines for the evaluation of national drug strategies,
o    hold Reitox Academy1 on the evaluation of national drug strategies,
o    launch series of national policy case studies (Portugal to be first), and
o    provide tailored support to member states if requested.
 
To date, Ireland’s national drugs strategy has only ever been subjected to ‘review’, as distinct from evaluation. While noting that there had been evaluations of individual projects, the Steering Group that drafted Ireland’s National Drugs Strategy 2009–2016 (NDS) observed that ‘there is a need to evaluate services in a more comprehensive way that assesses not only services being developed and provided under the NDS framework, but also closely related services provided in the broader social inclusion context’ (NDS: para. 6.40). Although this observation does not appear to have been followed up with an associated action regarding overall policy evaluation, the Steering Group did prioritise ‘the development of an overall performance management framework across all relevant Departments and agencies to facilitate the effective assessment and monitoring of progress’ (NDS: para. 6.101). Such a framework will help in undertaking an evaluation of the NDS at both its mid-term (2013) and its expiry (2016).
 
Policy evaluation was a theme of the recent annual conference of the International Society for the Study of Drug Policy (ISSDP). Presenters from Luxembourg, the Netherlands and Australia reported on recent evaluations of their national drugs strategies.2 They made several salient points regarding strategy evaluation.
 
In all three countries, external evaluators were engaged, including in one instance an evaluator from another country, and sometimes in conjunction with an internal evaluator.
 
 ‘Data triangulation’ was acknowledged as essential to validate the conclusions drawn by the evaluators. In Australia relevant documentation and datasets were consulted and the relevant literature reviewed, stakeholders were consulted, and case studies of aspects of policy undertaken. The information gathered from all these sources was collated and compared and assessed. With a ‘relatively small budget’, the evaluators in Luxembourg were restricted to administering a questionnaire to stakeholders; holding follow-up interviews with survey respondents to develop a SWOT analysis; and convening focus groups to recheck information, resolve contradictory answers and discuss recommendations/priorities for the new drug plan.
 
Finally, the choice of analytical framework to support the policy evaluation needs careful thought. The design will vary depending on whether it is an action plan or a strategy that is being evaluated. The Luxembourg evaluators recommended that a ‘programme logic model’ should be used ‘to monitor and evaluate the implementation and achievements of the [Drug] Action Plan in an effective and transparent way’. While the logic may be applied in a simple linear manner – inputs à activities à outputs à outcomes – they proposed a matrix form, in which the intervention logic, underlying assumptions, and objectively verifiable indicators and the means of verification for each stage in the model are also evaluated.
 
In Australia the National Drug Strategy (NDS) was evaluated using a framework with four key components:
 
1.     evaluate the NDS as a policy framework that informs stakeholders in the development of their respective drug-related policies and programmes;
2.     evaluate the outcomes of programmes under the NDS;
3.     evaluate the roles and workings of the advisory structures that inform the development and implementation of the NDS; and
4.     monitor the performance of the NDS with regard to actual and potential drug issues and drug trends in Australia during the period 2006—2009.
 
These components were used to assess the overall effectiveness and the efficiency of the NDS, and to identify future needs and opportunities for future process or other improvements.
 
At the ISSDP conference the consultants who undertook the evaluation of the Australian NDS made a case for using an additional analytical framework based on policy theories.3 They suggested that such a framework would provide evaluators with insights into the determinants of policy – how it is made, the drivers of implementation and its performance – which would strengthen the evaluation findings. In a similar vein, a recent overview of the development of Ireland’s drug strategy between 2000 and 2007 focused on Ireland’s NDS as a strategy process – how it was designed, developed and managed.4 The objective was to gain insights into how the infrastructural elements might have influenced the outcomes of the strategy. Arguably, such an analytical lens would further strengthen a strategy evaluation.
 
1.   Reitox is the European Information Network on Drugs and Drug Addiction, which is co-ordinated by the EMCDDA. The network is made up of National Focal Points (NFPs) in the 27 EU member states, Norway, the European Commission and the candidate countries. The EMCDDA holds 1–2-day seminars (academies) for network members to provide training on various topics relevant to the work of Reitox. Ireland’s NFP is located in the Health Research Board.
2.   See Trautmann F and  Braam R (2009) Evaluation of the National Drug Action Plan (2005–2009) of Luxembourg . Utrecht: Trimbos-Institut; van Laar M and van Ooyen-Houben M (2010) Dutch drug policy evaluated. Paper presented at 4th annual conference of the ISSDP, Santa Monica, California, USA, 15–16 March 2010; McDonald D, Cleary G, Miller M, Hsueh-Chih Lai S, Siggins I and Bush R (2010) Using theories of policy processes in evaluating national drug strategies: the case of the 2009 evaluation of Australia’s National Drug Strategy. Paper presented at 4th annual conference of the ISSDP, Santa Monica, California, USA, 15–16 March 2010.
3.   The policy theories included the systems model, the stages heuristic, the rational/comprehensive model, the bounded rationality model, institutional rational choice frameworks, the incrementalism model, the punctuated equilibrium theory, the multiple streams model and the advocacy coalitions model.
4.    Pike B (2009) Development of Ireland’s drug strategy 2000–2007. Overview 8. Dublin: Health Research Board.
Item Type:Article
Issue Title:Issue 34, Summer 2010
Date:2010
Page Range:pp. 12-13
Publisher:Health Research Board
Volume:Issue 34, Summer 2010
EndNote:View
Accession Number:HRB (Available)
Subjects:MP-MR Policy, planning, economics, work and social services > Programme planning, implementation, and evaluation > Programme evaluation
MP-MR Policy, planning, economics, work and social services > Policy > Policy on drugs and alcohol
VA Geographic area > Europe

Repository Staff Only: item control page