Graded grains make finer findings: appraising the quality of a wide range of research methods

Brian Taylor, Michael Donnelly

The experiential lessons gained in the course of conducting a systematic review of professional decision-making (about the long-term care of older people) will be presented, including reflections on approaches to appraising the quality of a diverse range of research methods, both quantitative and qualitative.

Discussion will revolve around the extent to which the traditional hierarchy of evidence is appropriate for appraising the full range of methods. Is a modified version of the hierarchy an effective way to appraise quantitative and qualitative studies by, for example, using more inclusive language and distinguishing more precisely between ‘lower level’ designs? Are other aspects of methodology (such as data collection tools and the naturalness of the task or setting of respondents) appraised better with a small number of individual short scales with inclusion criteria specified on each scale, alongside the hierarchical scale for classifying types of designs?

This work has been developed in the context of the integrated management arrangements for health and social services in Northern Ireland, and through a collaboration that includes a School of Medicine and a Social Work Department. It seeks to bridge the gap between the appraisal of qualitative and quantitative methods, and between conceptual approaches that are more familiar either in health care or in social sciences.

This workshop is likely to be of interest to those involved in the Qualitative Research Methods Group, and in the joint development of methods with the Campbell Collaboration.

Ottawa 2004 O-090