Audit of the York Nutritional Laboratory survey

Townsend Letter for Doctors and Patients,  August-Sept, 2002  

• E-mail
• Print
• Link

This report contains an independent audit of the York Nutritional Laboratory survey conducted by the Department of Health Studies University of York on behalf of the British Allergy Foundation

Background

Adverse reactions to food can cause a range of symptoms throughout the body. Some of these reactions are mediated through the immune system either by IgE (food allergy) or, more controversially, by IgG (food sensitivity). Over the last few years the York Nutritional Laboratory (YNL) have been conducting enzyme-linked immunosorbent assay (ELISA) tests on blood samples to detect raised food-specific IgG in the serum of people with one or more, usually chronic, symptoms.

YNL has conducted an extensive survey of people who have been tested and given dietary advice based on the results. The survey, was distributed to a random sample of 4,200 individuals in the UK who had taken a 'pin prick' food sensitivity test between February 1998 and August 1999. A 42% response rate resulted in a total of 1761 questionnaires returned for analysis. Approximately 50% of all responders reported an improvement in symptoms at point 4 or 5 (relatively high improvement).

Whilst these results are encouraging, survey results are susceptible to several forms of bias including:

1) Mistakes in data entry or data analysis: Errors in transcribing data from the form to the computer.

2) Responder bias: Respondents may overstate improvements in their health because it is a company questionnaire.

3) Non-response bias: Those who do not respond to questionnaires are not representative of the general population and may be reluctant to report lack of benefit or lack of compliance with the diet. As a result, survey results which are based solely on those who respond are likely to be biased in favour of the interventions.

The Department of Health Studies, University of York was commissioned by the British Allergy Foundation to carry out an audit of the York Nutritional Laboratory in order to be sure that the survey results are an accurate reflection of the views of food sensitivity test users.

Methods

The audit comprised 3 parts:

1) Accuracy of data entry and analysis.

A random sample of 100 cases was checked against the original questionnaires to verify the accuracy of data inputting and coding. An error rate of more than 5% would be regarded as significant.

2) Validity of responses of responders and estimate of nonresponders

Two letters were developed inviting previous responders and nonresponders to the YNL survey to participate in the audit. These were sent to a random sample of 150 responders and 450 non-responders, all of whom had been sent the YNL survey within the last year: 70 people refused to participate in the audit, 9 responders and 61 non-responders; 3 people had moved, 1 had died, 4 had not included their name in the contact details, 2 had not yet started to eliminate the foods indicated.

The telephone interviews resulted in data for 46 responders (just under the 50 intended) and 90 nonresponders. In order to increase the number of non-responders in the survey a second list of nonresponders was obtained from YNL and letters were sent to a further random sample of 300 clients. (Unfortunately a YNL administrative error led to certain people being included on this list who had already been approached, 7 let us know). This second mailing resulted in data for a further 24 telephone interviews, resulting in data for 114 nonresponders, less than the 150 intended. Due to shortage of time, we did not attempt to increase further, the sample of non-responders.

As slips were returned agreeing or refusing to participate in the audit, names were checked against the YNL survey database to establish whether individuals were previous responders or non-responders to the original YNL survey.

Two phone questionnaires were devised, one for responders and one for non-responders. Responders were asked a subset of the same questions as in the original survey, to see if they would now respond in the same way. In addition interviewees were asked if the reported benefits (if any) were still present. Non-responders were asked the same subset of questions from the original survey but with a few changes. The question about benefit from the dietary changes was changed to include a 'no benefit' score, and the question on time taken to feel benefit was modified to allow an answer of 'no benefit.'

The return slips were distributed to the telephone interviewers in order to ensure both had an equal number of responders and non-responders. Telephone interviews took place at a time specified on the contact slip. Previous responder responses were coded and entered into an SPSS file alongside each interviewee's original responses. Previous non-responder's responses were coded and entered as new cases in an SPSS file.

3) Statistical analysis

The results from the re-surveying of responders were compared with original responses and the percentage showing a reduction in symptoms of 4 and 5 (a lot or quite a lot) estimated. This part of the study will enable us to report the proportion reporting relatively high benefit with a precision (95.0% confidence level) of approximately plus/minus 12% points and will allow us to test whether the results are significantly different from the YNL survey.