Developing a questionnaire

Let's briefly review the story up to now: First I talked about how I came to study the health and well-being of people who fly for a living. Next I mentioned a few of the things that had to be considered in formulating the research question. In my most recent post I talked about some of the groundwork that has to be done when preparing to launch a new research project. Now we're ready to have a look at how the research questionnaire was developed.

Some parts of the survey questionnaire were 'borrowed' from other studies that were carried out with other groups. On the surface, it's a time-saver to use measures that already exist. But the real reason we do this is so that results from a new study can be directly compared to results from studies done with other groups. In short, by using measures that are identical to those used in other studies, you can compare 'apples with apples, and oranges with oranges.'

Another reason to use established measures is that they already will have been tested for validity and reliability. That is, we will already know that they do indeed measure what they're supposed to measure, and that they tap into the topic of interest in a meaningful way.

This is an important (and sometimes vexing) issue in social and behavioral science. When we think about how to measure concrete characteristics -- like height and weight, for example -- the task is easy and straightforward. But when we're studying abstract concepts like 'health' and 'stress' the task of measuring them in a meaningful way can be quite tricky. So, when possible, it's better to use measures that already have been tested and established.

If you are interested, I have posted on my website a brief description of the standard measures that I chose for my survey. When I get to posting the results for each here on the Flying For a Living blog, I'll talk about each in more detail.

There were no standard measures for the kinds of job-related stress I wanted to study. I wanted to find out what things were stressful about civilian flying jobs, and just how stressful they were perceived to be. Since I couldn't find any suitable questionnaire about that, I had to develop my own from scratch.

First I interviewed a handful of airline pilots and flight attendants, quizzing them about what was stressful about their jobs. These earliest interviews were quite free-form. I didn't want to prompt any particular things that I thought might be stressful. I wanted them to tell me whatever popped into their heads.

When I reviewed my notes after these interviews, I took particular note of the kinds of things that many -- or even most -- of the interviewees seemed to mention. These became the core items for that part of the questionnaire.

Next, I put together a paper version of the whole questionnaire -- the standard measures as well as the items that developed from the interviews. One hundred copies of that questionnaire were printed and distributed to working crew as a 'field test.'

You may recall that I said in an earlier post that I got the idea for this study from listening to the tales of members of my family who fly for a living. My flying relatives and some of their colleagues now took on the task of distributing the field test version of the questionnaire to their flying partners.

The instructions on the questionnaire explained that this was a field test version of the survey. Participants were asked to fill in the survey, but also to comment on whether they thought the questions were meaningful and relevant. I also asked them to note any topic that they felt was important that might have been overlooked, and invited them to submit additional items that might be appropriate.

Each questionnaire was presented in a postage-paid manila envelope, pre-addressed to me, so that the field test participants could fill in the survey at their leisure and then just pop the envelope into the mail. This was done for convenience, but it also ensured their privacy: No one else was going to look at what they wrote on the questionnaire.

The response was very gratifying. It was apparent from what I read on the returned questionnaires that the participants took the project very seriously, and that they were enthusiastic about it. The field test yielded many good suggestions for improving the survey. Some participants literally wrote pages of ideas and comments! And while the questionnaire was a bit lengthy, the participants skipped very few items. I now had very good test data to work with to develop the final version of the questionnaire.

Next: Launching the Aircrew Stress Study...

** Copyright © 2007 by Bobbie Sullivan. All rights reserved. **