Regular readers know that that Peter and I do a semi-regular podcast on obesity-related issues. This week, I have a discussion with psychology researcher (and fellow ScienceBlogger) Jason Goldman (UPDATE: Jason's thoughts on the podcast here). For the uninitiated, self-report data refers to information that people provide themselves - questionnaires and interviews are very common examples. This is in contrast to direct measurement, which is exactly what it sounds like - researchers measuring your height and weight themselves, etc. The podcast was inspired by a recent conversation where I bemoaned a press release which failed to acknowledge the limitations of self-report data, which I felt could lead to misleading conclusions.
The press release details a recent project which involved interviewing families on their cooking habits. The interesting finding was that most families said they were eating meals at home, which goes against the thinking that people are increasingly eating-out, which is thought to be a contributing factor in the obesity epidemic. Now that's intriguing, but it's hard to tell how much stock to put in that finding, since it's hard to remember how often you eat out, and since eating at home is a socially desirable behaviour - both common issues with self-report data (I would have found it really interesting to compare their responses to a direct-measurement of their eating behaviours). Fortunately the MSc Thesis that the press release was based on actually spends a lot of time discussing limitations of their approach - including some of the issues related to self-report data - but the press release does not.
I think this will come across in the podcast, but I want to be on record as saying that I have no real issues with self-report data per se. Almost all projects involve at least some element of self-report (e.g. even tightly controlled interventions rely on the participants to tell them about medical conditions, pre-study diet or physical activity, etc), and we frequently discuss self-report studies here on Obesity Panacea. But it's still important to understand the limitations of self-report, especially when they could fundamentally change the way you interpret a paper. Failing to do can lead to all sorts of trouble, all of which is nicely outlined in this review paper which Jason and I discuss in the podcast.
Enjoy the podcast, and don't forget to subscribe to future podcasts via iTunes! This is the first of what will hopefully be many podcasts involving
other members of the ScienceBlogs community. Thoughts and suggestions
would be greatly appreciated!
Travis
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54 (2), 93-105 DOI: 10.1037//0003-066X.54.2.93
- Log in to post comments
Where does the "thinking that people are increasingly eating out" data come from? I'm wondering what would make that the presumed default that the self-reported data are being measured against, rather than the other way around.
Good question. Here is one. It uses self-report data as well, but it lists the limitations in the discussion of the paper, and is also much larger, nationally representative, and examined the same issue over time.
Again, I don't really have a problem with this study or with self-report in general - I just think it's important to take a study's limitations into account, especially when those limitations may have a large impact on the conclusion of the study itself.
Travis
Oh, definitely. I was just thinking especially with the way the economy has been going lately that I would be surprised if people are eating out more, so I wondered where that trend came from.