Science: Usually not spectacular

The thing about science is that it only gives you reliable data about what you actually did in your experiment, rather than what you wanted to do.  Most of the time our reach exceeds our grasp, so while we’d like to be able to detect faster-than-light neutrinos what we actually do is measure the time it takes for a neutrino beam to go from point A to point B in what we hope is a fairly constant relativistic frame of reference.  “Holy shit, this is amazing!” results are often attributable to unexpected correlations in the experimental setup rather than holy-shit-amazing new properties of the Standard Model.

At least, that’s how physics works, when we’re doing it right.

Dr. Andro finds a new study purporting to show that:

[A] high-protein diet with a group-based dietician-led support and education programme easily deliverable in a real-world setting does not promote greater weight loss than the prescription of a high-carbohydrate diet

We can expect this to be trumpeted to the heavens by the hearthealthywholegrains evangelists, of course.  But, y’see, there’s a problem — and if your eyebrows rose to the heavens at the word “dietician” in that excerpt you’re not far wrong.  See, the “diet” under consideration asked participants to cut 500 kcal/day from their intake, completely unsupervised except for an initial meeting with that dietician and what Dr. Andro scare-quotes as “culturally-appropriate recipes”.  The instrumental variable the researchers thought they were studying was macronutrient composition, but a look at the data (click through to Dr. Andro’s writeup) shows that what they were actually studying was variance in instruction to the dieters.

“High-carb” dieters reduced their daily intake by about 250kcal, largely by cutting fat.  “High-protein” dieters reduced their daily intake by about 150kcal, largely by, er… that’s a good question.  They don’t seem to have eaten any fewer carbs than the “high-carb” dieters, so while this study is likely to be used as a bludgeon against low-carb advocates it is in fact nothing of the sort.  Furthermore, adherence to the “high-protein” dictum fell off drastically over the study’s two year span, which you’d have thought a qualified dietician — maybe even one certified in North Carolina — could have prevented or at least mitigated.

Note that as usual the researchers evaluated the success of the diet based on weight loss.  Was that lost weight adipose tissue?  Skeletal muscle?  Brain matter?  Meh, who cares; we got the result we were looking for, time to publish!

More and more I’m coming to suspect that the “secret” to fat loss is being enough of a nerd to (a) form a plan based on a hypothesis (this could be “fat will make you fat” or “insulin/leptin control is critical”), (b) implement that plan for long enough to see results, (c) evaluate whether the plan worked, (d) re-evaluate the hypothesis based on the results, and (e) iterate.

Next we have an admirable dissection by Paul Ingraham of a study that purports to show that post-exercise massage promotes mitochondrial growth:

• Massage reduces inflammation and promotes mitochondria? (SaveYourself.ca)

The problem isn’t so much the research as the breathless conclusions being drawn therefrom.  The post is difficult to excerpt, but here’s a hint:

These conclusions reach a fair ways past what the evidence can support. It’s a red flag that “clinically beneficial” is in abstracts, headlines, press releases: the physiological effects are too complex to assume that they are all therapeutic, or even benign. The research is interesting and worthwhile, but it badly needs a critical perspective that I haven’t seen anywhere else yet.

Click through and RTWT.

The moral of these stories is that science is more often a slow, arduous process than it is a spectacular one.