Chasing indicators

There’s a p=1 indicator of whether you’re at risk for cardiovascular disease: Whether you actually have cardiovascular disease or not.  (You can take the opposite vacuous tautology and point out that everyone is at p>0 risk for CVD if you like.)  If we want something a bit more useful as a predictor, though, we have to start looking at indicators.  This is where we get ourselves into trouble.

When people started looking at atherosclerosis and trying to figure out what caused it, they looked at the blockages themselves and discovered an awful lot of cholesterol — partly because it was an easy to find component, composing most of the mass of heavily-developed arterial plaques.  From this observation it was hypothesized (not unreasonably) that cholesterol caused atherosclerosis, and people were told to stop eating so much cholesterol.  (It turns out that cholesterol is synthesized in the liver to the tune of about 1g/day, dietary cholesterol intake is usually somewhere well below 0.5g/d, and endogenous cholesterol synthesis decreases in proportion to dietary intake.  So much for that idea.)

Then a curious thing happened: Efforts to confirm experimentally that cholesterol caused the formation of arterial plaques ran into roadblock after roadblock.  The nice simple hypotheses about cholesterol per se devolved into more and more complex hypotheses about HDL (“good”) and LDL (“bad”) cholesterol*, then something about triglyceride levels, and more recently the further distinction between “light, fluffy” LDL (“good bad cholesterol”) and “small, dense” LDL (“bad bad cholesterol, really we mean it this time”).  This causes people like me to lick our lips in anticipation of some good crunchy science and most people to throw up their hands and rant about “what those scientists are telling us this week”.

On the other side of the fence, though, observational studies of the cholesterol model seem to confirm it, at least at first.  Participants in a study who lower their cholesterol intake show lower CVD markers than the control group, until larger population studies eventually conform to the experimental hypotheses.  Then participants who lower their serum LDL levels show lower risk, at least until a larger study comes along and conforms to the next refinement of the hypotheses.  Statins look like wonderful life-saving drugs, at least for a few years.  What gives?

The first problem is the standard set of observational biases.  Particularly in endocrinological and nutritional studies it’s often difficult (if not impossible) to do a double-blind placebo-controlled study without screwing things up, leading to control groups that aren’t very controlled and rampant opportunity for confirmation bias.  In many cases the Hawthorne effect brings the entire experiment into question.

The second problem is that, especially in larger population studies, it’s often hard to separate correlation from causation.  You might find in such a study that whole grain consumption is correlated with lower CVD risk markers, or that late-night eating is correlated with higher body-fat percentage.  Neither one, however, is causal — people who eat more whole grains in response to our friends at the Wheat Board bombarding us with propaganda about how healthy they are** probably care more about their health in general, and are likely to get more exercise, consume less HFCS and vegetable oil, and so on; conversely, people who eat late at night are likely to do so because they’re inveterate snackers (and thus consume more and worse calories in general) than because they’re adhering to a particular intermittent-fasting protocol.

So it goes with education indicators as well.  Conventional wisdom leading up to the Occupy Wall Street protests was that having a Bachelor’s degree improved your chances in the job market; several billion dollars worth of student debt later, we discover that it may be more likely that the kind of person who gets a good job is also the kind of person who picks up a four-year college degree along the way***.  Similarly, we observe that high-school dropouts tend to have poor employment prospects, and thus President Obama has suggested that students be forbidden from dropping out of high school until they’re eighteen.  Steve Chapman doesn’t think there’s much of any causation in this correlation:

Most states now allow students to drop out at 16 or 17. As a general rule, though, quitting high school restricts your options and reduces your income. Few adults would advise a youngster to leave without a diploma.

But general rules don’t apply in all cases. The question here is not whether most students are better off finishing high school; it’s whether the kids who otherwise would drop out are better off being forced to finish high school.

That’s a very different question. Candidates who stay in the presidential race past April are far more likely to get the nomination than candidates who give up in January. But Rick Perry wasn’t going to win even if he had stayed in till Christmas. If you’re headed in the wrong direction, it doesn’t help to keep going.


The problem is that the youngsters who are most likely to drop out are the ones who are least likely to learn if they stay.

If they are 1) struggling to pass, 2) unwilling to apply themselves, 3) chronically tardy and absent, or 4) simply not very bright, they won’t learn much from being locked in a cell—I mean a classroom—for two extra years.

James Heckman, a Nobel laureate economist at the University of Chicago who specializes in education, is skeptical of the proposal. At the college level, he told me, “The returns to people who are not very able or not very motivated are typically quite low.” There is evidence that kids may get some benefit from being required to stay in high school until 16 instead of 15, he says, but “it’s a weak reed to lean on.”


It’s not clear that laws like this will even work. A 2010 Johns Hopkins University study found that when six states raised the mandatory attendance age, three saw no increase in graduation rates—and one saw a decline. Coauthor Robert Balfanz praises the 18-year-old mandate, but told The New York Times that “it’s not the magical thing that in itself will keep kids in school.”

If you want to keep unwilling students in school, you can spend money on truancy enforcement, which means taking money away from the willing students. It would be more rational to use the funds on education improvements so more kids will choose to stay.

Perhaps we ought to require that Presidential hopefuls obtain a minimum number of credit-hours in statistics and experimental design.  Hey, if more time served in the classroom is correlated with better performance….


* HDL and LDL are in fact cholesterol-transporting lipoproteins, not forms of cholesterol themselves.  This confusion drives me up the fucking wall.

** Whole grains aren’t necessarily any less of a problem from a blood-sugar perspective than their more heavily-processed cousins.  For example, whole-wheat bread and white bread have the same glycemic load.

*** This is of course a vast oversimplification of the issue.

7 Responses to “Chasing indicators”

  1. 1 mac
    January 30, 2012 at 16:21

    Reminds me of a Lewis Black routine:
    “Why do we trust sunblock? The people who told us about sunblock are the same people that when I was a kid said eggs were good. And then they said eggs were bad. And then they said they were good…then said they were bad…then, they actually said that the yolks were bad…the whites were good….MAKE UP YOUR MIND. It’s breakfast, I gotta eat.”

    Can I get some decent advice on eating better that doesn’t involve reviewing everyone’s research methodology, sample size, test results, and conclusions, in detail?

    • January 30, 2012 at 16:34

      Can I get some decent advice on eating better that doesn’t involve reviewing everyone’s research methodology, sample size, test results, and conclusions, in detail?

      I am just that kind of nerd. Here’s my version. (Also? Go ahead and eat eggs, they’re awesome. Bacon isn’t going to kill you, either, although there are one or two mild quibbles about it so I’d avoid going all Epic Meal Time.)

      You might also enjoy the paleo community (Mark Sisson et al.); IMHO they give a very conservative description of what “eating right” entails. Eat 90% paleo and you can pretty much quit worrying about your diet.

      • 3 mac
        January 30, 2012 at 17:12

        That’s actually why I visit here: ‘Come for the cursing and econ snark, stay for the nutrition analysis.’

        I don’t worry about eggs, or bacon. EMT is astoundingly funny, but my attempt at Jack Daniels’ Bacon Grease EggNog didn’t go well. Apparently, the nog should be warm first. Who knew?

        This past summer, I needed a physical for a scouting event. I got one at a Walgreens ($35, no appt? I’m there). My bp is a little high (130+/80, late in afternoon) and I get a lecture about seeing my pcp to get my cholesterol checked.

        Recently, the internets bring me commentary that the links between fat and cholesterol, and from cholesterol to CVD are a bunch of garbage that was sold to a few, key members of the USGov way back when. All these accepted nutrition guidelines begin to look like voodoo. Just because we’ve been sprinkling fresh chicken blood in a circle widdershins for the last 40 years doesn’t mean it works. Thus the reason for my despair.

        • January 30, 2012 at 17:51

          Come for the cursing and econ snark, stay for the nutrition analysis.


          Seems to me that the best way to avoid the nutrition voodoo is to either (a) look for advice that proposes a mechanism (“Avoid anything with bunches of fructose; only the liver can really metabolize it, and once it’s full up on glycogen all that fructose will turn into triglycerides”) and has at least a few observational studies to back it up, or (b) do what I do and find bloggers who’ll do (a) full-time, then cadge their links.

Leave a reply; use raw HTML for markup. Please blockquote quotations from the post or other comments.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

anarchocapitalist agitprop

Be advised

I say fuck a lot



Statistics FTW

%d bloggers like this: