maketheworldworkbetter

Statistically informed ideas on how to make the world work better.

Category: Statistics

The irrationality of modern child safety

Soren in his car seat

Soren in his car seat

Like most new parents, I devote a fair amount of thought to keeping my baby safe, but at the same time want to know when I can cut corners to reduce my stress level without increasing the risk too much. For example, the other day we were driving back to Quebec from Boston, trying to make it to the kennel to pick up the dog before it closed at 5:30. It was our first trip with Soren. Of course, traveling with a baby is more complicated than as a couple, and we got to a point where it was pretty close whether or not we could make it back. And then Soren pooped, and needed to feed…

Our dilemma, then, was whether Ju-hong would attempt to deal with one or both of these things while the car was moving (illegal of course) or whether we pulled over and risked all the inconvenience of not making it back in time to pick up the dog: an extra day’s charges, an extra hour of driving to get the dog the next day (with Soren in tow, of course), lost work time, feeling bad for the kennel owners, etc. In the end we stopped, and made it back a bit late but still got the dog.

Most parents would say stopping was the safe thing to do. Conventional wisdom says that letting a kid in a car without proper restraint is tantamount to murder. But most parents and conventional wisdom are wrong in this case, as were we. Read the rest of this entry »

Advertisements

Optimized charitable giving, evidence-based medicine, and the risk of thinking we can measure everything

GiveWell logo, taken from there website.

The GiveWell logo, taken from their website.

I read an interesting blog post this morning on Wonkblog about how some people are getting jobs on Wall Street in order to save the world: the idea is to make as much money as quickly as possible, live on next to nothing, and then use the saved money to save the world more efficiently than one could by joining the Peace Corps or becoming a doctor.

The post discussed a website/organization called GiveWell that takes a very hard-nosed, analytical approach to how we should most efficiently use our charitable dollars to do good in the world. The ballet or the symphony is nice, but by buying bed nets to prevent malaria you could be saving children’s lives for very little money, so guess which GiveWell recommends you to donate to? They choose a small number of top charities among a large number they review, and they are very careful not to make claims that the non-top charities are not useful, only that there is very good evidence that the top charities are useful. I am truly impressed with the thoughtfulness of the approach and the quality of the research they seem to have done.

But – and there’s always a but – it struck me that there is a limit to this approach to charitable giving, and it is strikingly similar to a limitation of evidence-based medicine that I’ve been bumping into recently. Read the rest of this entry »

Plummeting marriage rates in Quebec

In my previous post I mentioned that marriage rates are very low here in Quebec. I just came across this surprising statistic: In a recent study of 354 women giving birth here at the hospital in Sherbrooke, only 28 (8%) were married! (The 95% confidence interval is 5% to 11%, so random sampling does not explain the low rate. It’s less clear how well it could be generalized to Quebec as a whole – the sample is likely not perfectly representative, though it’s hard to know if marriage rates are higher or lower than would be expected generally.)

Statistics for current marriage rates are tricky: do we include people in their 60s who married when cultural norms were different? Do we look only at new marriages, and risk failing to capture people who are just delaying marriage but will eventually marry?

In this sense, the 8% statistic is particularly telling because it shows just how few people here consider marriage a prerequisite for a family. In the US, about 59% of births occur to married parents, or 7.5 times more than here in Quebec. The dramatic shift away from marriage here becomes even more clear considering that there are fewer single (i.e. unpartnered) mothers in Quebec than in the US (I’m making an educated guess here due to more poverty in the US). The vast majority of people looking to start a family here see marriage as irrelevant.

Ironically, of all the places I’ve lived, Quebec has the strongest family values. All of society is structured to give parents lots of time with their kids and to help them raise kids. This can be formal and legal: For example, public day care costs $7 per day, everyone gets lots of vacation time, and education savings plans are fantastic. More importantly, it’s also cultural. I’ve seen many instances of employers encouraging employees to take time off to be with their families under circumstances where that would be rare in other countries, and there are certain times of the week where no one participates in organized activities (e.g. sports) because everyone is with their families. My challenge to conservatives in America is thus to explain how Quebec can have such strong family values if marriage is essential for family values…

What crime statistics, standardized tests, and scientific researchers have in common

I had thought about calling this post “Cohen’s law for predicting distortions in incentivized systems.” Tongue-in-cheek of course – it’s approximate, and thus not really a law. And I don’t like the self-aggrandizing habit of naming a law after oneself. And it would have been a dry title, and you probably wouldn’t be reading this. Nonetheless, this post is about the single most important thing that everyone designing public policy should understand. It is about the principle that makes most public policy fail (or work less well than intended).

Most public policy is designed to achieve certain goals – lowering crime, improving education, advancing scientific knowledge, improving health care etc. And most of the time, these goals are achieved by trying to get the right people to do the right things: police to arrest criminals, teachers to teach well, researchers to perform well, doctors to treat patients well, etc. In order to encourage this, most policy incorporates some form of incentives: tax structure, salary scales, rewards for good performance, and so forth. Police departments are judged by their crime statistics, and in turn find ways to pressure their officers to deliver these stats. In US education policy, No Child Left Behind was supposed to implement standards to encourage schools and teachers to perform better. Researchers who are productive are more likely to get funded for their next research grant. And so forth.

Read the rest of this entry »

The benefits of a Mediterranean diet: thoughts on the new study

olive oil

A new study in the New England Journal of Medicine claims that a Mediterranean diet (lots of nuts, fish, olive oil, and fruits and vegetables, not too much dairy, red meat etc.) can dramatically lower cardiovascular disease events and mortality. My opinions on this study are a bit schizophrenic – it confirms what I’ve been saying for a while, but I don’t trust the methods. In the end, I think the study is largely correct, but somewhat by luck.

Read the rest of this entry »

Obesity and mortality: challenging the conventional wisdom, part III: BMI is nearly useless

I’ve just written a couple posts critiquing the recent study in the Journal of the American Medical Association claiming that overweight people are at lower risk of mortality. This is the third and last, and will be a detailed exploration of why BMI is a bad metric of obesity.

I am certainly not the first to claim this, and indeed there are many researchers who have been defending measures such as waist circumfrence for years. But my point is not just to trash BMI, nor to defend another measure in particular (I think waist circumfrence is also a red herring), but rather to use BMI as an example of how shoddy thinking can lead to the adoption of poor metrics and thus serious public health consequences.

Read the rest of this entry »

Obesity and mortality: challenging the conventional wisdom, part II: Categorisation is fatal

This post is the second in a series critiquing the much-publicized recent study by Flegal et al. in JAMA, which claims that there is a lower risk of mortality among overweight people than normal weight people. As in my last post, my main beef is that our standard categories of BMI are a problem, but here I argue against categorizing altogether. In fact, I think there is a good argument that researchers categorizing BMI is literally fatal for many people.

How so? Well, once we put someone in a bin such as “obese” or “normal,” we have a mental tendency to accept the categories as something real, something objective. But they are not. These cut-offs are arbitrary, chosen mostly because they are nice round numbers. This can be seen easily on a histogram of 53,000 BMIs taken from the NHANES study (American adults age 20+, roughly 1998-2010):

NHANES BMI dist Read the rest of this entry »

Obesity and mortality: challenging the conventional wisdom, part I

A recent article in the Journal of the American Medical Association has gotten a lot of attention for claiming that risk of mortality is lower in overweight people than normal weight people. I will write a series of posts critiquing this article, which I think demonstrates many of the problems with modern epidemiology and research in this field.

This first post is a very simple one: We do not correctly classify obesity. The standard classification used by the article is as follows:

BMI < 18.5 : Underweight

18.5 < BMI < 25 : Normal weight

25 < BMI < 30 : Overweight

30 < BMI < 35 : Obese (grade 1)

35 < BMI : Obese (grades 2-3)

I just calculated my BMI for the first time. BMI is calculated as mass (kg) / height×height (m). I weigh 175-180 lbs (80 kg) and am 6 feet (184 cm) tall. My BMI is thus  around 23.6 – 24.4, depending on the day. This puts me at the upper end of “normal.” In other words, anyone a bit heavier than me is overweight. Here I am:

207699_10150155487678094_5905628_n

While I am not underweight (at least not much), anyone who knows me will tell you I am a bit on the lean/slender side. I have to work to avoid being too skinny. I am not at the upper end of normal, I am at the lower end of normal – a pretty broad range on this scale.

Everyone who works with BMI admits that it is an imperfect measure – that it can classify body builders in heavier categories, for example, and that it doesn’t really incorporate body type. But I’m not a body builder, and I have a standard body type, and the standard scale is telling me that I’m not far from being overweight. According to the scale, I would have to lose 55 lbs (25 kg) to become underweight! Even when I had cancer and became a bit emaciated (to the consternation of everyone who saw me), I only lost about 20 lbs.

I use myself as an example, but I am convinced the standard cut-offs are wrong. Many people who are underweight (anorexic or wasting away due to cancer, and thus at higher risk of mortality) are in the “normal” category. And many people at the perfect weight are in the “overweight” category. So it’s no wonder we will see strange results like that from the JAMA study.

How about you? Is your BMI about right according to the scale above?

Guns are (probably) not responsible for the high murder rates in the US: how to understand the stats

As I stated in my last post, I support gun control, and I am really appalled by what happened in Newtown. But this is one (rare) case where I think the people on the right may understand the facts better than those in the center and the left. Two of my favorite columnists, Charles Blow at the NY Times and Fareed Zakaria at the Washington Post, had recent columns (here and here) on how the US is the clear exception in the developed world on both gun ownership and murder rates, and suggesting that solution is simple: limit guns, have less murders.

These are nice, well-argued columns. Unfortunately, they appear to be wrong. I’ve been digging in the data a bit, and I think both of these columnists have taken correlation to be causality. (Note: Correlation can sometimes imply causality, contrary to popular belief – one needs to know how to do the right analyses and have the right data. But this is not the case here.) So I will work you through the data, as an example of how to do a more thorough statistical analysis of a question like this. Don’t worry, I won’t get too technical, and I’ll keep it intuitive…

Read the rest of this entry »

Why personal advice columnists should consider population processes

Emily Yoffe as Prudie on Slate.com

I admit it. I’m an addict. An addict to the prurient, the salacious, the voyeuristic…advice columns. When Dear Prudie on Slate posts a new column of advice for the gay man who’s in love with his twin brother, or (yet another) bride-to-be fighting with her mother-in-law-to-be, or a couple whose sexual tastes are a mismatch, I’m all over it. I just can’t get enough.

And yet I’m also deeply disappointed in Prudie for her failure to understand population processes! After all, what reasonable advice columnist doesn’t understand population processes? OK, maybe I’m expecting too much. But Prudie is giving bad advice on a regular basis because she treats each case as if it were isolated from the rest of the world and not subject to population processes.

By now you’re probably wondering what a population process is. No worries, you’re not alone – you have at least Prudie for company. And I’m going to convince you that if you want to give good personal advice (or make good personal decisions yourself) you’ll be better off if you understand what they are. Read the rest of this entry »