Weight Loss “Success”: ShapeUp versus The National Institutes of Health – Not Exactly the Fight of the Century

I have written previously about the chicanery that abounds in weight-loss research. The underlying problem is that because no study over the past 30 years has demonstrated the ability of any of the programs tested to result in sustained weight loss for all but a small minority of the folks involved, researchers have to either play with the numbers or greatly lower the bar or both when they are claiming success with their programs. In a recently published article, “The Last Man Standing Fallacy,” I showed, using a study that was sent to me as an example of a “successful” weight loss program, how researchers often create success out of failure by playing with the denominators in their calculations. More recently, I was sent another article with a similar declaration; that this study was a demonstration of an employee weight-loss program that actually “worked.” I had much positive feedback on my dissection of the last study and so I thought I would proceed similarly with this one, with the addition of offering a science-based alternative for actually helping people with weight-related concerns.

The article, “Teammates and Social Influence Affect Weight Loss Outcomes in a Team-Based Weight Loss Competition,” was published in “Obesity” in July 2012. Please note that two of the four authors are intimately involved with ShapeUp, a workplace wellness vendor that has previously made mathematically impossible and clinically meaningless short-term weight loss claims, which they quietly retracted after they were “outed.” (Also, the customer cited in those claims fired them.)

Let’s examine together the findings and the claim that this program “worked.”

This was a 12-week, team-based intervention that involved more than 5,000 participants and included a wide variety of behavioral techniques including: self-monitoring, regular feedback, social support, online tracking, and a host of ongoing incentives offered every two weeks. In addition, over 700 free wellness activities including cooking lessons, basketball clinics, Zumba®, Pilates, yoga classes, nutrition seminars, and stress-reduction activities were offered during the 12-week program. A gala kick-off event and a closing session featuring recognition and rewards for the winning teams were included to boost participation and morale.

So, what were the results of this elaborately designed and executed weight-loss intervention? More than one-third of the participants dropped out in the three months.

1. 500 attended the opening session – 100 attended the closing session
2. Average weight loss of between six and seven pounds (about one BMI unit)
3. Weights were self-reported

It is hard to imagine how anyone could see a one BMI unit change in weight in three months with more than 30% of participants dropping out as particularly impressive. Many studies have shown much bigger weight losses in similar periods of time. But, this is of course not the most important issue by any means. Almost all weight-loss programs result in weight loss for a short period of time – and almost everyone in the field agrees that the only thing that really matters is whether the weight is kept off – and to repeat for the sake of clarity, no study has ever demonstrated that outcome, long term, for any but a small fraction of the folks who started the program. This study is no exception. As with most studies, there is no reporting of weight maintenance, no follow up for even a short period of time — because there is not likely to be much, if any, maintenance.

How many more studies like this do we need? We already have hundreds — all saying — the same thing: you can manipulate people to lose weight with all kinds of behavioral, social, incentivized hocus pocus — but they don’t keep the weight off. And like most of the studies that do track maintenance after program’s end, the paltry weight losses here will almost assuredly show a trajectory of regain in the next six months to two years.

The response to my critique of this study is particularly instructive (and in fact quite common). The argument went something like this: “You don’t know if people kept the weight off and you can’t draw any conclusions at all unless you know the answer to that question.” And then the clincher: “The program was for a period of time, but the product continues, so if those folks are still participating, then it seems possible they did keep the weight off.”

To summarize:

• No study over the past 30 years has demonstrated long-term weight maintenance for all but a small percentage of participants.
• Any studies that have traced maintenance after the program ends have shown a strong weight regain trend.
• This study resulted in a short-term weight loss of about one BMI unit (six-seven pounds) for the two-thirds of the employees that made it through the 12 weeks.

Weighing (pun intended) these results with the hundreds – (at least) of previous similar studies – is it warranted to suggest this as a successful intervention? As I have pointed out previously, making up definitions for “success” in weight loss interventions is unnecessary. It turns out that we have had a sound, science-based definition of what constitutes a successful weight-loss program for more than 20 years. According to the National Institutes of Health:

In evaluating a weight-loss method or program, one should not be distracted by anecdotal ‘success’ stories, or by advertising claims. The information that should be obtained about the program includes:

• Percentage of participants completing the program
• Percentage completing who achieve weight loss
• Proportion of loss maintained at 1, 3, 5 years
• Number who have negative medical effects as well as their kind and severity.”
(NIH Technology Assessment Conference Statement: March 30 – April 1, 1992)

So, the argument presented that “you don’t know if people kept the weight off and you can’t draw any conclusions at all unless you know the answer to that question,” is in fact TRUE!

Drawing the conclusion that this study presents a program that “worked” is neither appropriate nor supported by the findings. I invite all who are interested to read this study carefully and decide for yourself what an appropriate and accurate scientific evaluation of its claims really is. Improving our ability to accurately decipher the scientific literature can lead us to do more good and less harm with our interventions.

And speaking of doing more good and less harm, here is the good news! There is an evidence-based alternative available for helping employees with weight-related concerns. I have been involved with the Health at Every Size® movement for almost two decades and have implemented these workplace programs all over the country. Given the abject failure of traditional weight-loss programs for the vast majority of folks who participate, we offer a detailed explanation of this alternative in our book – and also in our FREE, two-part white paper series:

Part 1: Weight at the Workplace – Shifting the Paradigm
Part 2: Weight at the Workplace – 10 Things You Can Do Right Now to Ease Concerns About Your Weight & Improve Your Health

Give it a look – what have you got to lose?

With mathematical validation and editorial assistance from Al Lewis.




Jon Robison, PhD, MS, MAJon is an accomplished speaker, teacher, writer and consultant. He has spent his career advocating that health promotion shift away from its traditional, biomedical, control-oriented focus, with a particular interest in why people do what they do and don’t do what they don’t do. Jon has authored numerous articles and book chapters and is a frequent presenter at national and international conferences. He is also co-author of the book, “The Spirit & Science of Holistic Health — More than Broccoli, Jogging and Bottled Water, More than Yoga, Herbs and Meditation.” This work formed the foundation for one of the first truly holistic employee wellness programs — Kailo. Kailo won awards in both Canada and The United States, and the creators lovingly claim Jon as its father. Contact Jon at: jon@salveopartners.com or jonrobison.net.

View all articles by Jon Robison »