People living in neighbourhoods with more birds, shrubs and trees are less likely to suffer from depression, anxiety and stress.

OK, I have a few methodological questions based on the linked article. These are not condemnations, and the study is not linked so it seems a little churlish to criticize readers for relying on the article, but I am pretty skeptical about the value of this study based on the article.

Okay, first:

After conducting extensive surveys of the number of birds in the morning and afternoon in Milton Keynes, Bedford and Luton, the study found that lower levels of depression, anxiety and stress were associated with the number of birds people could see in the afternoon.

Well, this seems like a pretty specific thing to ask about -- "number of birds seen in the afternoon." Suspicious! Kind of smells like p-hacking! If you ask enough questions, then you are likely to get a p<0.05 result somewhere in the list purely by chance. And a hypothesis this bizarrely specific, particularly without justification, looks like the result of a fishing expedition. For example:

  • How many trees do you see in a day?

  • How many trees can you see from your window at home?

  • How many trees can you see from your desk at your workplace?

  • How many shrubs do you see in a day?

  • How many shrubs can you see from your window at home?

  • How many shrubs can you see from your desk at your workplace?

  • How much grass do you see in a day?

  • How much grass can you see from your window at home?

  • How much grass can you see from your desk at your workplace?

  • How many squirrels do you see in a day?

  • How many squirrels do you see in the morning?

  • How many squirrels do you see in the afternoon?

  • How many squirrels do you see in the evening?

  • How many squirrels do you see during the day from your desk at your workplace?

  • How many birds do you see in a day?

  • How many birds do you see in the morning?

  • How many birds do you see in the afternoon? <-- OMG P < 0.05, WRITE THIS UP!

  • How many birds do you see in the evening?

  • How many birds do you see during the day from your desk at your workplace?

  • How much of your day is spent in view of the sky?

  • How often can you see the sky in the morning?

  • How often can you see the sky in the afternoon?

  • How often can you see the sky in the evening?

  • How often can you see the sky during the day while you are at work?

Now, they try to justify this bizarrely specific hypothesis as follows: "The academics studied afternoon bird numbers - which tend to be lower than birds generally seen in the morning - because are more in keeping with the number of birds that people are likely to see in their neighbourhood on a daily basis." And yet they admit to also asking about trees and shrubs, and it seems likely that they partitioned all of the data by time of day. Welcome to the garden of forking paths, where at least one path leads to p<0.05 and every scientist is a winner!

Okay, second of all, "controlling for variation in neighbourhood deprivation, household income, age and a wide range of other socio-demographic factors" needs to be justified. Specifically, once you have a set of proposed controls in place, you need to make sure that the controls actually work. Find an even broader list of correlates with socioeconomic status beyond the obvious ones -- such as owning an SUV, having been married for >10 years, proportion of cousins > 21 with a college degree, you name it -- and make sure that none of these things show up anymore in your data. Spoiler alert: it's nearly impossible to achieve this degree of success by controlling for "socio-demographic factors". There is a reason why randomized controlled trials are the gold standard, and why there is a tremendous replication crisis in the social sciences. I just don't believe researchers who insist that they successfully controlled for something as broad as "socio-demographic factors" unless they show me that their controls succeeded at completely drowning out other uncontrolled but known correlates of what they were controlling for.

/r/science Thread Parent Link - eurekalert.org