p=.05 Or, the utility of messy scientific models

THE IMPORTANCE OF MESSY MODELS

I really like a recent commentary piece published by the complexity scientist, author and journalist, Mark Buchanan.

It is titled, appropriately enough A Natural Bias for Simplicity

As a fellow complexity scientist, Buchanan fully supports the argument that our scientific models need to do a better job embracing the complexity of the topics they seek to understand.

Nonetheless, given that models are -- in the vocabulary of complex and critical realism -- only traces of the things they seek to understand, they will always fall short of the evidence; that is, they will always be somehow wrong or incorrect or, at best, evolving approximations -- particularly as the topic becomes more complex.

Still, in the face of such a challenge, the reality is that we actually can and often do come to understand things, and usually rather well.  However, we do so -- psychologically (both emotionally and cognitively) speaking -- through models that are relatively messy and sloppy.

THE IMPORTANCE OF CONTEXT AND MULTIPLE PATHWAYS

Messiness does not, however, have to lead to a lack of refinement or precision.  In terms of the social and health sciences, for example, there are several very useful ways to handle this messiness.  I will quickly highlight just three:

• First, the messiness of a model is better understood and managed if we move away from one-size-fits all statistical modelling, which seeks one answer for all possible outcomes.
• For example, it is rediculous to take a model that works for a highly affluent school system or community and think it simply applies, without modification, to a school system or health issue in a poor community.  And yet we do this all the time: assume that the success of a model in one area means it applies to all.
• Related, we find that a treatment regime for blood pressure, for example, works for 85% of patients; and so assume it is good enough (p=.05); when in fact, the other 15% of patients constitute three or four different groupings for whom different models (algorithms) of treatment are most likely necessary.
• In other words -- and this is my second point -- it is important to think about the mulitple and different clusters along which a set of cases in a model self-organise.  
• For example, to push our first point further, an intervention that works for one school system or community may not work for another, given their different trajectories.  What is therefore needed instead is an understanding of how different pathways require one to push on different driving factors or forces to get a sought after outcome.  Related, driving down blood pressure for one patient group might require exercise (15%), while another medicine (55%), and still anther counseling or stress management (30%). 
• Third, messiness is better addressed if it is continually re-situated within the wider socio-ecological context in which is it being employed or used.
• For example, fixing a school system or implementing a policy to improve a community's health requires a participatory framework that informs the model on the unique profile of circumstances or factors that will ultimately determine its success, or worse its failure.

Still, whatever the fix, the ultimate point here is that messiness is a condition of complex modelling; in fact, the messiness of a model is often why it works.  However, a model's messiness can be managed, particularly when we think about the context in which it is situated and the multiple pathways along which it evolves (or, alternatively, clusters or groups).  And, when we realise that singular, one-size-fits all models meeting does not often fit-the-bill, even if we can demonstrate that it has a p-value of .05.