An article for Permaculture Activist Issue #93 Fall 2014. Reprinted here with permission of the author, Chris Warburton Brown, Research Coordinator, The Permaculture Association. This is an edited and much extended version of an article that originally appeared in Permaculture Magazine 77, Autumn 2013.

‘We are conducting a group experiment in transitioning to sustainability, which relies for its success on us being able to learn from each other, from our successes and mistakes.‘
– Ed Sears, Chair of the Permaculture Association’s Research Advisory Board

Just about everyone who practices permaculture is an experimenter. If you have ever asked whether feeding your peas with comfrey might improve their yield or how much electricity a solar panel on your roof could generate, and have then done something practical to find out the answer, then you are a permaculture experimenter. Yet one of the commonest criticisms of permaculture is the lack of scientific experiments that support its claims. Certainly, the number of articles describing experimentation in permaculture that have appeared in peer reviewed scientific journals is small (Rafter Sass Ferguson and Sarah Lovell have produced a great review of these which you can read here and see also my own archive of recently published permaculture related material. There is a deep irony here; a community of experimenters, all busily experimenting, who have produced very few scientifically rigorous experiments. This article considers why this situation has arisen, suggests some key remedies, and then describes what the UK Permaculture Association is doing to improve it.

Before addressing the issue of rigorous scientific experimentation in permaculture, we need to consider what we mean by rigorous scientific experimentation in general. Experimentation is one approach to finding answers to our questions about the natural or human world. It begins with a question or an idea (a theory), the truth (or falsehood) of which can be tested. For example, ‘does feeding broad beans with comfrey liquid improve their yield?’ and ‘Feeding broad beans with comfrey liquid will improve their yield’ are two ways of phrasing the same issue (as a research question and as a theory) which can then be tested by experimentation. My personal preference is for research questions, but traditionally science has often focussed on theory testing. The experiment itself then focuses on this single issue (feeding comfrey juice to broad beans), to see if this issue makes a significant difference to the outcome.

Three clear criteria define whether these experiments are scientifically rigorous; reliability, validity and generalisability. Reliability asks whether the experiment’s findings can be replicated by a different experimenter applying the same methods to the same question. Validity asks whether the findings are an accurate and true representation of what was found; a study in which an experimenter focuses only on findings which support their pre-determined ideas is not valid. Generalisability asks whether the results can be successfully tested in new settings, peoples or samples; a study based on one field in one season may well fail the test of generalisability. To be scientifically rigorous, permaculture experiments need to pass all three of these tests.

The Permaculture Association’s Research Strategy identifies two key criteria for permaculture research, which reflect widely held beliefs in the permaculture community; improving the practice of permaculture by permaculture practitioners and building a strong body of evidence to convince people of the value of permaculture. Improving the practice of permaculture is rooted in a comparison between what permaculture practitioners are already doing and doing things differently, to see if the difference leads to improvement. Building a convincing body of evidence means comparing permaculture practice to conventional practice so that meaningful comparisons might be made. Both of these criteria need robust, scientifically rigorous experiments to help prove the validity of permaculture approaches. Yet such experiments have, to date, been very few in number.

I think there are a four main reasons why the permaculture community has produced so few scientifically rigorous experiments. Firstly, there is an issue about the kind of people who are attracted to permaculture. Permaculture practitioners are unconventional folk outside of society’s mainstream, they are do-ers, they are investing their time and energy in projects. Moreover, many have an essentially intuitive commitment to permaculture; ‘I do what I do because it feels right’. I get the feeling that many permaculture practitioners like the idea of improving the scientific basis for permaculture but they lack the time, the skills or the inclination to devote to developing permaculture experiments in a scientifically rigorous way. All this is entirely appropriate in a movement that seeks to change the world by doing. However, it does raise problems for the development of a body of evidence for permaculture that is rooted in experimentation.

To meet this first challenge we need to find ways of harnessing the energy, passion and practical projects of the permaculture community while up-skilling people with simple research skills and without making excessive demands on people’s time.

The second issue is that of complexity. Permaculture seeks to engage at a level of complexity that conventional science generally does not. For example, if we want to compare food outputs from conventional farming with those of permaculture, the standard comparison would be wheat yield per hectare. But wheat yield per hectare is a very poor comparator for permaculture growing; where conventional wheat growing is monocultural and the entire crop is harvested in a single day from fields of several hectares, permaculture growing is polycultural, with crops harvested over a prolonged period from small fields or gardens. Moreover, permaculture practitioners are generally interested in a far greater variety of yields than conventional farmers. Participants in a workshop at the 2012 UK Permaculture Diploma Gathering identified sixteen different permaculture yields, from biodiversity to effect on personal happiness. Conventional science methods are perfectly capable of measuring each of these yields individually, but permaculture practitioners are interested in several of these yields at once; crop yield is only one part of the story. Even if we limit ourselves to looking only at crop yield we are not free from complex questions. Just weighing the apple harvest potentially misses some key yields: keeping quality, taste, length of harvest period, suitability for juicing (or drying, or chutney making), commercial value (possibly linked to techniques such as Community Supported Agriculture), quality of the fruit. Yields are also subjective: a grower might consider one sack of fruit from an apple tree with no labour investment a higher yield than two sacks from a tree that was pruned, cultivated and fed. Inputs of time, labour, fertiliser etc. need to be considered alongside yields. Designing rigorous experiments that can test more than one of these things at once is challenging, and not typical (though not wholly unknown) in conventional science.

To meet the second challenge we need to find ways to handle complexity. Simply trying to compare permaculture yields to conventional wheat yield per hectare actually misses the whole point of permaculture as a holistic, integrated system.

This leads directly to the third challenge; the philosophical approach to knowledge taken by much current scientific research. Preparing content for the Permaculture Digest, I have found little of use to the permaculture community in conventional plant science literature. Because researchers are expected to produce strong statistical significance from their experiments, work has become lab-based, not field-based; variables are far easier to control in a lab, and experiments can be conducted much more quickly. Moreover, in order to avoid complexity ‘contaminating’ the results, there is an emphasis on the smallest units of analysis; genes, microbes, chemicals. This boosts conventional crop yields, but inevitably leads to interventions at a microscopic level and to GM crops. If adopted, this kind of ‘rigorous’, lab-based scientific approach has the potential to undermine the key feature of permaculture; its basis in systems thinking and a holistic approach to life. Instead, we need to build on the scientific literature which places the growing of individual crops into its wider social and environmental context; research typically rooted either in low-tech, developing world agriculture (e.g Agronomy for Sustainable Development), or in a consciously chosen whole-systems approach (e.g. The Journal of Soil and Water Conservation). But this approach does not readily lend itself to traditional scientific experimentation.

The meet the third challenge we must find a scientific philosophy and methodology that lends itself to holistic, systems-based approaches, focusing on the macro rather than the micro. And we need to develop rigorous ways of conducting experiments that both grow out of and support this philosophy.

The fourth challenge is the minimal financial resources available for permaculture research. Although there have been some great small projects, including Masters and Doctoral theses, as far as I am aware permaculture has yet to secure funding anywhere in the world for a really substantial academic research project. The number of people employed on permaculture research is tiny. My own post, for example, is funded wholly by private philanthropy (thanks to Lush Cosmetics), my current funding ends at Christmas, and the projects I support rely on volunteer amateur experimenters. This contrasts with the millions of pounds of research funding spent on conventional agricultural experiments. If permaculture experimentation is to become scientifically rigorous, it needs to attract substantial mainstream funding and partner with existing research institutes.

To meet the fourth challenge we need to secure research resources, which means convincing funders that we can deliver results, and demonstrating that permaculture can solve some of the challenges humanity faces.

In the first half of this article I have tried to set out what I believe are the seven key criteria of scientifically rigorous permaculture experiments:

1. They should produce results which are valid, reliable and generalisable.
2. They should either improve permaculture practice or generate rigorous evidence that can convince the sceptical.
3. They should either answer clearly stated permaculture-related research questions, or test clearly stated permaculture-related theories.
4. They should harness the particular strengths of permaculture practitioners while addressing their weaknesses.
5. They should deal with complexity.
6. They should focus on the holistic nature of permaculture and communicate the advantages of such a focus.
7. They should be properly resourced.

I do not believe that this kind of scientifically rigorous permaculture experimentation really exists yet. By this, I don’t mean that there are no experiments being undertaken which meet my seven criteria, because there clearly are. I mean that we haven’t begun to build a systematic body of experimental evidence that meets these criteria. Nor has the permaculture community created a vision for how that body of evidence could be created and disseminated.

Permaculture is now 40 years old, and it is time for the movement to step up to this challenge of producing scientific experimentation that can support the claims it makes. A key part of my job, as Research Coordinator of the Permaculture Association, is to make sure that happens. Having set out the case for scientifically rigorous permaculture experimentation, in the second half of the article I would like to describe some of the ways my team and I have started to develop that over the last couple of years.

The lack of money available for research, coupled with the enthusiasm and commitment of so many permaculture practitioners, means that citizen science is a key tool for permaculture research. Citizen science approaches have recently become fashionable in the world of ecology, where volunteer networks are used to gather data such as the national populations of certain species of bats, birds or butterflies. This approach has been taken further by Garden Organic, the largest organic gardening organisation in the UK, which for many years has been running trials with its many members on soils, growing techniques and plant varieties. The Permaculture Association wants to take this yet another step, by actually training up a range of permaculture practitioners in basic research techniques. These citizen scientists could take part in experiments that the Association devises, but they could also design and conduct their own experiments. With this in mind, we have produced The Permaculture Research Handbook, a 60 page introductory guide intended to conducting your own experiments (the Handbook is free to download from here. While these citizen science experiments cannot hope to be as rigorous as those conducted by trained scientists with access to laboratories, big budgets and years of training, we believe that the volume and range of citizen science experiments in permaculture can more than make up for this. The Research Handbook stresses the need to conduct experiments which are reliable, valid and generalisable, and stresses the importance of clearly designed experiments which can produce good quality data. Over the last couple of years, we have also designed a training programme for potential researchers, focussed on a couple of residential weekends, but so far we have proved unsuccessful in attracting funding for this programme.

As well as training people in how to conduct permaculture experiments, we have worked hard to include practitioners in our own experimental research. Since 2011, the Association has conducted experiments looking at the yields of polyculture vegetable growing, assessing soil quality on permaculture growing sites, measuring crop yields from permaculture gardens, looking at small scale wheat growing techniques and mapping where soya can be grown in the UK. Some of these trials have been suggested by members themselves, others have been part of our core research strategy. Experimental trials like these take a fair amount of work to set up, mostly at the design stage, and they rely on the good will of individual members who invest their time, energy and possibly even money in the trial. However they can be run on a shoe string budget and have a wide geographical spread, vital in a country with highly varied soil and climate like the UK. As yet we have only published results from one trial (download pdf) but we hope to increase this soon as more results come in.

The biggest experimental project the Association has been running is the Soil, Biodiversity and Yields Project led by Tom Kemeny. Tom writes about the project in another article in this issue, so I will only describe it briefly. The project is attempting to generate the data to show that permaculture actually does what its supporters claim; produce healthy soils, high levels of biodiversity, and good crop yields. The aim is to create a series of simple tests that can be carried out by permaculture farmers and gardeners with no scientific expertise and no specialised equipment. So far, we have successfully developed soil tests, and we are now working on biodiversity. Next year we hope to expand the project to include economic well-being, physical and mental health and community building. Developing the tests is only the first stage of this project; they will need to be done by hundreds of permaculture practitioners around the world if we are to build up a big enough body of data to show that permaculture does what it claims. Once ready, the tests will be freely available on the Association’s website for permaculture practitioners and teachers to use.

In 2009 the Association began a ten year Forest Garden Research project. This project provided start up grants to eleven forest garden around the UK, and since then we have been following their progress closely. Ten of the gardens remain active participants in the project. Last year, we focussed on identifying the key research goals for this project through to 2019. We are still drawing conclusions from our detailed surveys and site visits to the gardens, but it is already clear that we will be able to draw invaluable lessons from their experiences to share with future forest gardeners. In 2012 Permanent Publications, the UK’s largest publisher of permaculture books and magazines, commissioned Tomas Remiarz to write a book on the current state of forest gardens around the world, and the Association is working closely with him to ensure the two research projects complement rather than compete with each other. Tomas writes about his project in another article in this issue.

In 2011 Anna Schmidt joined the Association on a year’s voluntary placement from Bradford University. She quickly identified that a lot of people round the world seemed to be involved in permaculture experimentation but that they were unknown to each other and had no way of making contact with one another. She therefore conducted the first International Permaculture Research Survey, through which she identified over 250 individuals in 50 countries who were engaged in permaculture research. Anna’s pioneering work was followed by three more surveys conducted by Cat Richards, which deepened our understanding of what work is being done and how researchers would like to be connected; see her article elsewhere in this issue for a fuller account of the survey findings. Out of this process has emerged the Permaculture International Research Network (PIRN), which has just been officially launched. PIRN will be a loose network of researchers across the globe, who will be kept in touch by an e-bulletin, website and social media, and who will conduct joint international research projects from time to time. The first research project will be on soil quality on permaculture growing sites, followed by biodiversity. This is only the beginning and we need input from permaculture researchers all over the world in shaping what we hope will be a genuinely international collaboration. Soon we hope that PIRN members will identify their own research interests and come together in interest groups based on geography and research topic. The International Permaculture Convergence will be in the UK in September 2015, and this gives us a great opportunity to strengthen the network and hold its first conference. So far, the development of PIRN has been generously funded by a donation from Lush Cosmetics, but it will need more funding as we move into 2015 and beyond.

A key part of IPC UK will be a permaculture conference. Plans for this are currently being drawn up, and key note speakers identified. It is hoped as many as 500 permaculture academics and practitioners will be in attendance, alongside many others from NGOS, green enterprises and other sectors. The theme of the conference will be ‘Designing the World We Want’. A range of papers and discussion groups will be delivered, and the conference will be a real opportunity to show case the best work being done in permaculture and in associated fields of agro-ecology and the social sciences. Not all the sessions will focus on experimental work, but hopefully a considerable number will show how well designed experiments can support the evolution of permaculture practice, and assess the positive impact it is making around the world.

Finally, it feels like it is now time for permaculture to gain more traction in academia. This is controversial because permaculture was born out of the conscious rejection of academia by Bill Mollison and David Holmgren. However, with the development of agro-ecology, complexity science and social psychology as academic disciplines, and the emergence of environmental issues into the academic mainstream, it is time to reconsider this position. The international research surveys have already identified over 100 university academics doing some kind of permaculture research, so permaculture is already in academia. However, so far none of these have permaculture named in their job title, and very few are teaching or researching permaculture as their main role. There is also a shortage of graduate students; although there is a steady trickle of Masters students researching aspects of permaculture, I am only aware of two PhD candidates looking at permaculture as their main topic; I hope there are more! A vital next step for permaculture experimentation is its mainstreaming in academia. This is in itself an experiment and a challenge. Exactly what form academic permaculture will take is not yet clear, and it needs to remain true to its roots as an activist movement outside of the mainstream. It may take different forms in different universities and countries, and even within the same university. The Permaculture Association is just beginning discussions about a permaculture Masters course; the first of its kind in the UK and perhaps the world.

But making a bold statement such as this one makes me think that perhaps there is already a permaculture Masters programme running at a university near you, maybe there are doctors of permaculture I have never heard of, maybe there is fantastic experimental work being done that is already answering some of the key questions that I want to see addressed. This brings me to my final point; how little coordination there is between permaculture researchers and experimenters around the world, how little is published on paper or online, and how there is not yet a plan to deal with this situation. Indeed it is not at all clear whose job it might be to coordinate and promote permaculture experimentation (and the resulting knowledge) internationally. The primary purpose of the Permaculture Association is to promote and support permaculture in the UK. However, we have identified that a key gap in current permaculture knowledge is a shortage of robust, experiment based evidence for what permaculture practitioners do and the claims they make, and we wish to join with all those round the world who share the aim of bridging that gap. We see the development of the PIRN, initially focussed on the 2015 International Permaculture Convergence in the UK, as a key opportunity to do that. So we have embarked on the biggest experiment of all; challenging the permaculture community to see itself as an experimental subject, generating scientific knowledge in a new way, where enthusiasm replaces big budgets, where thousands of experimental farms, gardens and communities replace laboratories, and where citizen scientists work seamlessly with academic experts. Ed Sears has described this whole process as ‘a social learning organisation devising a research capability in order to study itself’.

Last year, Ed Sears, Tomas Remiarz, Rafter Sass Ferguson and I attempted an assessment of permaculture research around the globe*. We established two things; there was a great deal of activity that we were able to discover and map, and a great deal more that we weren’t. It seems that so much permaculture experimentation is invisible, but it is surely time to make it visible. I hope that publishing this article will lead to a lot of new contacts and information on the range of permaculture experimentation currently being conducted both inside and outside academia, and inspire some new experiments too. Please get in touch with me if you’d like to be part of this exciting journey which is only just beginning (research@permaculture.org.uk). And above all, share the results of your experiments with the permaculture community!

The Permaculture Association is doing vital work which will make a significant contribution to the future growth of permaculture around the world. However, funding for the Research Coordinator’s post currently ends at Christmas 2014. This leaves a big gap running up to the International Permaculture Convergence in September 2015. If you can help fill this gap by donating towards our Research Coordinator’s salary, please visit this page. And if you know of trusts or organisations that may fund this work, please get in touch research@permaculture.org.uk

*A Social Learning Organisation Devises a Research Capability in Order to Study Itself, presented at the Radical Emissions Reduction Conference, The Royal Society, London December 2013 .