BrainScrum

Note: this post is longer than expected and is mainly about introducing and summarising the ideas that led me to wonder if there is a way to use the skills developed by using retrospectives (reflection on action) as a way to experiment while an event is still occurring (reflection in action).

Thanks to the work done by Norm Kerth retrospectives - a ritual held at the end of a project to learn from the experience and to plan changes for the next effort - are spreading nowadays in many different contexts and certainly in every Agile project that is really Agile. In time different types of retrospectives have been developed like project, heartbeat, alignment and incident (more info in the article Agile Project Retrospectives by Tim Mackinnon).

The first week of April I've been lucky enough to attend the Retrospective Facilitator Gathering 2006 in Germany and as often happens with events of this type those 5 days have helped me in reorganising some scattered thoughts I have on the subject and after more than a month I'm able to (start) write them down.

First of all why do I value retrospectives? Many different reasons but mainly because they let's you:

• learn from experience
• bridge the gap between theory and practice
• cope with ambiguity and change
• develop a critical awareness

Reflecting is something that we all do and Piaget found that this is one of the advanced skills adolescents develop as they approach adulthood. In fact I like to think that when professionals/teams/organisations move towards their adulthood they start to develop, value and apply a reflective approach to improve and grow. Or, to put it in another way, if they don't do it they still haven't turn the tipping point of maturity.

Learn from experience

Reflecting on experience provides the basis for forming more abstract ideas which can then be applied and tested in further experience. It enables us recognise new opportunities and learn from them.

Bridge the gap between theory and practice

As everyone has experienced in his life studying something doesn't necessarily means being able to apply it. We need to know how to apply the knowledge to real problems. Reflecting help us to identify how best to apply in practice what we know in theory.

Cope with ambiguity and change

Reflecting is essential in recognising the uncertainty we face in the modern world where the demand for fast and reliable results often increases stress and workload. This is particularly true in an Agile environment in which we tend to work closely with our customers and often have to cope with new and unique problems never met before.

Develop a critical awareness

We usually reflect both on our own behaviour, the behaviour of others and on the social/organisational context. This leads us to develop self-awareness and to become aware of things we need to change. Reflecting is then a key point for improvement and is far from being an innocuous process because it challenges the status quo, the way things are done.

Donald Schön began his research on reflection to challenge the predominant technical rationality which suggests that problems can be solved following a set of rules learnt during education. The result is a huge stress because we are seen as experts only because 'we are presumed to know, and must claim to do so' regardless our own uncertainty.

In reality the problems we face are often composed by a unique set of circumstances and the result is that the uncertainties as well as our knowledge of the other people involved are both relevant to solving the problems. Feelings, intuitions and critical thinking therefore play an active role and cannot be reduced to a set of rules learnt during formal education.

When we critically analyse our work we do it through reflection and this is when, in my experience, the tension between assimilation and accomodation happen.

In fact we may reflect in action (during an activity) or on action (after the event has been completed). In his book The Reflective Practitioner Schön analises how reflection is used by professionals to tackle the problems they face in their work and concludes that when faced with a new problem they become 'researchers in the practice context' conducting real world experiments.

He calls this approach reflection in action because the behaviour is an experiment itself. The general pattern recalls the scientific enquiry in which the professional experiences feelings and maybe anxiety that something is wrong, something is new and unique, never encountered before. A critical reflection then happens and he is ready to challenge his assumptions. At the end he may reframe the situation and come up with a new hypothesis and test it out.

It is also possible to develop the reflection skill after an event and Schön calls this reflection on action. This process is usually facilitated by another person. Retrospectives are then reflection on action and I wonder how we can use them to develop the reflective skill up to the point in which we can move from reflection on action to reflection in action.

Pure knowledge doesn't exist. I mean, there is not such a thing as an accurate and objective portrayal of the world unbiased by the beliefs, values and interests of those who provide it. This idea of pure knowledge stems from the early mechanicist view of the world in which everything can be explained with a set of objective phenomena behaving to a set of rules which can be readily observed and described.

In fact beliefs, values and interests of those who generate knowledge colour the knowledge itself. And this is true not only for the easily spottable cases like advertising or propaganda but also for knowledge that comes from a rigorous scientific experiment because it is created within a certain view of the world.

Thomas Kuhn called these views of the world paradigms and we all know that many different paradigms existed before, exist now and more will come in the future.

Different paradigms exist not only at the area of knowledge level (e.g. medicine: Western and Chinese) but also at the people and role level within a chosen paradigm (e.g. in a Western medicine hospital doctors and nurses operate within different paradigms).

The paradigm idea can be illustrated comparing two contrasting views of what knowledge is: Positivism and Constructivism.

In the Positivism paradigm knowledge exists in an objective sense, it's a complex pattern of facts and ideas and learning is seeking and absorbing this knowledge. Scientific knowledge usually reflects this paradigm because, as Jurgen Habermas defined it, it's coloured by the technical interest to control the world.

In the Constructivism paradigm knowledge is not just objective, it is coloured by our own meanings and feelings and therefore everyone's knowledge is different. This is the knowledge we need to understand human relationships that Habermas defined as practical interest.

Of course Habermas ideas reflect his own paradigm…

All this lecture leads us my main point: team dynamics and performance. A team to work efficiently needs to have shared assumptions operating at an inconscious level so that its members don't have to constantly stop and determine what they believe and how they should act. The beauty of assumptions is they become a shorthand for the team which, when faced with similar situations, just acts and doesn't re-evaluate each time.

On the other side though the team may fail to see new data that contradicts its beliefs and may miss an opportunity to improve its effectiveness. In fact missing details may false an experiment even if the results make sense because they fit our frame.

Piaget suggested that when faced with a new situation we have two options:

• Assimilation - happens when we handle a new situation by applying an existing schema. We make the new situation fit with our existing paradigm

• Accommodation - takes place when we make adjustments to our routine schema to cope with something new. In these cases our existing paradigm is modified to fit in with the new situation.

There is obviously a tension between assimilation and accommodation - between using old responses in new situations and acquiring new ones (or updating old ones) to cope with change. This is a continuous process and the result is a continuous learning.

Our team is a nice example: since the beginning of the project we have gone through many cycles of assimilation and accommodation trying, inspecting and adapting many aspect of our way of working: to date we have changed the duration of the iterations, the way we pair (2 or 3 times), the story lifecycle, the way we do the planning game, how different people interact and even the way we move the stories on the cardwall (plus many more details). We are now going to change the day and the way we do the showcase before the planning.

And everytime there has been a tension between our assumptions - as a team and as individuals based on experiences in previous projects - and the feedback - both as data collected and people feelings (retrospective but not only).

Paradigms, beliefs, values, assimilation and accommodation heavily affect the relationship between traditional and Agile people but this is, maybe, a topic for another post.

I've always disliked production line metaphors in software development (and I hated also the building construction one but only until before I discovered that in reality building construction is much more Agile than people know of). I don't like them because they assume people work in a row with work pushed towards them, usually at high speed, and they must push it, again fast, on to the next person down the conveyor belt.

This way of working doesn't encourage learning, mainly because problems/mistakes/errors are discovered during quality inspection that happens towards the end of the line - exactly when it's too late to make changes.

U-shaped or parallel production lines, like in many Japanese firms, create instead a natural team and make it easier for people to move from one job to another (and help each other). People walk to get work when they need it. Pulling work to them any error can be spotted early by the next person along. And even better they can try to solve it together.

Learning takes place constantly because problems and mistakes themselves provide learning opportunities. You can even stop the line and sort things out rather than letting them throught.

This is a problem I see threatening also us during our daily job: because we do (very)big projects, with lots of people and very fast a tempting, partially unconscious and naturally reassuring tendency towards a production line like:

BAs write stories -> QAs write tests -> Devs write code

might slowly take over.

As a general way of working, if shared by the team and naturally emerged and self-assessed, I found it both practical and reasonable. In fact I think the problem is underneath the surface where the values and principles lie. Entry and exit criterias between the stages of such a story lifecycle are useful to state in a binary way if a story is ready to be played or not (can it be 75% ready? is there something like a Running Tested Feature at the story level? Ready Tested Story anyone? does it really matter or even make sense when there is a team in the real sense of the word?).

If these criterias get in the way of communication and collaboration they slowly become a way to blame others instead of taking action: "it's not my fault, I'm waiting for the BAs to finish this story" or "I don't have anything to do because the QAs are having problems in writing some FIT tests").

Keeping in mind why we work in an Agile way, the core values and principles (not the practices!) that lead us towards this way of behaving we can avoid to fall in that easy, effortless state of mind in which entry & exit criterias are more important than working together. Retrospecting every now and then helps a lot, that's why I like retrospectives!

A friend of mine likes to say: "events are neutral, it’s our mood, our mind status in that particular moment that attaches the judgment to an event", I'd like to paraphrase and say "criterias are neutral, it's our way of using them that affects the team's ability to perform"

Recently some friends attended a ScrumMaster Certification class held by my friend Joseph Pelrine here in London. Considering they are all very experienced and very eXtreme/Agile (some of them were using eXtreme Programming when others were thinking “let’s try this new thing called waterfall…”) this evening at XtC I asked them if, even with their extensive Agile experience, that course had left them at least one interesting thought. They all said the self-organisation idea.

The strange thing is that I’ve always believed that this idea is common to every Agile approach but I recognise that my point of view is somehow contaminated because by nature I tend to assimilate ideas and put them together without really keeping in mind the exact original source…

By the way, the brief discussion that followed started with a phrase like “I like the idea but I’m a control freak, I cannot really leave the team in anarchy”. I don’t see self-organisation as anarchy even if I agree that different situations (and different people) need different degrees of independency.

Briefly, my view on self-organising team is this:

• the iteration is the shield that isolates the team and gives it the time and space needed to inspect and adapt
• the ScrumMaster (or whatever you call it, you can replace Scrum with every Agile approach) is the one that works hard to keep the noise out of the iteration shield
• the ScrumMaster is also responsible for making sure that the team follows Scrum - it’s more about the attitude, values and principles than strict practices and formal process. Do you remember? It’s the art of the possible

Point number 3 is where the coaching-mentoring skills matter instead of a command-and-control mentality: you do the right questions instead of imposing your answers, you help the team review what they are doing (and what they did - retrospective) in order to facilitate little steering every now and then.

I like the image of a set of little, coloured spheres in a vibrating container. If the vibration is constant they tend to find an equilibrium. If you need them to find another configuration you need to apply a little force to break the current formation and let them find a new configuration.

I see the same thing in both user stories and refactoring

User Stories (and Planning Game)

When you work on your user stories, splitting them so that they are INVEST (Independent, Negotiable, Valuable to users/customers, Estimatable, Small, Testable), your goal is to transform something that represent value for the system’s actors into something that represent value to be delivered to the customer. Splitting the stories the business system is separated into simpler compounds of utility more communicative, atomic, orthogonal and cohered (and the problem’s complexity is reduced as well). You then put all these cards on a table and compare them, remove duplicates and group similarities….. Isn’t this an emergent, self-organising configuration facilitated by little forces applied by us?

Refactoring

Our goal is to reduce the marginal complexity so that introducing a new feature/a change is cheap. We need the code to communicate clearly and in a simple way its design intentions and we need the abstractions to emerge. Abstractions are not done by construction, instead you put the behaviours (the code) on the screen, compare them, remove duplicates and group similarities….. Isn’t this an emergent, self-organising configuration facilitated by little forces applied by us?

End…?

A system in equilibrium naturally tends to resist to change and that’s why we need to apply a force (questions/retrospectives/etc for people, splitting for stories, refactoring for code), because we need to take the system at the edge of chaos to facilitate the emergence of a new, spontaneous configuration.

Some days ago I was re-reading the inspiring book The Tipping Point by Malcolm Gladwell and like the first time I read it a lot of thoughts arose in my mind. If you don’t know anything about the Tipping Point - a dramatic moment when something unique becomes suddenly common - Google is your friend (reading the book is better though).

Gladwell identifies three driving forces that make social and marketing phenomena and trends happen:

• contagiousness
• little causes can have big effects
• change happens not gradually but at one dramatic moment

I see some interesting relationship with my experience even if I’m missing something and the result is two directly contrasting propositions I would like to delve into.

First of all the last concept reminds me the scientific revolution ideas of Thomas Kuhn where science is not a steady, cumulative acquisition of knowledge. Instead, science is “a series of peaceful interludes punctuated by intellectually violent revolutions” [Nicholas Wade, writing for Science], which he described as “the tradition-shattering complements to the tradition-bound activity of normal science.” After such revolutions, “one conceptual world view is replaced by another”.

Then the idea of contagiousness as a geometric/epidemic progression versus a more simple proportional one. Gladwell writes:

“As human beings we have a hard time with geometric progression, because the end result - the effect - seems far out of proportion to the cause. To appreciate the power of epidemics, we have to abandon this expectation about proportionality. We need to prepare ourselves for the possibility that sometimes big changes follow from small events, and that sometimes these changes can happen very quickly”.

Little causes can have big effects immediatly reminds me a lot of things like Stephen Wolfram work on cellular automata, his A New Kind Of Science in particular but also Chaos Theory and Complex Adaptive Systems, the butterfly effect and so forth.

And now I’m ready for my absolutely non-structured thought burp (mainly related to the Agile software development but not only):

little less than enough, at the edge of chaos might mean just before the tipping point (?)

Emergent design (tdd + refactoring) can be/is a way to reach the tipping point without the need to forecast the tipping point itself or a way to avoid a “bad” tipping point keeping the code simple?

Little causes can have big effects is reflected by the fact that refactoring (which is a series of small trasformations by definition) can lead to very lean, mean, clever solutions. But can doesn’t mean that it will. There are other forces then and that’s why an Agile approach stresses the fact that the practices reinforce each other.

BDUF is a way to try to avoid any non planned tipping point (not being planned means automatically it is bad): from this point of view BDUF and requirements freezing (resist to change) make sense (!). The problem is that BDUF doesn’t take into consideration the fact that “little causes can have big effects”. Or better: it recognises it, and that’s why it attempts to forecast and plan everything in details but it misses the fact that even a very little and apparently not important change might have a geometric progression. This is because a mechanistic approach believes that a system/organisation is exactly the sum of it’s components.

An holistic/hermeneutic view considers an organisation from a systemic point of view (the whole is more that the simple sum of the components, self-organisation, etc, etc). This is directly connected IMHO with Maturana and Varela Autopoiesis theory. In their book Autopoiesis and Cognition: The Realization of the Living Humberto Maturana and Francisco Varela explain the autopoietic concept, a word coined in the 1972 from the greek autos (”itself”) and poièsis (”creation, production”): the creation of itself.

The authors first of all define the autopoietic machine concept as a homeostatic system to demonstrate how a living being it’s, at the end, a living autopoietic machine.

Now I think you understand what I mean when I say scattered ideas