Archive for Java
11.01.09
Posted in General, Java at 8:13 am by Jon Pither
I recently wrote a white paper and it has been added to the ThoughtWorks collection of white papers here: http://www.thoughtworks.co.uk/what-we-say/white-papers.html.
After having been involved in a few projects whereupon the client has a need to modernise an ageing technology stack, I thought I’d pull together from these experiences (and from those of some helpful colleagues) and do a write-up.
Here’s the abstract:
“Gartner makes the prediction that by the end of 2010 ‘more than one-third of all application projects will be driven by the need to deal with technology or skills obsolescence’. The process of refreshing technology platforms – Modernisation – is one that is fraught with risk. From spiralling complexity to software development teams losing business focus, there is ample opportunity for a modernisation agenda to fail.
This paper has been written for the non-technically minded business and information stakeholder without prior knowledge of Agile. It looks at using principles and practices originating from the Agile philosophy for mitigating the modernisation process.”
The actual PDF can be downloaded here: http://www.thoughtworks.co.uk/pdfs/agile-modernisation.pdf
Thanks to Liv Wild and Simon Brunning for peer reviewing, and to Davie Moston and Gaurav Sood for giving me feedback on its various incarnations and iterations.
Permalink
07.28.09
Posted in General, Java, Uncategorized at 4:40 am by Jon Pither
This is a post about the practice of risk management through use of a ‘DEFCON Door’.
A DEFCON door? Well if you’ve seen that early 80′s film ‘WarGames’ starring Matthew Broderick, or if you simply know your US history, then you’d know that DEFCONs – Defense Readiness Conditions – are used to be a “measure of the activation and readiness level of the US armed forces” (as described on wikipedia). During the Cuban missile crisis DEFCON 2 was reached, with the US airforce readied up to go and bomb their cold-war rival the USSR.
And how might this be applicable to your typical day-to-day IT project? For starters, once on a project what we did was to find a spare door and to configure it as such:
It’s very simple really. Along the top column headers are the various DEFCONs. DEFCON 1 would mean that it is the consensus of the team that the project is going to fail because of a particular issue; that a very serious risk has indeed materialized. DEFCON 2 risks are still very serious, and DEFCON 3 ones are merely lesser risks that the team feel are important enough to warrant exposure at a particular given time.
What I found most pleasing about the DEFCON door was that it seemed to be a fairly natural extension of the Agile software development practice and was therefore well-adopted by individuals on the project. For instance in a similar way to how we do story estimation a person would read out a ‘risk card’ and then the team would count to three and display with a finger count the the DECON level that they thought the risk had reached. No fingers at all meant that the risk could simply drop off the door altogether – which ought to be a good thing.
Normally every couple of days or so after the standup we would do a quick review of the door. Sometimes, we also did it after a retrospective, because a few things that had been brought up were indeed risks that the team thought were ‘DEFCON appropriate’.
Risks would sometimes traverse up and down through the DEFCONs as the team would attempt to grapple with them, and often risks would be broken up into multiple risks, or instead merged together to form all encompassing ones. No one team member should own the door as it works very well as a collaborative tool where everyone can have input.
Thoughts on effectiveness:
I think often individuals within a team will have very real concerns about some particulars of a project. Personally speaking, one of the wants that I have when I have such concerns is that my concerns will be listened to and registered by the team, and I think the DEFCON door to an extent gives you that.
It also brings the team closer to the risk tracking that would ordinarily be done by project-managers. Although I don’t think it should be a replacement as it best works as a lightweight tool, there’s real value in having team-members contributing to tackling and identifying risks on a day-in, day-out basis.
The DEFCON door also has uses outside the immediate vicinity of the team, such as the ability to show project stakeholders the door and say “these are the major risks that the team feels are present right now”. Often stakeholders not operating within the team will have a want to gauge a collective opinion about how the project is going and the challenges that are being faced. Here the door can play a useful role.
Potential Challenges:
Firstly, it sometimes can be difficult to locate a usable door, especially a nice wooden one. If need be you could use a wall, but then you’d be harming the linguistically pleasing flow of the “DEFCON Door”. It is however, really up to you.
Understanding what the scope is of the door can be at times slightly difficult. For example it can be tempting at times to throw up enterprise-wise risks up on the door that are not directly project-related (e.g. is this architecture appropriate for adoption by the wider organization?). A strategy a colleague of mine came up with is to have a DEFCON corridor, but I fear this may be over-egging the solution. I would endeavor to keep the door simplistic, and if the door identifies risks and questions to be managed elsewhere then that can only be a good thing.
We should also note that this model of risk management is very lightweight and simple, and that there is much more about risks than can easily tracked and measured on the door. Therefore it would seem appropriate to use it to augment a more sophisticated risk management process.
In conclusion, the DEFCON door is a simple tool that aims to bring risks explicitly into the highly visible and participatory realm of an empowered, self-organising Agile team.
Permalink
06.13.09
Posted in General, Java at 8:13 pm by Jon Pither
What’s it all about? It’s easy to communicate without violence isn’t it? One would have thought that the majority of us would be able to convey a linguistic subtlety without having the need to head-butt the other person in the face (unlike some of my neighbours, but then that’s a different story 
).
NVC is about communicating without getting people’s backs up; their heckles raised. It’s the ability to enter into a discourse with compassion and empathy, and to uncover what the real wants are that people have during a conversation, including those that we have for ourselves. And while it may seem like it comes naturally to some, it’s actually very difficult for the vast majority of us. In society we’re conditioned to think in certain ways, for example to judge and to offer our immediate evaluations; we are unconscious of some of the blockers that we have towards effective communication. This results in an inherent violence that stirs in each of us.
So what specifically is NVC? Thankfully it’s actually quite simple. It’s a four-step process. I’d ideally advise people to pick up a book on it, as it’s one of those things you really have to take on board and to heart, rather than to simply acquire an intellectual grasp of the concepts.
1: Separate observation from evaluation
It’s habitual for many of us to combine observations and evalutions. For example, if my wife tells me that I always leave the place looking messy, then I’m bound to feel defensive about that. “No, I don’t”, I’d probably say, “I actually spent a couple of hours doing some cleaning at the weekend”. I’d feel attacked, and bad mood for both myself and my wife would probably ensue.
But if she were to state that there were a couple of socks on the floor in the bedroom, then I probably wouldn’t really have much of a feeling about the situation at all. I’d likely instead develop a want to pick up the innocuous pair of socks, and I might just darn-it go and do it.
Confronted with an evaluation, we typically either defend ourselves or submit. Confronted with an observation, and it’s a different story. According to the Indian philosopher J. Krishnamurti, the ability to separate out evaluation from observation is the highest form of human intelligence. Does it always rain in England? Or did it simply piss it down last year at Wimbledon? (2007 rings a bell)
2: Feelings
When you communicate, state your feelings. I feel X about Y. I have to admit, I’ve found the acquisition of this particular skill to be particularly difficult. Why? Because for a great many of us we haven’t really delved into the workings of our emotions. I don’t always know if I’m feeling mostly angry, mostly frustrated, irritated or even aggressive (life on the London tube does that for you). Like when I taste a glass of wine, what’s actually at play here? Is it the vintage, the oak in the barrel that it was fermented in, the field it came from, or the blend of grapes? There’s a lot going on in people’s minds that it can become difficult for us to categorise our various emotions and to label them.
People can spend a lifetime in the pursuit of unpicking their various emotions (Buddhist scripture states that there are eighty four thousand negative emotions alone). In the west we’re starting to label the ability to recognise one’s emotions as Emotional Intelligence (EQ). People also use the term ‘awareness’; e.g. are we aware of ourselves?
So communicating a feeling requires an awareness of the feeling in the first place. And it can be remarkably powerful. If I can be aware, open and honest to state what I’m feeling, then I invite empathy. If my wife were to say to me “I’m feeling a bit frustrated with the state of upstairs”, rather than “you’re pissing me off leaving these socks around”, then I’d probably have a connection with that. I’d understand and gain some empathy for the emotion she is experiencing, and I’d likely develop a want to do something about it.
As I said it’s difficult. At work one time someone asked me why I seemed a little down, in contrary to how I was when I first joined the project when I was more upbeat and positive. Rather than state my feelings of frustration at my apparent inability to influence some positive change fast enough, I defaulted to bravado. What me? Nah, I’m cool…. honest. I’m a bad-ass ninja-type character. Seeing as the person asking me about my feelings was an influential person on the client-side (I’m a consultant), it was an opportunity missed.
3: Needs
I really like this part of NVC. Like the all 4 components of the methodology, it’s an independent skill that one can muster (and is therefore in-turn measurable by the lovable Dreyfuss model of skill acquisition). I wanted to call this skill “needs based thinking” as I think it’s so powerful.
If my wife were to say to me “I’m feeling frustrated as upstairs has some clothes on the floor, and I have a need for the place to be tidy this weekend so that we can enjoy it”, then we’re really on the money. I know some more of what her needs are, and I therefore know better how to meet them.
Frustration, some people say, is the feeling one gets when their needs aren’t met. A big personal realisation for me is that I’m not totally aware of all my needs. I didn’t quite appreciate my need for engaging with others, having challenging intellectual skirmishes, my needs for adventure, for approbation… (goes without saying that not all needs are admirable). I, like others, am a complex person with complex needs. If I can identify them better, them I’m sure to have a happier life right? Conversely, one should ignore their needs at their peril.
It follows therefore, that if I can communicate them, then there’s a much greater chance of having those needs met. I’ve found that when I’m honest about my needs to others (without wrapping them in bravado or dressing them up likewise), then I tend to get some compassion back.
4: Making a request
So, now I know that my wife is feeling frustrated because there are some clothing items on the floor in the house upstairs, and that she has a need for the place to be tidy so that she enjoy the weekend. Great. But what I really want to know is what the next steps are. Thankfully in this situation I can infer that the action item should be to get off my arse and to pick up my clothes.
But it’s not always this clear. I can reveal my observations, feelings and needs to others, but without solid requests we can’t be sure that the other person knows what to do to make a situation better. I’m anxious that we have more preparation to do for this presentation and I need to get a handle on it… any chance we can hook up tomorrow evening to go over some things?
In Conclusion
NVC is certainly challenging to embrace, and of course there’s a fair bit more to it that the quick primer I’ve just given. I myself am probably on the journey to becoming more aware of it’s impact, and am hopefully a little more “consciously incompetent” than “unconsciously incompetent”. I’m a novice, but it’s a communication technique that I really, really want to learn. If you’re interested, I’d recommend that a good place to start is by picking up this book:
Permalink
03.31.09
Posted in General, Java at 1:22 pm by Jon Pither
“Self Organising” teams are somewhat of an elusive goal for Agile/XP projects, in the sense that the application of theory to practice can be far from intuitive, particularly for those coming from a traditionalist software management background. Despite this, having a team with the ability to ‘self-organise’ can make all the difference on a project, and not only in terms of productivity but also for purposes of satisfaction for those working within the team. Having seen some teams succeed and fail in this area, it’s a concept that I find especially interesting.
So, what precisely is a self-organising team? In the context of IT at least, we should be able to essentially understand that a self-organising team is a team that has the freedom and ability to shape itself accordingly to meet the varying challenges of software development (and of-course actual production).
Shapes itself how though? Well, to start with on an Agile/XP project we can look at some actual specifics, such as a team being able to choose the order of the software requirements to be developed (stories), and of whom exactly on the team gets to work on what particular task. Furthermore we can easily imagine that a self-organising team is one that decides when and how it should grow as to meet the demands placed upon it, and of projecting what sort of timescale to accomplish those demands is achievable given the current resources at hand.
So therefore it’s likely that we can all have a vaguish but regularly acceptable sense of what a self-organising team is. I would stipulate though that one of the problems with this term is that it’s frequently used too liberally, and can be quickly become quite a nebulous misunderstood concept associated with the best of consultancy-type babble. What I’d like to do here is to cut through some of the fuzziness to get a better understanding, and I’ll do that by initially focussing on what in my opinion a self-organising team is certainly not.
So what is not a self-organising team? I wouldn’t say that the converse means that a team that is necessarily un-organised. Just because a team isn’t self-organised doesn’t mean that members shouldn’t be able to turn up to work on time or to keep the build green. Instead, I’d say that the direct opposite of a self-organised team is one that would be entirely managed; the ‘managed team’.
In the scenario of software (a depressingly common end of the spectrum), the picture would be of a team that is not really a team but a collection of micro-managed people. Developers would simply turn up to work, and would be told of what particular piece of functionality they should be working on, and of exactly whom they should be working with. Any real challenges, including those of a technical nature, would have been worked out in advance by key personnel. Therefore we can see that in this idealised traditional vision of the workplace (borrowing from Frederick Taylor 1856-1915), all that the developer really needs to do is to shoot into an ‘open goal’; that is to do his/hers assigned and ‘plotted out’ series of tasks.
Managing a team in this way can be an extremely difficult job. Not only is there huge pressure on the management team to make the software production process flow more quickly, but managers will quickly discover that there is a never-ending torrent of things that can be managed. Who should be in the right meeting? Should developers pair-program the whole time? Can we trust them not to? Why are these developers discussing a discussion that has been discussed before by other discussers? Why are these developers discussing at all? Decisions have been made, challenges supposedly put to bed ahead of time… Frustration beckons as the developer’s open goal is proving to be not that open.
On such an example of a project, developers would find their innovative capabilities stifled (as any innovation would be managed via some specific process of having just the relevant technical stakeholders present at a particular time), Instead, developers will simply plod on, and any potential deviation from the course (such as refactoring some technical debt) would be fed back to the project management team and…. managed. Quickly managers would find themselves not just managing the production of software requirements, but also having to deal with the extremely blurred area of ‘technical debt’ that some developers keep on talking about, and they would need to ensure the eradication of a whole ranges of bugs that range from the critical to the downright trivial (as in why-don’t-developers-just-fix-them-right-away type trivial).
There is also the underlying serious contention that morale will decline on such a project. And with morale, a sense of real commitment to the overall mission being completed successfully. An easily observable, steadfast symptom of this is quietness within a team, present in one that is managed as opposed to self-organised. Communication when not occurring organically, along with morale, has the tendency to fall of a cliff.
Which brings me on to talk about the other side of the coin; the self-organising, positive side. Having looked at what takes place in a ‘managed team’, we can perhaps postulate that the goals of communication and morale are actually emergent properties of a self-organising team. If you want people to go faster, i.e. developers to develop software at a faster pace, then let them self-organise. Let the team itself work out who is best placed to work on what, and of how it should be done. Empower the team to make choices, and to take responsibility.
Trust extended out to a team in this way can be a big ask, but one could make a straightforward case that what fundamentally makes or breaks the self-organising team is exactly the matter of trust.
And whilst trust from above is fundamental, there are other important pre-conditions that need to be satisfied in order for the self-organising team to prosper. A self-organising team will not simply just rise out of the ground of a group of people, rather like a seedling, it has to be nurtured, and it comes with its own specific needs.
Leaders
A team needs strong leaders. In the software team, leaders are required to set positive examples of how communication should be done (such as freely, openly and respectfully). Leaders are essential to generate and preserve room for the team to innovate and to grapple with issues of a technical nature head on. Leaders act as the buffer between the rest of the team and the greater organisation the team finds itself in; they are the damn holding back the water.
Experience
The best people to help make a team self-organise are those that have had experience working in one before. They will have context for what a team with good communication feels and acts like, and will encourage other team members to take ownership of problems and to take action. Like leaders, they are the ones to set an example. Having experienced, high calibre people on-board will help to assuage any managers’ concerns that the team is on-course, and they will in-turn be less likely to want to step in and take charge.
Encapsulation
As I wrote about in a previous post, we need to avoid ‘dev team leakage’. This is to say that the internal day-to-day activities of a team need to be kept in-house; inside the team. Against a contract of clearly defined inputs and outputs the team must have the space to get what it needs to get done in order to produce working software. In order for people to take ownership of problems and to deal with them quickly, we need to give developers the room to tackle impending issues without the need for tracking and reporting to the wider business.
Strong management
It takes a strong management team to sow the seeds of a good team, and then to stand back and watch it grow (and to be there providing help when needed). The team needs to be allowed some trust so that it can make some mistakes, as well as the space to be able to continually adjust to better meet the needs of the business. Managers will need a host of other skills too, such as strong people skills, and to be adept at thinking strategically over the long term versus the tactical short-term. Typically there will be those in the business wielding considerable influence who are strictly results-orientated, and who will want to see direct hands-on action leading to direct results. These stakeholders will need to be handled adeptly.
And of-course, the self-organising team is no silver bullet, and is no easy ride. Communication can quickly become robust, and it may appear to those looking in that situations are becoming over-heated. When passionate people are striving towards the same goal, that passion and drive to make things better will occasionally bubble to the surface.
Resources
The team needs to be of the right size. Judging the amount of people a team initially needs is crucial. Too many people can lead to a lack of communication, where there exist too many links in the system for there to be a shared vision and a collective ownership of problems. If teams are to be split, they need to be split as to be relatively self-contained. Teams that have integration points that are simply ill-defined or too numerous will be unable to generate self-organisation independently of each other.
The self-organising team is difficult to get right, and the devil really is in the detail. It takes hard work, a belief in others and above all a large amount of trust engendered from those responsible for putting the team together in the first place. The results, including that of heighted morale and communication makes the process of allowing a team to grow organically hugely beneficial. And not to mention that the in end, such a team will be far easier to manage.
Permalink
02.06.09
Posted in General, Java at 2:18 pm by Jon Pither
It’s a bit of an anti-pattern that I have observed and want to discuss. Firstly though, I want to cut straight through an analogy that I believe helps to describe the situation (curiously, I have received some feedback on my propensity for using extreme analogies from a superb linguist colleague of mine who later was inspired to write an article on the subject: http://siddiqui.saleem.googlepages.com/sublimeanalogies).
If we can see the dev team as an animal in the jungle (and so forth we have other ‘animals’ such as QA teams, BA teams, Project Managers etc), then ‘dev team leakage’ refers to the cutting open of such a large and volatile jungle beast and of letting it’s gooey entrails slide out and slither amongst the wider natural setting of which the poor animal finds itself.
Yep. Basically it’s about breaking encapsulation. In the case of a dev team, there are many inner workings that need not be externalised to the outside world.
Technical ‘to-dos’ are one such example. On a typical agile project, the technical team would have a series of technical tasks that are being raised and dealt with on a continuous basis. This is part and parcel of what it means to develop software. A smart dev will think of something that needs doing that is not directly in his/her line of fire, and will flag it up as a task for execution at a more convenient time. The anti-pattern I am discussing would occur here if this ‘task’ was picked up by the wider process, and managed like any other software requirement. Suddenly the inner-workings of the software development team are exposed and tracked.
In this scenario, the project will seem a lot ‘busier’ than it should be, with issues largely of a fine-grained technical nature polluting the wider process. Woods would become difficult to see through the trees, and the ‘bigger picture’ – that is the production of functionality the business really cares about – is obscured.
Additionally, we can see that important business stakeholders may be presented with new, heavily-detailed information that they will probably not understand or frankly care about. In this example, it is easy to see that some stakeholders could get disenfranchised pretty quickly, especially if they were asked to start prioritising what technical tasks they would like developed in a particular order. They may also start to see the stream of ‘techno-babble’ being directed at them as a sign that the wider team is not focussing on what really matters, and they’d most likely be right.
This is a singular example of dev team leakage. Others include the micro-management of how developers might ‘pair’, or of dev stand-ups being folded into a wider stand-up consisting of various differing teams (missing the opportunity stand-ups offer for devs to give each other a ‘heads-up’ about various technical issues). Furthermore, there could be no mechanism (such as focussed retrospectives) in place for the team to continually improve, as improvement is dictated from above.
In taking this anti-pattern to the extreme, one can envisage the real risk that the ‘team’ may not actually be a ‘team’, but a collection of people acting as cogs in a larger machine. All the really good and essential stuff we get from ‘tight’ dev teams, such as collective code ownership, and not to mention a real sense of purpose and morale, would be lost.
At this point, I must stress that in order to avoid ‘dev team leakage’ we are not required to treat the dev team solely as a black box. Whilst the team may want to shield the complexity of its inner workings from the encompassing project, it ought to be able to stand up to scrutiny, and to justify the behaviours it extols on the wider system (a case in point is of the team justifying why they need more time to focus on cleaning up the house first, before tackling some additional requirements).
Here the root-issue is transparency. There is a traditional tendency of software project management that places the requirement on development teams to ‘push’ out a whole host of data out to the wider process, that might otherwise be kept internal. In the ‘push’ model (as oppose to pulling out data as needed), there is the danger that such data may be used for tweaking purposes, and that micro-management will eventually start to sink in, leading to a ‘command and control’ style of project execution.
The development team is of paramount important to any software development project. It needs to be able to hold up its head with dignity and will require experienced leadership to actively safeguard its own integrity. Importantly also, is that there are people on the outside helping to nurture it, providing the team with the space and resources it needs to operate.
Permalink
01.18.09
Posted in General, Java at 3:21 pm by Jon Pither
It already sounds difficult doesn’t it?
A bureaucracy, as defined by Warren Bennis (1969) cannot cope with:
- Rapid and unpredictable change
- The increasing complexity of modern organisation
- Diversity of specialist experience required in many organisations
- Humanistic, participative management styles
But before we start weighing up if an organisation is a bureaucratic one, we ought to be able to envisage what’s over on the other side of the spectrum. Well, this then would be an Adhocracy:
Adhocracy is a type of organisation design with is temporary, adaptive, creative, in contrast with bureaucracy which tends to be relatively permanent, rule-driven and inflexible. Adhocracy is similar to the concepts of organic and integrative styles; bureaucracy equates with mechanistic and segmentalist approaches.
I would add that an adhocracy is an organisation that is adept at continually improving itself.
Adhocracy Vs Bureaucracy
As a software consultancy we’re often asked to ‘transform’ an organisation from using a set of traditional software practices to more modern ones that we know achieve better results. Taking up some of these practices has direct repercussions on the way that IT departments organise themselves. Having been involved now in a few of these ‘organisation transformation’ projects, I’ve become increasingly convinced that from the outset we really need to get to grips with knowing exactly what type of company it is that we’re dealing with.
For example, as consultants if we go into an idealistic adhocracy our ‘change objective’ is going to be a great deal easier. Largely this is due to the client being used to change anyway; in fact change is part of their way of life and is embraced as such. In this situation we can do what good consultants do best, which is to implement change via a participatory approach. We can engage with workers in the ‘trenches’, and the good practices that we introduce will naturally ‘bubble out’ in an organic way, delivering change across the organisation.
The reality is though that most companies will have some bureaucratic traits in them. In order to answer this, we need to examine the question of ‘how do we tackle organisations that are veering towards being bureaucratic?’
Some consultancies, particularly the one I work for, would wish to encourage a participatory, collaborative way of spreading change through an organisation. By working with employees on a day-to-day basis, we can foster relationships that engender trust, and pave the way for individuals to try out new ways of doing things.
However, we must acknowledge that in organisations veering towards bureaucracy – i.e. ones with virtual ‘walls’ in place that separate out segmentalist units – such ‘organic’ change isn’t going to spread very far. Rather, by working in conjunction with people in the upper layers of an organisation, we will have the ability to propagate change downwards, and to try and possibly move a company along towards being more adhocratic. We can factor this approach in along with the ‘organic’, ‘bottom-up’ one, and so long as we synchronise up our efforts we may get a little more bang for our buck.
Going further, some researchers – Dunphy and Stace (1990 and 1996) argue that incremental and collaborative or consultative modes of change implementation can be highly inappropriate. For the survival of an organisation, where rapid change is absolutely necessary, transformative approaches will be effective when carried out in a directive and coercive manner.
It makes a little bit of sense in the end, that in order to change a ‘heavy’ bureaucracy we have to first embrace what a bureaucracy actually is and to play to its strengths.
Needless to say, if we’re working closely with the upper echelons of an organisation to help plan out a ‘directive’ change strategy, we’ll need to have a robust, comprehensive plan in place. We’ll need to have drawn up a roadmap of actions based on the original business objectives for change, performed an analysis of the major stakeholders, worked out how we’re going engage with the various sub-sections in the company; it’s a big task. Simply placing consultants in varying roles throughout the organisation and letting them work their magic isn’t going to cut it. The approach needs to be cohesive and well communicated.
It sounds somewhat obvious, but it’s really worth stressing that we really need to know up front what type of company it is that we’re dealing with when we’re asked to implement strategic change. An adhocracy? Great. A bureaucracy? Hmm…
Permalink
01.02.09
Posted in General, Java at 6:14 pm by Jon Pither
There have been a number of times when I’ve sat down to code away at a software problem, and then after a while discovered that I’m getting quite frustrated in some way. If I’m honest I wondered if this was simply me being me, hypothesising that there’s a part of me that just sometimes gets irritated if I’m investing all my efforts to reach a specific goal, and I’m being hindered in some way. It may be that I just can’t seem to go fast enough, I simply can’t do it with the resources at hand (i.e. a deficiency in my own skill-set), or if I’m being disturbed too regularly.
But after spending the majority of the previous year coaching other devs, I’ve realised – thankfully – that I’m not alone in getting somewhat p*ssed off when my best laid plans do not come off in the way that I’d expected.
Why though? I know I’d be happier in my day job if I didn’t get into this arguably ‘destructive’ mindset. I’d be better able to reach that Zen like state of ‘flow’ where the mind is cohesively primed to tackle the problem at hand. I’d be less stressed for starters, and would simply have more fun when I’m doing what I’m doing for a living. Here in this post, I’m going to have a stab at discussing some commonplace frustrations I see in myself and others, and then look at what options – if any – we have for handling them.
Before I go any further, we do have to consider if it’s really that much of a bad thing that we do get frustrated at work. After all, it’s often desirable to be passionate about what we do. Surely the upside to frustration is that we are incentivised to remove whatever blockage is in front of us that’s causing frustration in the first place?
As to avoid getting out of my depth in philosophising what exactly frustration is, I therefore choose to refer to frustration in the terms of creating a destructive mind state, as oppose to a constructive one. This is drawing on a book I read by Daniel Goleman entitled ‘Destructive Emotions’. As he writes “With a destructive emotion, there will always be a gap between the way things appear and the way things are”. From my understanding, frustration will impede us when we consequently start to get bogged down in negative thoughts and doubts. Frustration is a natural emotion, and will happen to some more than others. It’s what it leads to and how we handle it that makes all the difference.
What frustrations are there in IT? Some will immediately argue that they are abundantly obvious. For example, I can vouch that software development can be quite a stressful place to operate in. We are given expectations and deadlines, we are constantly reviewed, we act as guardians to preserve the quality of the code we produce, not to mention that business requirements are always changing, and we are frequently interrupted with various meetings and with a whole host of other initiatives.
Other pressures include those of working in a team with a wide range of available skillsets. It’s very much a part of the day-to-day that you’ll be working alongside people that know much more than you in a given area, or on the other hand with people that know very little and require extra time to get up to speed. Looking at the Dreyfus model, where it is postulated that through the process of skill acquisition we move through five different stages, from Novice to Advanced Beginner then right through to Expert, it is logical to assume that each level comes with it’s own set of nuanced frustrations – as Dave Thomas writes about with level one.
So what’s the best way of handling frustration when it occurs? I wouldn’t claim to know comprehensively what the answer is, and individuals do vary greatly. I can think of colleagues that get regularly frustrated, or of those that possess such a Vulcan-esk like mentality that they wouldn’t seem to be able to grasp what all the fuss is about.
What I would offer is that improved self-awareness of when we are feeling frustrated is a good first step towards making our work-life more enjoyable. A second step is a measure of acceptance; it’s OK to feel frustrated, everyone does anyway so I don’t need to give myself a hard time over it.
Once we’ve accepted frustration, another option we then have is to be explicit to others about how we’re feeling. This is particularly useful in pair-programming. For example ‘I’m getting a bit frustrated with this coding problem as we’ve been at it for some time’, or ‘I’m lacking some understanding here, it’s just not making sense to me’. Being explicit about our concerns gives colleagues and ourselves a chance to invent some options about what to do about it.
I was in a pair-programming mentoring role the previous week where upon my pair immediately said as we sat down to begin: ‘I want tell you up front that I’m not convinced by this pair programming practice; I haven’t seen it to be much more effective and I’m finding it very frustrating’. This guy was much better to pair with than the majority of other developers I’ve worked with. Since I knew how he was feeling I was better able to help make things easier and not to aggravate his concerns. I suspect he was happier as he’d got immediately off his chest what was bothering him, and was able to have a clearer focus on the work we had to do.
All this said, about gaining awareness of frustrations, accepting them and making them explicit if need be, is much easier said than done. Doubtless, it comes much easier to some than others.
Meditation is one tool I know of that can help us to improve in this area. By increasingly our self awareness so that frustrations can be detected earlier, we give ourselves more options for dealing with them. Having a structured work plan is also effective. Making explicit the concept of ‘core’ hours where developers are immune to meetings and interruptions, and having frequent breaks when pair programming will certainly help.
Mainly though, let’s just be aware that software development can be… or is… a frustrating practice.
Permalink
07.22.08
Posted in General, Java at 9:03 am by Jon Pither
I was involved in an interesting chat with some colleagues about the apparent incompatibilities between pair programming and being in the ‘zone’; a Zen like state where everything just clicks and one is able to perform seamlessly to their optimum ability given the present moment. From a developer viewpoint our egos become almost on autopilot as our hands reach out and tap away at the keyboard, the mind harmoniously serving up a variety of designs and strategies that may help us solve the problem at hand. We effortlessly approach excellence.
Psychologists refer to zone as ‘flow’. Daniel Coleman – author of Emotional Intelligence – describes flow as being “emotional intelligence at its best… the ultimate in harnessing the emotions in the service of performance and learning”. Emotions generate a plethora of thoughts for us, including distracting worries and concerns. Therefore having our emotions working for us enables us to be more focussed and to think more clearly and objectively about a specific problem.
Pair programming is a not stress free activity. Though it has been proven to raise productivity it does require more focus on the part of the developer – asking much more of them than were they coding solo. Being in a “pair” you are constantly required to explain your intricate thought processes to another person. You are challenged, and you have a responsibility to ensure that the person you are pairing with gets up to speed with the material at hand and that you are providing them with a progressive learning experience.
We need to appreciate that developers are humans, and sadly are not perfect coding punching-out machines. Pair-programming introduces an emotional burden on the developer. Thoughts such as how the other person in the pair is coping and performing are one of many, along with speculation as to what the other person is thinking… Are they frustrated that I can’t keep up? Are they worried I’m going too fast? Or worse still do they feel guilty for slowing me down? – a common concern as developers strive to be productive. In order to deal with these worries a whole new – totally justifiable – plateau of thinking opens up. The brain switches over from analysing the current software problem to instead work on strategies for making the pair more effective: ‘they have a need to see this area of the code-base as it will help them to contextualise the problem better… I want to get some “quick wins” to build our momentum as a pair, therefore it makes sense to pick off this little bit of low hanging fruit…’
I believe that whilst pair-programming has the capacity to inhibit flow, the emotional intelligence within us is the tool to help minimize the impact distractions have allowing us to closer reach a level of excellence on an individual level. EQ is essentially about being aware of our emotions and the impact they have on us – to realise when we’re worried, or when our frustrations are mounting. A higher EQ will not only help us to better meet our own needs, but it will give us more empathy and understanding to better meet the needs of the person we’re working with – to help us create a more effective “pair”. EQ is essentially about being aware, and the best thing about it is that unlike IQ it can be learned.
Lastly, I want to refer to the premise that “stress makes you stupid“. We underperform when our emotions run unchecked. I’ve seen first hand developers resigning from jobs they sought hard for because pair programming has been too much of a difficult experience. If we can improve our awareness of the distractions and difficulties pairing introduces – and in doing so our EQ – then we’ll likely make it easier for ourselves and the person we’re pairing with.
Permalink
02.07.08
Posted in Java at 5:57 pm by Jon Pither
Some time ago I was in a meeting room of a big company with a large development capacity. Rather luckily I was present to observe the company’s strategy of going forward with software development.
It went something like this:
“So this is the way we want to develop; the new process we’ve been working on. We’ve pulled together our experiences so far, and drawn up the best uniform approach that will deliver software in the most stable fashion.”
A powerpoint presentation begins. Below was a model (as I remember it) of the new process:
It’s essentially a water-fall process. The idea is that after the analysis phase designated architects would present to the business a set of possible technical solutions and the business, weighing up the pros and cons, would make a decision as to how to proceed. Once the problem had been essentially ‘cracked’ by a team of analysts and the odd architect, the development work could be passed over to some developers to do. A merit of this process is that the developers could be located offshore.
It’s easy to sit back and make snide comments claiming that we know better, but it’s interesting to dig a little deeper into what’s actually going on here. A process has been created to minimize the role of the developer; it seems that this was top of the list of paramount concerns. One can imagine that in the past the business had been burnt by failing delivery projects, and so the solution is to reduce the involvement of the culpable developers in the future. I suppose it’s a bit like expensive plumbers. They’re a necessary evil one must endure, and so the less exposure and involvement of these plumbers in your day-to-day life the better.
Somehow the organisation has stopped trusting developers; they want little to do with them.
The problem as I see it is that a vicious cycle has formed. Once developers are deemed to be a failure point, the next time the overall software engineering process is refined the humble developer’s role will be further minimized. This increasingly disadvantages the developer, so that they’re even less likely to the deliver the next time. Hence the process is refined further and so on.
It would be a tough sell to this business that developers ought to be involved more, not less.
Permalink
Posted in Java at 4:50 pm by Jon Pither
I once worked as part of a team with a clear mandate to select a middleware product that would dramatically speed up our productivity. We were given a short list, pre-determined by a separate consulting company, specialising in the area of document management systems (DMS) and business process management (BPM) tools.
What were the motivations for doing so? Well, we had to develop a large green-fields application, consisting of many business processes and requirements for integrating with a multitude of third parties. The business employing us had a fear of us developers getting bogged down in developing a technical infrastructure – ‘plumbing’ was the word used. Instead, it was envisaged that with a serious tool on board we wouldn’t need to worry about all that stuff; we could rock on and start developing business value quicker. Up to a million quid was reserved for this initial expenditure.
That’s obviously quite a crude summerisation of the reasons behind this middleware seeking. There were other factors at play. Large tools give the impression of having well-thought-out concerns taken care of such as security, data-protection issues, audit-trails and logging, scalability and performance, and transparency into the underlying business processes. Given these benefits, spending a million or so quid – which can be relatively quite small in the grand scheme of things – could be seen as a favourable risk.
The arguments against are somewhat subtler, and when squared up to the proposed benefits pushed by a ferocious sales force, they can be seen in a somewhat dimmer light. I’ll attempt to bring some of them out into a clearer focus during this post.
Testing.
Testing a message-bus or restful SOA has been done many times and useful patterns and tools have arisen to make this a straightforward process. Trying to wrap an automated test-harness around a vendor product can be much trickier to accomplish. Why? There can be some major hurdles. First, is to try and get some hooks into the product to see what’s going in and going out; for example by using stubs that can be switched in and out automatically. Many middleware products come as a black box, which means it’s difficult to see the processing going on inside them. As if often the case, the documentation is fragmented and hard to trawl, and the consultants provided by the middleware company simply direct challenging questions back to HQ which is probably in a different country. A common complaint is that vendors don’t get testing. That is to say that they haven’t thought through how automated testing of business processes could be applied, let alone how such testing could be integrated into a continuous build environment such as CruiseControl. A reliance is placed on the product being so simple and easy to use that manual testing will suffice. In reality the complexity of some business processes cannot be avoided, and there will always be hazardous bugs that automated testing could catch.
Sometimes help is on hand. Open-source or third party products can crop up around the testing of large vendor based products. There are many, such as BTSUnit which tests BizTalk. A problem is that as the vendor product evolves incompatibilities arise with the testing tools; they aren’t developed to keep pace and thus they can fall into a state of dis-repair. A littered cloud of debris ends up orbiting the “big name” products.
Ease of development
A big selling point of large middleware solutions is how quickly they are to develop with. It is easy to see demos where within a few clicks someone has integrated with Ebay or Amazon. It’s so easy! A few shapes on a diagramming tool, some directional process lines and the entry of some credentials is all it can take.
Troubles set in however, when a development team with more than a few members start to work on the tool in unison. If developers can’t have their own locally installed instance to work with (for whatever reason, i.e. the lack of version control, or it’s too difficult to store the configuration), they may be forced to work on a centralized server. How the conflicts are managed depends on the tool. When I once used WebMethods it was recommended to us we have our own ‘package’ (think of sandbox) to work on, and they we manually copy across the changes into the production packages when we’ve finished. Disheartened by this, as continuous integration would be a very big challenge indeed, we opted to write our own tool that sucked in all the configuration our of WebMethods, and was capable of storing it as an XML document which could be versioned; the build server could then configure a fresh WebMethods instance with that information. It was however a nightmare. We got it to the point of working effectively so that a WebMethods consultant wanted to buy it, but it cost us time to develop and maintain, and it still had some frustrating issues.
The ironic thing is that the product was bought in order to prevent us spending time ‘plumbing’, yet by working alongside such a big middleware product, we got our hands far dirtier than expected.
Another portentous factor to consider is that big products might not do all they say they can do. For example, does it integrate well with SOAP? Yes – Great, which version? Oh, not 1.2? It’s a nice image that all the work can be done inside a single tool, yet if there are many third parties to integrate with the chances are that some additional software will be needed. How it’s all glued together is an additional complexity introduced.
So, if a large tool is easy to write automated tests for, easy to develop with a medium to large sized team, is it then going to deliver the promised impact on productivity? How does it cope when large complex business processes have to be entered into it?
Managing complexity
The art of BPM (business process management) tools is to make complex business processes look simple. This way they become easy to understand:
The complexity however does not go away; it’s simply no longer fully in view. There is a large danger that one may edit the business process using a pictorial tool such as the above without awareness of the underlying code. A separate problem is that without a highly skilled team, the underlying code may lose its coherency and simplicity, as it’s scattered around the various parts of a wider orchestration of which it is not aware. If the team is not careful the code will consist of procedural hacks and a series of work-arounds for logic that could not be clearly and simply expressed elsewhere. New skills will have to be learnt in order to develop cleanly and simply. It’ll be easy to do trivial work, but when a business process strays from the happy path and becomes ever more elaborate, it’ll be much more difficult to maintain the virtues of process visualization and simplicity. As has happened on more than a few projects ThoughtWorks have taken over from, the complexity modelled in middleware solutions has spiralled out of control.
There’s an underlying theme that these tools are aimed at business people, not developers. In the project I was a part of where a considerable amount of money was spent on WebMethods, the end result was that the product was side-lined in favour of a clean, well-tested code-based solution. Once the painful decision was made to discard the vendor based product, we went many times faster.
Permalink
« Previous entries
