Monday, October 3, 2016

Impressions from the first day of GOTO Copenhagen 2016

I have just returned home after spending the whole day at GOTO Copenhagen 2016, listening to great speakers talking about software development.

As I wrote in my last post on GOTO Copenhagen, I had planned to spend the day watching the Effective Delivery track, and this is what I did.

For someone like me, who works with projects in the public sector, this was a fantastic track.

The first talk was Tony Grout's Hand to hand with a gorilla won't work, about how you introduce agile in a very large organization.

Basically, it takes a lot of effort, and some cunning, and it won't be pure, but as Grout said, "good luck with pure". Sometimes you have to be pragmatic in order to reach your goal.

One interesting thing that Grout said, was that a fairly simple, yet very effective tool, is an ordered list across the organization, allowing everyone to know what they should focus on first. Introducing such a list in Skype, increased the productivity ten-fold.

Next up was Jez Humble's When DevOps Meets Regulations: Integrating 'Continuous' with 'Government'

This talk was about the efforts of introducing DevOps in government projects in the US. A big barrier to this are the regulations they have to follow when making "information systems".

As I write in my tweet, the list doesn't seem overly long to me, compared to what we experience in Denmark. As the talk progressed, however, I realized that the US process for fulfillment is much more cumbersome than the Danish one, and it definitely needs an overhaul.

One cool thing that Jez Humble introduced us for, is an open source cloud platform for government projects in the US. This seems like a great idea, and I'd love to see something similar in Denmark.

After the US, the turn came to the UK, in the form of Stephen Forshew-Cain's Building an effective delivery culture, where he talked about the Digital Service and its culture.

I am planning on writing a longer blogpost about Jez Humble's and Stephen Forshew-Cain's talks and how they relate to the Danish situation, so I won't go further into the talk here.

Having spent some time on government projects, the time came for talking about effective teams - this came in the form of Camille Fournier's Building a High-Performance Team is Everyone's Job. This was the title in the program, but she had given it a different name at the time of the talk (the title is unfortunately on a photo in my phone which has run out of power).

I had never heard Camille Fournier speak before, but I will most certainly do so again, if I ever get the chance.

It was a very amusing, and highly informative talk, where she spoke about her experiences, and what she had learned from them, when it came to leadership.

last, but not least, on the track, Emily Webber gave her talk Communities of practice, the missing piece of your agile organisation, in which she talked about how to create communities of practice inside your organisation.

All of the talks were great, and I certainly got a lot out of them. Hopefully the same will be the case tomorrow, when I spend my time at the Tactics for better Teams track.

Wednesday, September 21, 2016

It is time for Danish politicians to form an opinion on IT projects in the public sector

As in most other countries, public IT projects have a bad reputation in Denmark. It seems that most people think that the public sector is horrible at doing IT projects (much worse than the private sector), and that the majority of public sector IT projects fail.

This perception is caused by a number of very big IT projects failing, causing not only the loss of the money put into them, but also the loss of state revenue, due to necessary IT infrastructure not being in place in order to e.g. ensure that tax refunds are paid correctly.

In my opinion, the public perception is wrong on two areas:
  1. The public sector is no worse than the private sector at doing IT projects.
  2. The majority of public sector IT projects go well.
The reason for both of these misperceptions is that people don't hear about projects that fail in the private sector, nor about IT projects in the public sector that does well.

But even with this caveat, there is no doubt that there is uncomfortable large number of IT projects that fail in the public sector, having a negative impact on the Danish economy.

Given this, I think it is remarkable how little the Danish political parties seem to care about IT projects in the public sector.

The politicians generally seem to care only on the most superficial level, saying that the public sector need to get better at doing IT. If they go beyond that, they generally focus on ensuring that laws are not getting in the way of implementing new IT systems and work rutines.

I think it is great that they are willing to take a look at how the laws in a given area might get in the way of eg. digitalization, but this is not enough. The head of the Danish Business Authority, which has just gone through a fairly succesful five year modernization program, has stated that it is frequently not laws that gets in the way, just interpretations and existing workflows.

It is often not clear that the interpretations and workflows will get in the way of development of the new systems, before the actual development work starts.

So, in other words, it is not enough that politicians look at the laws, they also have look at fostering an environment, where it is not only OK, but pretty much required, for public servants to reevaluate interpretations and workflows. For this to be possible, it is necessary to get them directly involved in the IT projects, not only as end-users, but also as sparring partners for the developers. This requires that resources are put aside for this.

Another area where I'd like politicians to focus, is creating an environment where different ministries, departments, and agencies learn from one another. There are some great attempts at this on the local level (eg. some departments look around at other departments to see what works before they start up), but it has to be structured better, and the responsibility anchored in one place.

There are a lot of experience spread across the different ministries, departments and agencies, but all too often it cannot be easily located, and each IT project is cause of re-learning everything from scratch.

A good attempt of drawing on the experiences of others, is the use of IT-projektrådet to evaluate the risks of IT projects larger than 10 million kroner and to follow up on high risk projects.
This is only done to federal projects however, and not on projects on the municipality level or regional level, even when they are much larger than 10 million kroner.

Obviously there are other areas where politicians could, and should, form an opinion on IT projects in the public sector, but I think that the areas mentioned above are some of the more important ones.

Sunday, September 18, 2016

What I want to see at GOTO Copenhagen 2016

I have taken a look at the schedule of GOTO Copenhagen 2016, trying to figure out what I want to see.

On Monday the 3rd, there are a few interesting tracks for me.

First of all, the description of the Effective Delivery track sounds very intriguing:
Anyone with less than 15 years in IT has never known a time when there wasn’t Scrum or XP, or cross-functional teams or automated tests. They take these things for granted, but they also believe this is the state-of-the-art. Mainstream agile methods are around 20 years old now, and the world has moved on. Planning every 2 weeks makes sense if you have a 3 month release window, but what about if you are releasing every week, or even several times a day? Servers are now an inexpensive commodity, and cloud infrastructure means my expensive data center is now someone else’s pay-per-use model. How are modern practitioners taking advantage of this? What about challenges like large-scale delivery, regulated government departments, or heavyweight corporates? And how can you navigate a career when half the companies on your CV no longer exist? This track passes a critical eye over the current delivery landscape, and hopefully gives you some tools and techniques for navigating the modern world of Effective Delivery.
I like that the track takes for granted that we understand the basics of Agile development, as most of us have worked with it our whole career, and build upon this.

It is highly likely that I will spend most of the day at this track, and there are a couple of talks I definitely won't want to miss. One of these is Jez Humble's When DevOps Meets Regulation: Integrating 'Continuous' with 'Government'. As someone that works primarily with projects related to the public sector, this certainly sounds like something that I can use in my daily work. The same is probably true for Stephen Foreshew-Cain's talk Shepherding Government towards Effective Delivery, but there is no description of this talk yet.

If I am not at the Effective Delivery track, I'll probably be at the Deep Learning Analytics track, where there are several interesting talks, including Michael Hunger's How the Investigative Journalists of the ICIJ used modern Open Source Technologies to unearth the stories of the Panama Papers, where Neo4j will play a major part of the talk. Neo4j is a technology I have followed for years, and I would love to hear about how it was utilized by news organizations, so they could map the data.

On Tuesday the 4th, there is no doubt that I will be spending my day at the Tactics for better Teams track. The track is described thus:

Let's get practical. Often small simple changes have a huge positive effect on team performance and work-life satisfaction. This track presents a set of simple tactics that you can bring home to your team and start using today to make a better team. The track will present both technical ideas like software visualisation as well as more process oriented initiatives.  
The reason I'll be spending my day at this track, is partly that it is the track that is most related to my daily work, but mostly because of the speakers on the track. Of the speakers, the only one I don't know if I have seen before is Corey Haines. The rest of the speakers of the tracks are people that I don't only know I have seen before, but also know that I love - especially Linda Rising, who is a brilliant speaker.

Hadn't that been the case, I'd probably have spent some time on the Microservices track, trying to figure out what all the hype is about.


Monday, September 12, 2016

The Dangers of Conferences

In just under a month, it is time for GOTO Copenhagen 2016, which is the 20th anniversary of the GOTO conferences (which started out as JAOO). I am looking forward to listening to some great speakers and network with other participants and the vendors.

When talking with other people in the field of IT, I often come across people whose employers won't pay for conferences, and sometimes even won't let them take time of to go there.

I have in the past tried to make the case for allowing employees to go to conferences, but I think it is time to also acknowledge and address the dangers of allowing employees to go to conferences.

The rest of this post will list the major explanations I've heard, and try to address them.

1. The company only work with legacy systems, and all the new stuff mentioned at the conferences is irrelevant for us.

A conference like GOTO doesn't only present new languages and technology, but also have sessions on methods, architecture and many other things that are several layers of abstraction away from actually systems development.

Even if your company doesn't make use of the newest technology, conferences present a rich source for re-evaluating current practices, and address inefficiencies in the current setup. Many, if not all, of the speakers have not only worked on cutting edge greenfield projects, but also on brown-field projects, that have turned into big balls of mud, so they understand the need for taking such projects into account.

2. When my employee go to a conference, they will met people from other companies, and be tempted to go there.

There is an old anecdote about a the leaders of a company talking to a consultant, and asking "what if we train our employees, and they all leave?" To which the consultant answers, "what if you don't train them, and they stay?".

If you are afraid that the employees will leave if they hear about conditions at other companies, maybe you should address the conditions at your company, rather than letting them get worse, by not allowing people to go to conferences.

Also, have you considered the idea that the employee might convince other people to check out your place as a potential workplace?

3. If I allow one employee to go to a conference, everyone wants to go.

First of all, I don't think that is true. In most companies, there is a large group of employees who don't particularly like going to conferences, and thus won't ask for it.

But even if that's the case, then I don't really think it is a problem. There might be an economic limit to how many can go, but that can probably be solved by creating a turnus, where employees get to go on alternate years.

If a large group of employees go, then they can either use the conference as a team event (if they work together), or as a way to get to know people in other parts of the organization better (if they don't work together). Either way, it makes for a closer knit mass of employees.

4. Conferences are expensive

"Expensive" is relative.  IT employees are usually highly paid, and hard to recruit. For many good IT people, conferences are a factor when considering whether to stay or change jobs.

If someone changes jobs, the cost of replacing that person is much, much higher than the cost of allowing them to go to a conference.

5. My employees will get frustrated over hearing about new tech that they can't use

Yes, this is a real risk, but there are ways around that. Maybe allowing them to slowly introduce new tech in their current projects, or by allowing them some "play time" to build tools and other programs that can help them in their daily life.

Trying to keep them away from learning new tech is only going to work for the most incurious of employees. If you are fine with your employees not wanting to keep up to date on their field, that is fine, but if that's not the case, then you have to find a different solution than keeping them from konferences.

6. My employees will try to implement the newest fad after the conference

Somewhat related to point 5. Yes, this is a risk, but you can limit the risk by ensuring that the employee get an outlet for their creativity.

Sunday, September 4, 2016

Is microservices the new SOA?

Within the last couple of years, Microservices has become all the rage of systems development.

Microservices, like many other terms in systems development, doesn't have an exact definition, but I found a good working description at an article by Martin Fowler and James Lewis:
The term "Microservice Architecture" has sprung up over the last few years to describe a particular way of designing software applications as suites of independently deployable services. While there is no precise definition of this architectural style, there are certain common characteristics around organization around business capability, automated deployment, intelligence in the endpoints, and decentralized control of languages and data.  
Looking at the description, there is nothing that doesn't reasonable, and reading the article linked above, it becomes clear that Microservice Architecture is a valid way of building systems, and one that should be taken into consideration when designing a system.

Unfortunately, it seems like people don't really consider it, but rather just choose it by default.

A decade ago Service Oriented Architecture was all the rage, and all systems had to be based on SOA. Unfortunately a lot of people didn't understand SOA and thought it was all about having a servicebus as the central component through which all communication went through. This was so widespread that I and many others started to say that SOA stood for Servicebus Oriented Architecture.

It seems like Microservice Architecture is moving down a similar path.

Many systems supposedly based on Microservice Architecture are not really based upon it, but rather on a misunderstanding of what Microservices are all about, creating systems which mimics the properties of Microservice Architecture, but don't implement the actual characteristics of Microservices.

Some symptoms of this misunderstanding of Microservices, could be:
  • Code dependencies between services. If code changes to one services means that you also have to change the code of other services, then the architecture doesn't fulfill the concept of independently deployable services.
  • Functionality dependencies between services. If the system always have to call service A before service B, and only call service A when service B needs to be called, then it seems like the lines of the system hasn't been drawn the right place.
  • The need for a service portal (or even a servicebus).
There could be good reasons for each of these symptoms (though I am hard pressed to think of one for the first one), but they are definitely something that should make you consider whether you have really created a Microservice Architecture, or if you have just peppered your system with services.

Monday, August 29, 2016

Think smaller

Over at my other blog, I mentioned the fact that the Danish Tax Department is going to go through a major overhaul (Denmark overhauls its tax department).

One of the major triggers for this overhaul, is the problems that the Tax Department have had with creating new IT systems in recent years. Two major IT systems have failed - one which was going to calculate the taxation value of property, and another which was going to be used to collect overdue taxes and other debt to the state.

The failure of these IT projects have cost billions of dollars in both wasted development costs and in lost revenue to the state of Denmark.

As a frontrunner to the overhaul of the Tax Department, the Ministry of Taxation had already decided to not let the Tax Department do the development of the property system but instead do it themselves. This was a major break away from the traditional roles of the Ministry and the Department, where the Ministry was focused on law-making, while the Department had the responsibility of enforcing the laws (e.g. collect the taxes), and make the systems necessary to do so.

As part of the Ministry of Taxation's decision of taking the responsibility of developing the property evaluation system, they also decided to do it in-house, rather than using public tenders to get it developed.

I was recently at a talk given by Steen Larsen, the head of development at the Ministry of Taxation, called "A New Way of Doing Public IT", where he explained the things they had done to create a succesful project, which would deliver a working system on time.

This was very interesting, though I think the title was overselling it a bit, as there are several public agencies and departments which do much of the same as what he described.

Listening to the talk, and seeing the ideas behind the overhaul of the department, I can't help worry if they have identified the right problem.

It seems to me that they still have a tendency to think in large monolitic systems.

As Steen Larsen described it, the IT system they were building was both a system for evaluating the properties, and for the case workers to do their work.

It seems to me, that it should be possible to split these areas of concern up, so they could be considered two different systems, and making the sizes somewhat more manageable. Generally, this allows for a greater chance of success.

Hopefully, the tendency of thinking in monolitic systems is only when they describe the systems, and not when they do the actual development.

Project managers vs Product owners

I frequently come across discussions on the internet about using project managers as product owners in Agile projects.

These discussions often spring out of discussions on what the role of a traditional project manager is in an Agile project, and since project managers often prioritize work in traditional projects, it makes sense to think of using them as product owners, who, after all, should prioritize the work that the developers do.

I disagree.

When Scrum was created, and the roles of Scrum Master and Product Owner came into existence, the people who got those roles, were the traditional project managers - at least according to what Jeff Sutherland said when I got my Scrum Master certification at his course.

That probably made sense back then, but I think it is a bad idea to do so now, for several reasons.

First, and foremost, I think it is a bad idea, because you need the project manager to focus on the project and its priorities. Unless you run an extremely agile project, there is still a need for a project manager taking care of the non-development aspects of the project (e.g. resource planning, dependency management with other projects, and other tasks ignored by Scrum and other Agile methods). In other words, project managers need to focus on the restraints and barriers of the project, and ensure that these doesn't affect the project and its progress.

Product owners, on the other hand, should represent the business/customers, ensuring that they get the most value out of the project as they can, within the boundaries set by the project manager.

Secondly, I think the skillset required to be a good project manager is orthogonal to the skillset required to be a good product owner, and it is best to use people in the role they fits the best.

A good project manager is good at forecasting and evaluating risks for the projects, ensuring that the people inside the project can perform optimally. A good product owner, on the other hand, understands the needs of the business/customers, and can translate this in a way that the developer team (including testers) can understand. This is why good product owners often are business analysts with a developer or tester background.

Thirdly, in most projects, being a product owner is a full time job. The number of developers in a project generally outnumbers the product owner by a fair margin, and having the product owner do project management as well, will slow the project down - or at least limit the number of developers that can be in the project.

All of the above is based on the premise that projects still need project managers, even if the development process (e.g. Scrum) doesn't call for it. If that's not the case, then the only relevant objection is my second point, about the necessary skillset being quite different. But this is a major point! In my experience, the role of the product owner is essential for the success of an agile project, and all too often, the selection of the product owner is taken too lightly.

If you want to find a good place for your project manager, look at the Scrum Master role, which has many similarities, focusing on sheltering the developers and create the framework for the development process.

Monday, October 5, 2015

Looking back on the first day of GOTO Cph - missing a bit of edge

I am at GOTO Copenhagen these days, and thought I'd post a quick overview on my thoughts on the conference.

First of all, unlike last year, it seems like the venue is too small for the conference. It is hard to get around, and many of the sessions are filled up 10 minutes before they start. It was said that the conference would be at a different venue next year, which I think is a good idea.

The first keynote was by Dr. Sengupta and was about the Mars Curiosity Rover. It was absolutely fantastic, and when there were some technical problems with the sounds in the middle of the talk, she managed that brilliantly as well.

Then there were the sessions.

I mostly went to talks in the tracks "game changing methods and practices" and "fast and continuous delivery".

I am not going to talk about the individual sessions, but I am going to quote a couple of tweets I wrote during the sessions there

There is no doubt that the process/method track gave us new techniques for doing Agile, but in my opinion, it would have been greater if the process/method track would have challenged us, and made us re-evaluate how we do systems development, rather than just provide new tools for doing it the same way.

An example of how this could have been done, would have been to find some organization that did waterfall projects with success, and get them to come an tell us about how they manage. One such organization is probably NASA, since I have a hard time imagining that the software for the systems used for landing the Mars Curiosity Rover was developed in an Agile project.

I think many of us would have loved to have our preconceptions challenged this way.

What I wrote could equally apply to the "fast and continuous delivery" track.

Sunday, September 20, 2015

Code of conducts at conferences

Looking at the GOTO Copenhagen website, I notice that there is a code of conduct linked at the bottom. The only reason why I found it, was probably because I went looking for it.

As far as I know, this is the first year that has one for a GOTO conference in Denmark, though there might have been one last year as well.

I am quite happy to see one, since I wasn't sure that there would be one, as Danes seems to have an aversion against codes of conducts for some reason or other. Earlier this month, I was at the Coldfront conference, which does have a code of conduct, but where one of the organizers felt it was necessary to half-apologize for it.

I am glad to see that code of conducts become more widespread, since I think they are necessary in order for conferences to become more inclusive.

A code of conduct is important to conferences, especially with an international crowd, for several reasons:
  1. It sets up clear boundaries of acceptable behavior.
  2. It helps enable people to speak out if they feel harassed or uncomfortable.
  3. It explains people what they should do, in case of harassment.
  4. It helps unveil the scope of the problem.
In regards to the first point, many people say that it shouldn't be necessary, and to some degree I agree, but unfortunately, experience shows us that it is necessary - especially at conferences where there is a large disparity between the genders.

It is also worth remembering that at an international conference, there will be people will different social and cultural backgrounds, and there are different boundaries in different cultures and social circles.

Regarding the point about enabling people to speak out, the existence of a code of conduct demonstrates that the conference cares about the well-being of all the participants. This in turn, encourages people to speak out when they feel that the boundaries are being pushed.

Since it often comes as a surprise for people that they are making others feel uncomfortable with their jokes or behavior, it allows those people to change their behavior in ways that makes it a pleasant occasion for everyone.

And even if there are some people that don't care if they are making other people uncomfortable, then there is the option of reporting them, which the code of conduct should provide clear instructions for.

For the organizers, it also means that they will hear about serious incidents straight away, and not through some backdoor channel long after the fact.

I think most of us would prefer that there was no need for a code of conduct, but history has shown us that this isn't the case, and it would be naive to think that if we ignore the problem, it will just go away. Instead, provide a code of conduct which provides clear boundaries and guidelines.

It seems like GOTO Copenhagen is more or less adopting the code of conduct provided by the Ada Initiative. Note that it has a public version, and an internal version for the conference staff, which includes clear instructions on how to enforce the code of conduct.

The later part is an important part of the code of conduct, which all too often was lacking in the past, creating situations where conference staff either overreacted or ignored any reports they received, and where documentation of incidents were non-existent - allowing some conferences to claim that there had been no reports of harassment at their conferences, even though several people have reported such.

Saturday, August 22, 2015

Social science and systems development

A couple of weeks ago, I wrote a blogpost on Agile and pseudo-science, which led to an interesting discussion on my Facebook wall about the lack of science and data in systems development.

As I wrote in my post, there is an unfortunately tendency of systems developers to rely on pseudo- or popular science, rather than proper science. A lot of decisions are made on assumptions for which there is no real evidence, but which somehow has become embedded in the world of systems development - be it project management, programming, testing, or some other aspect.

We tend to think of systems development as off shot of computer science with some cognitive psychology thrown in. While computer science and cognitive psychology certainly are important for creating systems, they are very much related to the "systems" part of systems development, while they hardly add to the processes, methods etc. that adds up to the "development" part.

A comment in the before-mentioned Facebook discussion suggested that we should look at social science when talking about science and systems development.

This seems like a good idea to me. A lot of what goes on in systems development is about humans interacting with each other, rather than about math, algorithms, cryptography or other aspects of computer science.

I once worked on a project where we had a horrible high turnover, which is definitely a well-known sign of a doomed team. Yet this team was one of the best teams you could imagine - continuously over-performing in the sense of delivering over scope, under budget and without flaws. From everything we know about team dynamics, this team should not have performed like this, but it did, and it could have been good to have had someone qualified there to analyse how this could be the case.

Equally it would be good to have someone look at the opposite type of teams - the ones which in theory should perform well, but which continuously under-perform. The ones where good developers suddenly seem unable to actually finish anything, where simple requirements turn into hideous bloated things, and where the only cure seems to be to kill of the team, never to let them work together again.

In both cases we know the anomalies relate to people, the interaction between them, and the culture of the team and the organization surrounding it. We just don't seem to be able to figure out how to create the first type of team, while avoiding the second type.

Given how many systems development projects fail, maybe it is worth looking at these things? And if we do, maybe we should try to find the proper tools?

Social science would seem like a good fit, and it would seem like a good idea for large organizations with many systems development projects, to take a look at how the research methods of social science could be applied to systems development, in order to become better at it.