Bringing agile to IT infrastructure: ING Netherlands’ agile transformation

IT executives seldom associate agile with infrastructure. In this interview, the head of ING Netherlands’ IT infrastructure function explains how his team carried out an agile transformation.

ING Netherlands, the Dutch bank within the global financial group, began introducing agile ways of working at its headquarters in June 2015. One year later, the bank extended the effort to a domain where agile methods are far less common: IT infrastructure and operations. The change increased the speed and stability of the bank’s IT operations and generated substantial gains in efficiency and employee engagement. In this interview with lapetitemort.info’s Vito Di Leo, René Visser, leader of the IT infrastructure tribe at ING Netherlands, explains why the bank chose to bring agile to infrastructure, how it brought about the transformation, and how it worked through the challenges that it encountered along the way.

Video

lapetitemort.info: Why did ING Netherlands consider transforming its IT infrastructure organization?

René Visser: Within ING, both on the business side and in the application-delivery part of the house, agile practices were more common. The infrastructure part was lacking a bit of that movement and ending up with a lot of operational dependencies.

As soon as you’d deliver a new application, for instance, you’d reach out to infrastructure colleagues to put some virtualized infrastructure in place. And in each of the next steps—delivering your application and operating and maintaining it—there was a need to touch the infrastructure for whatever reason. Every effort meant reaching out to infrastructure teams, which resulted in a huge backlog of operational needs and service requests. That was basically the issue we had to solve.

Other parts of the IT function were contributing to this problem too. Application teams didn’t always build applications that were easy to operate. So the infrastructure organization had to make extra efforts to prevent application failures.

lapetitemort.info: How did the agile transformation address these issues?

René Visser: The agile transformation for infrastructure is twofold. On the one hand, it means delivering infrastructure as a product. Not only the core infrastructure but also the full range of means to operate it, so that application-development colleagues can do that autonomously, according to their own plans. On the other hand, it means changing the way we deliver the productized services. That comes back to coding—software development.

It’s possible to do that, because our infrastructure organization mostly delivers software, not hardware, and we know agile methods work for software. From my time in application development, I had learned how to work in an agile way, and agile had become the predominant way of operating for our IT function outside of infrastructure. Aligning the working style of the infrastructure organization with the style of the rest of the IT organization makes a lot of sense.

We also knew that if we could enable the application teams to deploy applications and maintain the underlying infrastructure on their own, that would bring us much closer to a DevOps model and consequently accelerate cloud adoption, which are two core elements of our IT strategy.

lapetitemort.info: Going into this transformation, your team had to know that it might be challenging.

René Visser: We expected that it would make some people uncomfortable. Infrastructure’s mission would have to change from building and operating mostly custom solutions to creating tools for developers to deploy standardized infrastructure on their own. We had some engineering talent, but we knew that not everyone would have high-caliber skills or want to do engineering work. We also needed to shift people out of their functional silos and put them onto cross-functional teams.

The application side, the “consumers” of infrastructure, would have to adapt to a self-service approach. Although the application teams were DevOps teams by definition, many of them still needed to learn about operations for infrastructure.

lapetitemort.info: How did you get people on board?

René Visser: We knew the change would not be easy, but we also knew that there would be many benefits for our employees. In the new structure, we would simplify processes, reduce handovers, and increase our contribution toward overall IT delivery and agility. Furthermore, the new organization and roles would open up new opportunities and career paths.

We spent the next few months engaging our employees in a regular and open way. I invited our infrastructure colleagues to talk to me by the coffee machine in our office. We held these “coffee corner” meetings one to two times per week. People asked me direct questions, and I answered them as honestly as I could. So many people came that sometimes we had to move the discussion into an open space to fit everyone.

I also spent a lot of time with leaders in application development to explain the new vision and get their views on it. They helped us decide what kinds of services infrastructure should deliver, and how they should be organized.

lapetitemort.info: What happened next?

René Visser: We did a complete redesign of the infrastructure organization, covering every last team and role, based on two key changes. First, we moved from silo teams that worked on service requests to multidisciplinary engineering squads that created infrastructure products and services. For example, the Linux squad would engineer a compliant operating system image and the tools required for application teams to perform second-day operations. We also set up squads to create products for specialized infrastructure technologies such as a mainframe or data lake. Second, we moved from a broad range of specialized roles to a simplified set of engineering roles, about ten in total. We also don’t have project, service, and business-relationship-management roles anymore. The most common roles are engineers, product owners, chapter leads, agile coaches, and architects. These two changes made the target organization much simpler and slimmer than the one we had until then.

lapetitemort.info: How did the infrastructure leaders select the right talent for each role and squad?

René Visser: We asked everyone to apply for up to three roles in the new organization, and then we put a lot of effort into assessing people. That’s the standard reorganization approach at ING Netherlands.

We determined that, because we will be delivering a new product, we would also need people to have a different attitude and software-engineering skills. We made that very explicit. We posted our new job profiles quite early in the process, and we opened them to everyone in the organization.

We also had a very strict process of assessing people’s engineering skills and mind-sets—as we call it within ING, “Orange Code” behavior. We also gave people a coding test. All of that together gave us good insight into people’s willingness and ability to succeed in the new organization.

Everyone also went through at least two interviews, one on technical know-how and one on agile mind-set and way of working. We did all this for hundreds of employees in just ten weeks, because we wanted to minimize the uncertainty for them.

lapetitemort.info: That sounds like an enormous effort. Who was involved in carrying it out?

René Visser: Interviewing all the employees was indeed a lot of work, in fact more than we could handle. I ended up asking people from the application side to help with the interviews. Fortunately, I had spent most of my career at ING on the application side, so I had good relationships there, and enough people stepped up to help.

Involving application-team leaders in the interview process didn’t only alleviate the burden of conducting the interviews. Bringing the two sides together in these interviews also allowed people to gain a much better sense of how they’d be working together in the future. That was at least as valuable, since we weren’t just transforming the infrastructure organization but also the whole model for collaboration between development and infrastructure.

lapetitemort.info: Once the redesign of the organization was complete and people were matched with new jobs, how was the new structure implemented?

René Visser: We came up with a creative way to bring the new squads together. Employees on the same squad got hats with the same color so that they could find one another and meet face-to-face. In one day, we switched from the old structure over to the new one. We also asked the new product owners to redesign the seating areas, so that interaction among the new squads could be encouraged where it would be most needed.

To ease the transition, we set up temporary teams that would fulfill service requests, so that the new squads would have time to automate our infrastructure services and build new tools for developers. Because the transition teams would become obsolete, we staffed them with external contractors so they could be phased out more easily. Since the launch of the new infrastructure organization in November 2016, we have reduced the transition teams by more than 50 percent.

lapetitemort.info: What was the approach to getting the agile squads up and running?

René Visser: We kicked off the squads with boot camps. We assigned them a purpose at the outset, but then they refined their own purpose. And they also developed, with one another, the first product road maps and the first backlogs. Coaches were helping with that. Later on, to help the squads get in a rhythm, the coaches helped the squads with alignment to other squads as well as with personal coaching, helping them to achieve their new goals.

There was a lot of room for training. But the best way to learn is by doing it. At least, that was my own way of working. Just do it—don’t be afraid to make a mistake. We will not freak out. But let’s learn from every mistake and make it even better next time.

lapetitemort.info: How have the infrastructure squads developed over time?

René Visser: We had all varieties. Some squads were doing really well from the start. Others were struggling with agile, not finding the right rhythm. But we were very persistent about how we wanted teams to go, and so we switched our agile coaches or tried approaching things from different angles. There was quite a lot of room and freedom for teams to find their own way. And after a year, most of the teams were in good shape.

Looking back, I think we should have tried to steer the squads toward the same working methods at the very beginning. The coaches had a lot of freedom to decide what sort of coaching each squad needed, and some worked more on things like individual communication or working styles. Some teams got comfortable with agile methods after half a year, while others were still figuring things out. So the leadership team made all the coaching efforts more consistent. We wanted squads to adopt DevOps and be more uniform in how they worked.

lapetitemort.info: Did ING’s application developers respond well to the new infrastructure model?

René Visser: Some infrastructure squads did a better job leading application-development teams to use the new self-service tools. The best squads tried to understand the situation that developers were in, then spent time explaining the tools they had built. Others just told development teams what tools were there and said they had no choice but to use them. That didn’t go as well.

Overall, development teams accepted the self-service model quickly. It lets them work faster and with more flexibility. For example, our agile squads have created self-service tools for development teams to do infrastructure patching. Now those teams can apply patches when they want.

lapetitemort.info: How has the agile transformation affected the resilience of the IT landscape?

René Visser: The application developers’ job is to develop and deploy applications that perform well. Yet the responsibility for making applications run smoothly ended up with infrastructure. That allowed development teams not to worry whether applications were prone to failure. For instance, we might have had an application that would crash because of a two-millisecond infrastructure interruption. There’s no way for infrastructure to prevent outages completely. But that’s what was expected.

Now we’ve increased in stability. The biggest push was not only with the infrastructure part of the house but also with colleagues from application-delivery teams. It takes two to tango. We improved our IT resiliency quite a lot by acknowledging that we need each other. It’s not only infrastructure that drives 24/7 availability. You also need a good application architecture.

Our CIO in the Netherlands, Peter Jacobs, has really helped to shift the balance. His message has been that if applications fail, then developers need to fix them. They can’t just expect infrastructure to keep things from going wrong. In line with this principle, we made the application teams accountable for the support and stability of their applications and systematically reduced the centralized level-one and level-two support functions. Development teams got the message and made their applications more resilient. It took them only a couple of months. Now the entire IT landscape of ING Netherlands is more robust.

lapetitemort.info: What other benefits has the organization realized?

René Visser: Our software-development teams now have the tools they need to operate infrastructure for their applications, and the responsibility to do so. That’s very much like a cloud operating model. The transformation has allowed our development squads to gain experience managing infrastructure in that way, so the next steps of adopting cloud will become easier and more natural. From a technology perspective, we have also made strides to that end. For example, we made upgrades at the level of data center and connectivity to align our environment with the requirements of our cloud architecture.

The agile transformation also helped us to improve in most operational areas. We have significantly reduced planned outages, incidents caused by infrastructure, and weekend work. Now changes are normally done during working hours. We have become more efficient. With fewer people, we have increased delivery levels by shifting activities toward engineering and automation.

lapetitemort.info: What would you recommend to others who are embarking on a similar transformation?

René Visser: Don’t try to fix things on the infrastructure side only. You definitely need your colleagues on the application side of the house to help approach [the transformation] from end to end. Which means engaging them and benefitting from their feedback from early on as you define what you’re going to design, and then bring them in the loop on what you’re delivering. And take them with you when you have your first successes. For instance, making patches during the daytime will go against all kinds of traditional perspectives, so you should celebrate that kind of success.

About the author(s)

René Visser is the head of the IT infrastructure tribe at ING Netherlands. Vito Di Leo is an associate partner in lapetitemort.info’s Zurich office.

Related Articles