Written by Matt Smith, Head of Cloud & DevOps Practice at Ten10
When it comes to DevSecOps, one of the challenges you face when you get into DevOps is the speed of change is so high that businesses can’t keep applying security changes as frequently as they would like. If you go back 10 or 15 years, infrastructure would typically be built and would sit there for 5 years and that would be considered a done job. But when we talk about DevOps, we are lucky if that infrastructure lasts more than a week at most. It is changing all the time.
Therefore, when talking about the integration of DevOps and DevSecOps, we need to ask how those security principles can be applied more often. For example, you might maintain an intrusion detection system that you update on a regular basis based on your logs. In this, it is vital that businesses understand who is making changes to their systems.
And one of the great things about DevOps is it guarantees immutability so you know exactly what the state of the infrastructure is. This means that even if your infrastructure is compromised, when you re-release it, you can rebuild it from scratch which guarantees any changes are reverted. So, you’re giving yourself a fresh slate every time and ensuring that every release you are at the latest update. Combining this with modern log management and observability means you have much more visibility and proactiveness compared to 15 years ago when someone would have to trawl the logs manually. It gives you far better visibility when you compare it to what we used to do 15 years ago, which was put the measures out there and hope no one did anything.
The rate of change in the security landscape is so fast-paced and there are always new threats and new threats to keep on top of. The idea of being able to build that process regularly into the framework means you can keep on top of it. If your system can’t scale, you can’t grow and if it is vulnerable the damage is immeasurable.
The mantra ‘shift security left’ is often associated with DevSecOps. Can you elaborate on what this means and why it’s crucial?
It may sound obvious but when releasing a product or a service, businesses often don’t know what they don’t know. From a business perspective, the priority is just that the product is valuable in some way. This sometimes means that a product is released that has some areas that have the bare minimum; if no one is going to use the platform yet there is no point in locking it down.
Over time, businesses can then change and develop. The architectural change goes through enhanced security and deployments far quicker than it used to. Almost on a weekly basis, businesses can change their architecture, something that is very hard to do when you have a traditional mindset. The traditional mindset process follows a pattern of thinking that “right, this week the architecture is this, let’s go through the review boards, let’s get people to sign it off, now this week the architecture is this”.
Those processes don’t work on week-long time scales; they work on month or quarterly time scales. Having that embedded mindset and having someone who is the security-focused member of the team means you are then able to release things that are more secure than they would have been. It also has the knock-on effect of a continuous improvement attitude. Therefore, when the next sprint comes along, you make it better again. There are definitely a lot of benefits to shifting it to the left because you address those problems far sooner.
What are some of the primary challenges organisations face when trying to adopt a DevSecOps culture?
The biggest challenge to DevOps adoption no matter what capacity you do it in is whether or not you can buy into the process and accept that things change. Unfortunately, in many organisations, there is a difference between what they want to do and what they are set up to do. In an ideal world, everybody would want to be able to deploy security patches as quickly as possible, but the problem often is that you have two worlds and cultures often colliding.
On the one hand, you have those who come from a change background where everything is done really slowly and with a strong emphasis on audit which doesn’t play in nicely with releasing updates 20 times a day. On the other hand, you’ve got those with backgrounds in engineering for example who are more open-minded to this level of change. The problem is that for those in change management, this slow and audit-focused process is their job and by automating that aspect, you leave many wondering what their purpose is.
It is also hard to take teams of developers who are accustomed to writing code in a certain way that maybe doesn’t have security at the forefront of their work. And that is a massive cultural shift; trying to get your team developers to become aware of the potential security implications of the code they are writing day in and day out. You’ve got tools that can catch those in your DevOps pipelines.
From a cultural perspective, having awareness of security throughout the entire engineering team is a challenge. When consulting, its common to hear that “we know what we know and we are good at what we do but we are really struggling with the people we’ve got as they can’t switch this mindset.” They are looking to bring in new people with fresh ideas.
It’s true that there has been a mass adoption of DevOps but only about 10% actually implement it correctly. So, you have this mass adoption but no one understands why they’re not getting the value because they haven’t solved the cultural side.
For businesses looking to transition from a traditional DevOps model to DevSecOps, what are your top recommendations for a smooth and effective shift?
Part of any change is about leadership and who is leading this shift. I’ve seen organisations with 20,000 software engineers under a CTO and they have an operations team under a CIO with another 10,000 people and the DevOps is being done to them, both areas independently. I don’t think this is a good approach because you end up getting good software released but the operational support is poor. The reality is that it needs to be merged and it needs to have a concise unit that is doing both sides of it, or else businesses are just creating a DevOps wall between the two.
The problem is that if there are people who are already in an organisation, particularly if they’ve been there for four or five years, they aren’t going to make recommendations that are conducive to the best output for the business. They are going to make recommendations that are conducive to them and that’s one of the biggest challenges that you can see already.
A lot of big organisations aren’t able to make this shift because there is such an aversion to change and there is a difference in priorities with the board level and the tech teams. When teams talk about security, in comparison to other things that will be more revenue-generating, it just falls by the wayside. But what often isn’t considered is that security tends to not be a problem until it’s a big problem. Having that mindset and keeping that in mind will help for an effective shift.
It starts with priorities and businesses need to establish what priorities are at the top. If the mindset and culture can be embedded across your security, IT, operations and financial areas, businesses will have a far better position of balance between what the business needs and what it requires. It’s about finding a balance and people just need to work with each other rather than in silos.
To really be successful with this transition, businesses need to have someone who is at the board level who is happy to lead and own this change. Often the best way of doing it is more like a snowball; start with a team and get the processes up to scratch and in doing so, accidentally prove that it’s better. This can then be treated like a virus that keeps spreading throughout an organisation. Flexibility of mindset, but the thing that underpins that is consistency. With consistency, you can make logical adaptions and monitor your progress. I think that’s one thing a lot of people don’t do. It drives scientific behaviour of what is going to change.
