Guy Daniels, TelecomTV (00:24):
Hello, you are watching the Cloud Native Telco Summit, part of our year-round DSP Leaders Coverage. I'm Guy Daniels and today's discussion looks at developing operational excellence. We want to discover how telcos build a continuous learning culture, set meaningful performance metrics and future proof their operations and I'm delighted to say that joining me on the program in alphabetical order are is Izzat Bamieh, who is VP of engineering at Wavelo. Francis Haysom, Principal Analyst at Appledore Research. Matt Rees who is Chief Technology and operating Officer for Neos Networks. Franz Seiser, VP, Tribe Lead, TDAT, Deutsche Telekom Teknik, and Odded Solomon, Director of Product Management at VMware by Broadcom. Hello everyone. It's very good to see you all. Thanks so much for taking part in our discussion. Well, let's start by talking about what organizational structures and leadership roles have proven to be the most effective for cloud native operations and is I'm going to come to you first. It's the first time you've been on our cloud native summit, so let me come to you for your thoughts.

Izzy Bamieh, Wavelo (01:47):
Thank you. Thank you so much. Yeah, we at Wavelo that we moved cloud native six years ago and we faced the challenge of how we can organize our teams in an effective way where we can ship fast and secure products to our customers and the most effective one, the most effective structure that we found is basically create small teams. Those teams have full control over their banded context and since now cloud infrastructure all managed by either by a third party or even if you manage it internally, it's all like infrastructure as a code. So everything is being way more simplified than before. So that team will basically have autonomy over everything related to their banded context and that team is responsible end to end from infrastructure code, continuous deployment, production and eventually of course documentation and sharing that knowledge with other teams.

Guy Daniels, TelecomTV (02:57):
Great. Izzy, thanks very much for sharing for that knowledge winners. Francis, I'm going to come across to you next. I mean what do you see in your conversations with the ecosystem telcos and vendors?

Francis Haysom, Appledore Research (03:10):
We recently actually had a great conversation in a podcast with Rudd of Stryker, of Swisscom and I think he highlighted two key areas that I think are really important. This organization, they fundamentally understood that whilst they were doing everything in a sort of cloud native way, they weren't getting the benefits and a lot of it was about just change, changing the priority, really focusing on domains, getting everybody associated with a domain in one team that is delivering to that one and putting a very lightweight architecture on top of it, not one that is imposed on everybody, but rather becomes something that enables each team to do things easier rather than reinventing the wheel all the time. So just very subtle changes and I highly recommend people listen to that one because there's a lot of information about how to change the structure and leadership role.

Guy Daniels, TelecomTV (04:10):
Great, thanks Francis. And it certainly seems to be working for Swisscom from what you hear and read. I'm going to come to all of our guests now, but Odded, let's come to you next. Again, first time on the cloud native Telco summit. So what's your advice?

Odded Solomon, VMware by Broadcom (04:25):
So when we speak to telcos, we notice that there is a siloed approach. Different teams on the IT side of the organization and some other teams on the network side of the organization are quite disconnected. I think breaking those silos and bringing those teams together, they will find out that the cloud native infrastructure is very common. The use cases are very similar. Yes, the operations are different, different requirements on how to operate under the maintenance windows, constraints and so on. But in terms of the cloud infrastructure capabilities, those are very common and my advice is basically to have the skills that we have in the IT team to go and work across the network side as well. So to converge the IT and the network DevOps teams, SLE, the cloud platform engineering, all those teams that are operating the cloud is infrastructure across the teams and this will drive more efficiencies within the telco organization. Telco organization is quite complex as we all know, and breaking the silos is going to be key to reduce the operational expansion.

Guy Daniels, TelecomTV (05:43):
Great, thanks for that Odded. Appreciate that and we'll talk to our two telcos on the panel. Now, Franz, I'm going to start with you. What can you share with us from your experience at Deutsche Telecom?

Franz Seiser, Deutsche Telekom (05:55):
Yeah, so what we did basically two things. The first thing we moved the architecture from vertical silos to horizontal layers. So we have on the bottom one team that really deals with infrastructure, so even we talk cloud native, we build our own private cloud, so someone needs to deploy servers, cable servers provide all the networking, so that's done by that team. They provide infrastructure but they also provide compute network storage. On the very top we have our automation teams, so they're in charge to build the complete automation layer, help the teams to integrate and discern, and I'm one of these, we provide the workloads. We have internally a picture that we compare that burger with hamburger, bottom patty cloud top petty bottom breadth, sorry is cloud top red is automation and I'm one of the petties in the middle. So we really provide and run the cloud native workloads. So it's three different layers and the team are interacting within the teams. We changed completely to an agile organization. So we have squats, we do quarterly plannings, discuss and prioritize per quarter what to be done and as we'll come in later question. Of course, the key to operate that smoothly besides a full automation can't run in a non way is of course making sure the teams are interacting properly and also in case of issues, you know how to involve teams, how to figure out where collaborative manner.

Guy Daniels, TelecomTV (07:47):
Thanks very much for detailing that process there really interesting. And Matt, let's come across to you. What are your thoughts here?

Matt Rees, Neos Networks (07:54):
There's definitely some common threads with what my fellow panelists have just talked about. I think in neo spots we did was exactly what some of the others described. We put together those product or service centric squads primarily. We did that in a matrix way. What we decided though 18 months, nearly 20 months or so ago was that maybe we needed to go one step further to really break those historical silos between design, deployment, engineering and operational teams. And we collapsed effectively our CTO organization and our COO organization together into my combined role. So what we now call tech ops, borrowing from the software development world of DevOps of that's been around for a lot longer. So really breaking down those boundaries, getting the teams not just working together in a matrix way but far closer together organizationally hierarchically as well. So that's where we would pull out, go one step further and really seen some acceleration of guests of that over the last 18 months or so since we broke down some of those historical organizational boundaries. So I think just one additional step that we've found is really successful for us beyond the matrix version of that.

Guy Daniels, TelecomTV (09:13):
That's really interesting. Tech ops is fascinating. I've got to look into that a little bit more. Thanks everyone for those insights. Really valuable contributions. Well let's move on to another question and again, it's a question that we hear from our viewers for the past few years. Just like the first one, how do we foster this culture of continuous learning, which is so essential and cross-team collaboration within our cloud native environments. Francis, can I come across to you to start us off?

Francis Haysom, Appledore Research (09:46):
Yes, certainly. I think at the center of this is really new KPIs and they're cross organizational KPIs. We've talked about bringing in domains or systems or solution areas and creating teams across all the way from planning network all the way through to assuring networks for example. You need to have KPIs which are owned by that whole plan. You need alignment also. You need to be capable of exposing and understanding the conflicts and compromises that each part of the team needs to make and that everybody understands why you're doing that. I think that's particularly the case when it comes to as we move to a cloud native approach where everything becomes increasingly software based software, software type timescales, you need to understand that it's never perfect software. We have a kind of break points in our existing processes often within hardware where we develop hardware very much offline and then we put it in the network. Software is a much more continual process. So everybody in that process needs to understand both the challenges of moving new software into production and also the follow-up feedback of that in delivery back into the development process. And that's something that's typically within a hardware network being broken up before. So building KPIs into that thing to create that bridge between operations and development. So everybody's seeing the compromises but also the benefits of that.

Guy Daniels, TelecomTV (11:32):
Thanks very much Francis, that's really helpful. Exactly the sort of question we get during our live audience q and a shows and I'll have more information about that towards the end of our panel. Let's go to Franz as well and pick up on this one about collaboration and continuous learning. Franz.

Franz Seiser, Deutsche Telekom (11:51):
So in the very first place I would say it's all about mindset of the people because we're talking about a fundamentally different way of building and running networks and the very first thing is I said we go from silos to horizontal and important pieces. Everyone. So everyone every layer needs to understand none of them can do it on their own. It only works if people collaborate. So yes, for example, my guys, we the service in front of the customer, so we provide mobile core network but if the infrastructure layer below us has a problem, nasic works and if the automation is not working properly on top as well as something needs to happen and always something needs to happen in the cloud environment wouldn't work either. So it's super important that people understand they only can succeed together and work together and figure out how to best collaborate people being in charge for the silo in the past with having your supplier, your favorite, I dunno, victim or the one you can always complain about, but now it's internal colleagues so you're in a completely different world and how you need to collaborate here as well together with the suppliers, but each and everyone really needs to be aware.

(13:09):
It only can be done together and it's not the one team being guilty and the other not. It's always a team effort and team effort across different organizational entities. So bringing people together, make them aware and work with them that they understand is one of the key ingredients to make this successful and also work to establish the proper operational roles. As said before, understanding case of issues, it's clearly defined who is going to be contacted in what cases. Again, how people have enough trust on each other that everyone is working with the same focus and the same overall goal. And that's of course providing always the best service to your customer.

Guy Daniels, TelecomTV (13:56):
Absolutely. Fran, communication and collaboration, so critical. Let's go across to Matt as well Matt, let's get your insights.

Matt Rees, Neos Networks (14:03):
I think recognizing that all individuals have their own preferred way of learning. I think when you're trying to encourage people to cross pollinate from maybe they're a network expert and you want them to start to pick up coding skills as an example, what you don't want to do is put them off by not giving them access to the materials and community I guess that they can turn to for support if they get stuck whilst they're on that development journey. So I guess what we try and promote is that community feeling where you've got your experienced coders in this context are available and willing and supporting those people who are trying to cross Gill from being a historical routers and switches person to somebody who can write Python code to develop that use case. So I think making sure that people are not just, it's the education and it stops there, but there is that community that provides them the support resources. There's no come back to that culture point. There's no protective, this is my skillset and I'm the expert and I want it to just be me, but people are willing and in fact eager to share their knowledge and support people on that journey as they cross skill. I think that's the key really is building that openness and that support network for people who are learning something different.

Guy Daniels, TelecomTV (15:24):
Yeah, absolutely. Matt, thank you very much indeed. Yes, we've been following the need for years now to focus on the culture of the organization. Odded, I'm going to come to you and then come to Izzy, but Odded, let's get your thoughts.

Odded Solomon, VMware by Broadcom (15:36):
Yeah, so I think that the blameless postmortem culture is what we touched with the previous responses. So I just want to emphasize that this is the evolution of DevOps into SRE into platform engineering and this is something that creates a stronger sense of ownership of the platform. So people, engineers, operations, people that are interacting with both the application layer and the infrastructure layer. Again, it's not about blaming where the issue is, it's about working together and the industry, the ecosystem as a whole needs to provide tools to help those two distinct team to break the silo. So basically having the application team and the cloud infrastructure team working together, understanding during triage where the issue is learning from that improving, enhancing the overall quality of the solution. This is really the aim when you go cloud native. Just an example of the approach that we took in our company to build this capability is to configure the infrastructure when we deploy the application. So if in the application you describe the set of features and functionality that the application needs to be configured with in order to support the workloads, when you do it, when the application is deployed and you're doing it with the change management GitHubs pipelines and doing it properly, you can really increase the time to market, reduce your meantime to resolution of issues and this is really the way to go. The ecosystem needs to help and support telcos in pursuing this way of working.

Guy Daniels, TelecomTV (17:25):
Great, thanks very much for that Odded and we'll come across to Izzy as well.

Izzy Bamieh, Wavelo (17:30):
Yeah, I agree with everyone, but in addition to what everyone said, we at Wave Law we basically created a team structure that's a bit diverse. When I say diverse, I mean diversity from background, skillset, you name it, even geographically diverse, that diversity automatically fostered a sense of learning and continuous learning. In addition to that, we created this new concept called the practice lead. That practice lead basically spans across multiple teams and that practice lead head is basically a hat that anyone can wear. The goal of that practice lead is to continue become an advocate for continuous learning and conducting lunch and learn, conducting a pairing sessions and we have a practice lead for every layer in our team from network engineering to qa, software engineering, agile practices. And this the key here for that practice lead. It's a hat that hat can be worn by anyone in the company and it keeps that getting path one point person to another. Right. So that was really helpful and for us to foster the continuous learning culture

Guy Daniels, TelecomTV (19:01):
You see. That's great. It's great hearing everyone's experiences and thoughts in this area really useful. Well let me ask a follow-up question. It's related to this previous question and we touched on it slightly in some of the discussion there, but how do tecos best engage and manage the partner ecosystems, including the open source communities and also startup companies? Odded, I'm going to come across to you, what would your advice be here

Odded Solomon, VMware by Broadcom (19:31):
In general? I think that the telcos should move into a platform led cloud first approach from a vendor led or siloed approach. I think that choosing the cloud first will attract more developers and in that sense we can treat startups, smaller companies and big NS like Ericsson, Nokia and others as cloud applications, cloud enabled applications that are running on top of the infrastructure. Now cloud infrastructure is working with the open source community. We are contributing a lot into the open the community, the upstream Kubernetes specifically, I can talk about my organization, but in VMware we are contributing heavily into Kubernetes, we're influencing the roadmap. We deliver a lot of the functionality that is being consumed by the NS but also by other vendors in the IT side as well. It's all about open APIs, SDK, that made available for easy consumption of the infrastructure. And the key here when we talk about the mobile core is to have a joint certification program with the larger ns. Okay. So this is something that we are working on and the ecosystem as a whole is investing quite a lot in making sure that those big nps are well integrated with the platform to provide this seamless as possible integration and the quality that the telcos are expecting when we deploy in production. So the certification program is very, very important and the way that those nets are consuming the infrastructure needs to be tested, validated and this is what we are working on.

Guy Daniels, TelecomTV (21:30):
Thanks Ade. That's very encouraging to hear. And Francis, what are you seeing? What's your advice? What advice do you give when you talk to your clients?

Francis Haysom, Appledore Research (21:41):
I think the important thing is particularly looking at startups in OS communities is that startups need partnership and mentoring. You will kill them with endless underfunded proof of concepts and science projects. And I know from history I had the privilege of being part of a successful startup in the two thousands, but that was in an environment of comparative Telco Richards and today's heavy procurement and risk management will kill startups and frankly within the open source community is largely ignored. Open source is about collaboration, it's about participation. It's not about immense closure of risk unless you're buying open source already prepackaged. So I think my advice here particularly with partnerships is if you want to leverage innovation or changed behavior from both open source and startups, you have to find a way around this. You have to be managing some element of risk. You have to be managing some element of investment in that and the management of the risk of that investment with people. Otherwise the open source communities I think largely ignore Telco.

Guy Daniels, TelecomTV (23:01):
Yeah, absolutely they do. Good advice there Francis, thank you very much. Well let me move on. Francis was talking there about endless pox. Let me move on to the challenges that telcos face in scaling their processes and scaling their teams as their cloud native operations mature. So what are these challenges when they scale and how should they overcome them? Matt, can I come to you for your thoughts?

Matt Rees, Neos Networks (23:31):
I think there's several. I think finding and the right talent journey I guess to get people on as you make the transition here from as we talked about earlier, people moving from traditional networked roles to more hybridy roles and into the roles that you're going to need for the future. There's definitely a mentality shift, a mindset shift that you have to work starting with culture on getting people willing to come on that journey with you so they're not going to be setting that whole, this is the way I've always done things, fear of change mindset and promote people's willingness and desire to learn and shift into new roles so you can make that transitional journey, especially for an organization the size of Neos as opposed to some of our larger rivals in the UK market. We aren't going to be able to bring in large numbers of people who are already suddenly made that shift into the cloud skillset.

(24:30):
We could have to do a small amount of recruitment there and bring people on a development journey with us and it's finding that I guess the right balance of those two things, but making sure that you promote that culture of making the transition. I think you've also then got to make sure that you don't forget about the processes that sit behind all this stuff. So it's great having all of your new technology and automation processes that you, automation software that you've developed, but you need to make sure that things like your change management processes keep up with that. So shifting towards again those agile mindsets, the DevSecOps principles of having a continuous improvement, continuous development pipelines appropriate controls around release to production, but maybe some of those historic onerous processes with big heavyweight cabs and all that sort of stuff are not appropriate for all of those releases. So I think it's finding those right balances as you transition from an old monolithic mentality to a more cloud mentality. You've got to make sure that you take your people with you but take your processes with you as you make the technology shift.

Guy Daniels, TelecomTV (25:43):
Great advice. Matt, thank you very much for sharing And Francis, let's come back over to you. What would you say on the challenges and solutions to scaling the operations? I think

Francis Haysom, Appledore Research (25:55):
I just could highlight a couple of change mentalities which will help in the standards. We need to understand that cloud native is very much a software process and to a degree we inherit a kind of hardware centric standards and dare I say it, generational view of the way technology is developed. So the balance needs to shift if you want the dynamic team, you need to evolve your traditional view of standards and I'd highlight one example is for example with Rakuten and Open Rat, it's very easy to see that Rakuten is a trailblazer for open ran, but actually Efan was a late joiner of open ran as a standards thing. What it was you doing was using open ran as a trailblazer to enable its business. So looking to shift the balance in terms of seeing things as standards and waiting for standards to be set and seeing it much more as a dynamic thing. The other thing that will enable scale I think is understanding the cloud native needs to work across the whole supply chain. It's no good just dealing with cloud native within the operator or cloud native within the suppliers to the operator. It needs to be seen as a continuum across them and working across them. And again, that's a very important thing, building the trust across those supply chains. If you don't have that trust but however agile and cloud native you are, it's very easy to revert back to the traditional sort of waterfall and delivery mechanism.

Guy Daniels, TelecomTV (27:31):
Yeah, absolutely. Thank you very much Francis and Franz comments from you on the challenges and solutions of scaling operations?

Franz Seiser, Deutsche Telekom (27:40):
Well actually defining we have is that scaling in cloud native is much easier than scaling In the legacy world, SU don't do cloud native without full automation anyhow. It doesn't matter so much if you are deploying on 3, 6, 12, I dunno how many sites because that's all done by scripts and automated anyhow. Whereas in the past a lot more manual work means more manpower means more time was involved. So if you have designed everything and that's what we're aiming for is designed for scale. Scaling out in a cloud native environment is not leading to significantly enhanced demand of workforce. So I said in that sense, cloud native is much easier from a scaling perspective and also to the other comment we also found recently as of today, it's easier to find new talented new people in the cloud native area versus, and I had a concrete example just few weeks ago versus getting people replaced, retired or whatever. In the legacy world, it's super difficult to find people in that area. Once you talk about 5G core and other highly software centric topics, it's much easier to find people who are join on these topics.

Guy Daniels, TelecomTV (29:08):
Great, thanks France. A couple of great areas there that were well worth highlighting here. Thanks very much everyone. Well got time for a final question. So I have to bring up ai, how are telcos incorporating AI driven insights and processes into their long-term operational roadmaps? And Izzy, can I come across and get your views?

Izzy Bamieh, Wavelo (29:34):
What we did here at Wave Law is that before even this AI movement started, we designed our system and our infrastructure to make sure that everything emits events and we put all of these events in one giant event stream and then when the machine learning models started to mature and foundational AI models started to mature, we were easily able to push this data to make something useful out of it. Right now the challenge becomes like what do we want to do? What do we want to automate, right? And there's so many cool ideas are in development at this moment. Some of it is around anomaly detection, some of it is around automating part of your incident response. Some of it is like a quick resolution to incidents and you name it. There's so many things that can be done with, but the key here is that you should start with making sure that everything you have emits events. This would help you to run machine learning and AI model at scale and real time.

Guy Daniels, TelecomTV (30:52):
Great, thanks very much for that advice Izzy. And we'll go across to Francis as well. Francis.

Francis Haysom, Appledore Research (30:57):
Yeah, I think we're just about to release an apple door agentic AI tracker and as part of that we're looking at just general adoption of AI in terms of roadmaps. The honest answer at the moment is what we're seeing is that operators are looking at ai whatever term is in a very siloed and process specific thing. So you're seeing a lot of work into things like chatbots enabling better customer experience for example. And you have to say you've already got examples of AI in things like root cause analysis for example in networks. What we're not seeing is a lot of what I would term is next step in terms of network operations beyond the trials that are proof of concepts and the trials at this stage. And I think the thing that's most important for the roadmap in terms of AI is that building of trust. I think everybody's got a long list of ideas of what AI can be used for, but there's a high degree, there's a high resistance in terms of trusting the data to support that. And I think in terms of building things into the roadmap, a lot of it will be coming about how do we make sure we've got the data, how do we make sure we brought data together and how are we building more and more accurate data on that one? That's the thing that's going to enable the log jam to be broken.

Guy Daniels, TelecomTV (32:27):
Great, thank you very much Francis. I look forward to seeing your agen tracker when it gets released. Odded, let's come across to you as well.

Odded Solomon, VMware by Broadcom (32:38):
So the way I'm thinking about this is the telcos are split between two major goals. There is the goal to reduce the opex and CapEx and basically be more efficient and leverage AI for that. And there is the other track which is what are the new use cases that can be enabled to increase the revenue. When you split those two use cases, you can actually put them on a chart and see what are the things that are easier to achieve the lower hanging fruit that we already see implemented, like agents that are helping on customer support, agents that are helping with reading logs and increasing the observability and the other things that to the point made earlier that it'll take longer to trust fully autonomous networks. This is like the holy grail. Obviously everyone is following that, but developing the trust is going to take some time with telcos, but by then we are building infrastructure to enable those use cases. When you look at what we have been doing recently in VMware by Broadcom, we've been investing both on the software side and the hardware side to enable AI at scale and the cloud infrastructure support new GPU capabilities. GPU virtualization that we see will be very useful for telcos to run generative AI workloads to run agents and basically prepare telcos for the next wave of network demand that is driven by the AI workloads.

Guy Daniels, TelecomTV (34:26):
Thank you very much for that. And we'll go across to Matt as well. Matt, can you share what you might be doing with AI driven insights or processes?

Matt Rees, Neos Networks (34:34):
Yeah, I think there's some common threads. I think with some of the things that Izzy talked about and Francis has just talked about as well. I think my view is that lots of the large brands software houses in CRMs and in ITSM applications have been very quick to market with their AI agents that really help you in the business operations world of managing customer experience and whether that's the chat bots or feeding information around customer interaction on sale. I think what I've probably seen is that we've much as for a long, long time, I think the whole industry has been working on the gathering data from your network so that you can predict failure of links or hardware so that you can self-heal. I guess what I've not seen so much of yet is some of the legacy or the incumbent network vendors having introduced some of those AI capabilities into their native software stacks quite so quickly as the BSS guys.

(35:36):
But I think we still continue, I think, across the industry to use those foundations that we built of gathering events, gathering data and detecting pattern. So I think we can still do that ourselves without relying on the vendors. I definitely think that we'll see, we're starting to see we the first signs of those network vendors bringing that into their native management software, the new AI releases of that and the features we'll bring that will help accelerate it. I'm hoping that we can do that at an appropriate cost point and that all we'll have to continue to do that ourselves through our tech ops developing our own code to continue to build on that. I think say the BSS going was so much quicker out of the gates with I think releasing their AI agents than some of the network vendors have been that I'm hoping they work too late.

Guy Daniels, TelecomTV (36:33):
Yeah. Thank you very much, Matt. Well look, thanks very much everyone. We do have to end our discussion there for now a lot for our viewers to digest and I'm sure they're going to have a lot of questions we'll be sending in questions later on based on what we've been talking about. But for now, thank you all for taking part in our discussion and if you are watching this on day two of our cloud native Telco summit, then please as I say, do send us your questions and we'll answer them in our second and final live q and a show, which starts at 4:00 PM UK time and the full schedule of programs and speakers can be found on the telecom TV website, which is where you'll also find the q and a form and the poll question For now though, thank you for watching and goodbye.