Learning Library

← Back to Library

A Billion Software Engineers by 2027

Key Points

  • Experts on the show predict a surge to roughly a billion software engineers by 2027, driven by widespread code‑assistant tools and the rise of “silicon” (AI) coders alongside humans.
  • GitHub’s recent blog data shows a notable increase in developer numbers, especially as AI‑powered assistants like Copilot make coding more accessible.
  • Python’s explosive growth is highlighted as a key factor, spurred by its dominance in data‑science and machine‑learning projects.
  • The panel sees this trend as a democratization of programming—everyone, from hobbyists using Scratch‑style platforms to professionals, will be able to write code without traditional training.

Full Transcript

# A Billion Software Engineers by 2027 **Source:** [https://www.youtube.com/watch?v=V6vxTXrDCrA](https://www.youtube.com/watch?v=V6vxTXrDCrA) **Duration:** 00:39:41 ## Summary - Experts on the show predict a surge to roughly a billion software engineers by 2027, driven by widespread code‑assistant tools and the rise of “silicon” (AI) coders alongside humans. - GitHub’s recent blog data shows a notable increase in developer numbers, especially as AI‑powered assistants like Copilot make coding more accessible. - Python’s explosive growth is highlighted as a key factor, spurred by its dominance in data‑science and machine‑learning projects. - The panel sees this trend as a democratization of programming—everyone, from hobbyists using Scratch‑style platforms to professionals, will be able to write code without traditional training. ## Sections - [00:00:00](https://www.youtube.com/watch?v=V6vxTXrDCrA&t=0s) **AI's Impact on Future Engineers** - A panel of experts debates whether AI will boost or reduce the number of software engineers, referencing GitHub data showing rising developer counts from tools like Copilot and Python’s surge in data‑science use. ## Full Transcript
0:00does the rise of AI mean that there will 0:01be more or fewer software engineers in 0:03the future Chris Haye is a distinguished 0:06engineer and CTO for customer 0:07transformation Chris welcome to the show 0:09what do you think a billion software 0:12Engineers by 2027 wow 2027 okay uh 0:17Farney is a senior partner Consulting on 0:19AI for US Canada and Latin America 0:22uh what's your thought everybody will go 0:24from becoming a programmer to being a 0:26pro at grammar I will ask you to explain 0:28that more in just a moment Kar El mcra 0:31is a principal research scientist and 0:32manager at the AI Hardware Center uh Kar 0:35welcome um what do you think I think 0:37it's going to be a different breed of 0:39software Engineers that we will be 0:41seeing all that and more on today's 0:43mixture of 0:45[Music] 0:48experts I'm Tim hang and welcome to 0:50mixture of experts each weeke brings you 0:53the analysis debate and banter that you 0:55need to stay ahead of the biggest 0:56developments in artificial intelligence 0:58today we're going to cover for cyber 1:00security and the launch of search gbt 1:02but first let's talk about software 1:04engineering um there's a fascinating uh 1:07blog post that came out from GitHub uh 1:10the other week um basically reporting 1:12out some data that from their perch 1:15github's reporting that there appears to 1:17be a rising number of developers um uh 1:21driven largely by tools like co-pilot um 1:24and second they also point out that 1:25Python's incredibly uh becoming a really 1:27really popular language driven largely 1:30bu data science and machine learning 1:32applications and this is super 1:33interesting to me and this is one of the 1:34reasons I wanted to bring it up as our 1:36first uh story of the day which is had 1:39you asked me I would have said look 1:40where code assistance is going we're g 1:42to eventually just replace all the 1:43software Engineers there's going to be 1:44no wor no more software engineers in 1:46about a decade um and maybe Chris I'll 1:48toss it to you first because your 1:49prediction is that if anything we're GNA 1:51have way way more software engineers in 1:54I think 2027 right so literally like 24 1:56months from now um why do you think that 1:59I think if for two reasons number one is 2:02with code assistant being everywhere and 2:04with things like jet gbt large language 2:07models pretty much in the everyday 2:09person's hands everybody can become a 2:11coder so you don't need to go and pay 2:13money to go and get somebody to do that 2:15you can literally have a go yourself and 2:17I think that is just going to open up 2:19this sort of democratization of coding 2:22that we've all kind of hoped for and I 2:23think more tools will come in like uh 2:26you remember scratch from kind of MIT 2:28then I think we're going to see more of 2:30that style side of things and everybody 2:32is going to become a coder the other one 2:34is you didn't say in your question Tim 2:37whether they had to be humans did you so 2:40the carbons and the silicons and there's 2:42going to be a whole bunch of silicon 2:44coders to match us carbons so uh when I 2:48multiply that up by 2027 there's going 2:50to be a billion buddy okay all right 2:52that's really interesting yeah I guess 2:54kind of what we're talking a little bit 2:55about is almost like that the I guess 2:57the question is whether or not like the 2:58the the job coder or the category code 3:01software engineer is really going to 3:03make sense in the future like it almost 3:04feels like no one's like Oh I'm a word 3:06processor right like kind of everybody 3:08knows how to write um show I know your 3:10response seemed to kind of suggest that 3:12you think some of the some of the skills 3:13you'll need are going to have to change 3:15yes absolutely I think uh all of us will 3:17become pro at uh writing good grammar 3:20and the way you ask a question and how 3:21you describe what you want to get done 3:23uh it's the a good technical PM does a 3:26really good job at explaining what 3:27exactly they need so that the developer 3:29can go and execute the code to the 3:31vision of what the PM had right so I 3:33think that's going that's going to shift 3:34quite a bit let me just spend a minute 3:36on just appreciating how far GitHub has 3:39come we just you just refer to their 3:41annual report that talks about the 3:43GitHub all the numbers uh last week we 3:47were at the at their Pig GitHub Universe 3:51uh event and this is where we as IBM 3:53sponsored that as well just to give you 3:56a sense of how far they have come GitHub 3:58is the world's biggest repository like 4:0090 90% plus of all Fortune companies use 4:03it 98% developers what not we're at 4:05about what 100 million developers Plus 4:07on GitHub today Chris not quite at the 4:09billing that you want there to be but in 4:11the last like N9 10 years now this 10th 4:14year they've been running this they've 4:15had like what six like close to 70 4:18million GitHub issues people have solved 4:20like 100 like almost 200 GitHub polls 4:23like 300 million plus projects and 4:25whatnot right the way I look at it open 4:28source is is the biggest team sport on 4:31Earth it's not soccer it's not football 4:34it's open source as the biggest team 4:35sport right it has been crazy growing so 4:38when you hear from from Tom the CEO of 4:41of GitHub they're giving you actual 4:43stats of what they're seeing with people 4:45developing more and more and he's very 4:47true right to to say that AI the 4:49threshold of creating code for AI 4:52engaging with guub repositories and 4:53trying it out downloading it 4:54contributing back to it IBM has been a 4:57big proponent of having a very open 4:58community had a really good relationship 5:00with GitHub and now that the GitHub is 5:02opening up quite a bit it has Cloud 5:04models and Google models that can be 5:06leveraged in addition to all the open AI 5:08models I think this is just an 5:10Unstoppable Force right now in the 5:11industry and more and more programmers 5:13will have access to tools that we just 5:15could not imagine we had a couple years 5:17back yeah I think one of the most 5:18interesting things in the report that 5:20they did was also that it seems like the 5:23geography of software engineering is 5:24changing right that there's like a lot 5:26more coders from they're seeing from the 5:28global South come online on GitHub um I 5:31guess sh do you think that's related to 5:32code assistant or I'm kind of curious 5:34about how you feel see like the role of 5:36these assistants in even potentially 5:37kind of like broadening like the 5:39geographic scope of of who gets to be a 5:41software engineer yes so I I spent a lot 5:44of time with Latin America clients as 5:45well u in Americas and I see a lot of 5:48centers developing where all of a sudden 5:50the threshold of being able to have 5:52economic benefit in the region has dis 5:55plemented so people can go create code 5:58and go contribute to to other other 6:01locations other countries and 6:02increasingly so a lot of my clients are 6:04starting to build their Latin American 6:06presence the time zone helps in the US 6:08as well but just the access to tools and 6:10being able to create in every language 6:12right now now I have an opportunity to 6:14know Portuguese and Chile and be able to 6:16code and get some assistance in 6:17Portuguese while I'm creating code right 6:19that did not exist earlier so the 6:21barriers have come down significantly 6:23and you see a a higher threshold this is 6:25one additional thing I would I would add 6:27to this we should not we should also 6:29look at the way energy uh movements 6:31happen across the the world right if you 6:33look at countries like Chile or Latin 6:34America there's a lot of energy that's 6:36been created there and you want the AI 6:39models to be trained closer to where 6:40there's energy because energy 6:42consumption is going to be so much I 6:43would anticipate more pull towards Latin 6:45America or centers where there's energy 6:47production in Surplus it used to take a 6:49lot to move that energy from Latin 6:51America to say serve customers in the US 6:53now the AI models will be will be 6:55created closer to where the energy 6:56sources are counter I want to kind of 6:57turn to you is you know building on what 6:59chit just talked a little bit about is 7:01that um you know I think when you 7:03responded to the opening question you 7:04said well it's going to be more about 7:06like asking the right questions um and I 7:08think that's like one interesting is 7:10like item here to kind of pull on one 7:12thread to pull on is maybe actually in 7:14the future it's actually we're going to 7:16have a lot more technical PMS than we 7:18really will have software Engineers 7:19because it feels like the role that 7:20people are increasingly having is 7:22they're kind of managing this agent that 7:23does the coding not really doing 7:25software engineering themselves and I 7:27guess kind of I'm curious if like the 7:29right way to think about this actually 7:30is we're going to just have a lot more 7:31PMS in the future yeah I see of course 7:33you know the the skills will be changing 7:36shifting and this for example co-pilot 7:39what it's doing is deifying coding for 7:41people without uh formal training uh 7:44turning more people into kind of Citizen 7:46developers so this means that 7:48professionals from diverse Fields such 7:50as data analysis design Finance uh 7:53Healthcare Etc can now use code to build 7:56custom tools without extensive studying 7:58or training or syntax Etc and this kind 8:03of heading towards a word where basic 8:06coding becomes as common as using 8:08spreadsheets or even presentation 8:10software so just learning it's kind of 8:13they're trying to be prompt Engineers 8:15also but specifically designing good 8:17prompt for software engineering so I 8:21think it's also time to start 8:22reimagining what's the right developer 8:24workflows here for experienced coders AI 8:28can handle repetitive Tas letting them 8:30focus on higher order problem solving 8:33for example and this might alter the 8:36skills expected in software developments 8:38with these with coding transitioning 8:41more from syntax heavy work to strategic 8:44thinking and Architectural design so I 8:47think those really would be good skills 8:49to start acquiring not really focusing 8:51on the syntax but more on how do you 8:54build systems how do you design systems 8:57how do you put them together and then 8:58using the C Pilots to help do the syntax 9:01work I think this also could have 9:03implications even for Education right 9:06now curriculums they focus a lot on 9:09syntax and uh so if AI can assist with 9:12coding should really Educational Systems 9:14shift from focusing on syntax to broader 9:16problem solving or even collaborative 9:18design especially as mentioned open 9:21source it is actually the the biggest 9:23team sport right now so I think 9:26acquiring those skills how do you 9:27collaborate you know do all these things 9:30uh do PRS and learn how to work in a 9:33team going to become a really important 9:36skills in the future so I think the 9:39traditional computer science curriculum 9:41really need to adapt emphasizing 9:43creativity ethical coding practices 9:46Advanced debugging and also 9:48collaborative coding yeah for sure and I 9:50think Chris I mean it kind of puts a 9:51tough question to you I mean your title 9:53is distinguished engineer uh so you 9:55spent a lot of time getting really good 9:57at the software stuff right um 9:59but uh you know I think if a kid 10:02approached me today and say should I be 10:03a software engineer should I just tell 10:05them not to like it kind of feels like 10:06where we're headed is like is there any 10:08more value in actually learning how to 10:10code anymore right I think is the 10:11question I want to put to you no they 10:13should go and play soccer ball or 10:15something like that you know yeah yeah 10:17no no I'm no I 10:19think so I think the question I would 10:21say is what happens when it goes wrong 10:23right so if if we really think about 10:26history of software programming right 10:28it's you're kind of back in the kind of 10:29the Punch Cards and the ones and zeros 10:31and then the Assembly Language came 10:33across and then you know and then see I 10:36mean there was a whole bunch of other 10:37languages for try Etc but then it really 10:38kind of took off I would say from the 10:40kind of C onwards which was which is 10:42very close to Assembly Language and then 10:44the abstractions got higher up and now 10:46we're at python Etc and then you know 10:48now got rust blah blah blah so the 10:50number of languages are increasing but 10:52it's abstraction layer after abstraction 10:54layer after abstraction layer we've went 10:56from Hardcore kind of Punch Cards to 10:59assembly to lowlevel languages to 11:02garbage collected languages to higher 11:04level languages blah blah blah blah and 11:07again all I would say that's happening 11:09here is we're moving to another level of 11:11abstraction and that level of 11:12abstraction is natural language um I 11:15think it will be better because with 11:16agents we'll have tools Etc but you're 11:19still going to want to know the 11:20fundamentals because what happens when 11:23you get a bug and and it can't fix it 11:25are you are you going to be like the 11:27Homer Simpson you're just going to be 11:28hitting the keyboard G try again try 11:30again try again or or you're going to 11:32have to go oh my God I'm going to have 11:34to I'm going to have to use my brain how 11:36dare you make me use my brain so I think 11:38I think the fundamentals are still going 11:40to be there I see this becoming a higher 11:42level of abstraction now don't get me 11:44wrong if the models become good enough 11:46at some point then there may be a 11:47different abstraction where models may 11:50have their more native language Etc and 11:52that that's a whole different discussion 11:54but I think I I see this as an 11:56abstraction because we need we need to 11:58explainability we need the reasoning 12:00somebody's going to have to maintain 12:01this and look at it and you can't be 12:02fully dependent on the AI um I do want 12:05to address one thing though Tim on that 12:08GitHub report like Python and we 12:11mentioned python there being we didn't 12:12talk about that aspect all that much so 12:14yeah yeah python being the most popular 12:16language I just want to point out one 12:18thing right and I love all languages I 12:20love python but when number two and 12:23three are typescript and JavaScript 12:26which are effectively the same language 12:28my friends and more JavaScript like 12:30people are becoming typescript people 12:32you know if you add the two things 12:35together who's number one again I mean 12:38yeah I had the same reaction I mean I'm 12:40I am a python die hard but I do feel 12:42like that was a little a little bit 12:44funny in the counting if I might add I 12:46think there are also some risks here uh 12:48there are potential risk for AI created 12:51code especially as more code is 12:52generated by AI quality control becomes 12:55a concern here how do we ensure AI 12:57generated code is secure efficient 13:00maintainable so there is also the risk 13:02of overreliance On Tools like co-pilot 13:05which could lead to a drop in 13:06fundamental coding skills among you know 13:09the new programmers so of course you 13:11know there are lots of advantages here 13:13in terms of 13:14democratizing having more developers uh 13:16lowering the bar of entry and things 13:19like that but we we shouldn't also 13:21ignore the risks that will come with 13:23this especially around quality assurance 13:26control ethical consideration security 13:29and also when things fail so can we 13:33ensure that we have skilled programmers 13:35or people like Chris hey mentioned that 13:39no bug and figure out what's going wrong 13:41or we will have less skilled people in 13:44those fields so what's the right balance 13:48here yeah I think it's always going to 13:49be this tricky balance between kind of 13:51you know democratizing making it 13:53accessible making it usable and then 13:55kind of like the Reliance on these 13:56abstractions um my mom who was like a a 13:59coder when she was before her retirement 14:02has a story about like in her early days 14:04like carrying a bunch of Punch Cards to 14:05the computer and then like dropping the 14:07Punch Cards everywhere and it basically 14:09like and her having a good enough sense 14:10of the program to basically reassemble 14:13the program like physically by the cards 14:15and I was like that is like a level of 14:17diligence that like modern Engineers 14:19just would not be able to accomplish so 14:23but obviously we are happy that we've 14:24moved past the punch card era for sure 14:31I'm going to move us on to our next 14:32topic uh there was a great and very 14:35interesting story that kind of follows 14:36on I would say a sequence of stories 14:38we've had on Moe for the last few weeks 14:40which is thinking a little bit about the 14:42application of AI and specifically kind 14:45of agents to the computer security space 14:48um so Google did a blog post from their 14:50security project project zero that 14:52basically reported that they have a 14:54cyber security agent called Big Sleep um 14:57that was able to find a vulner ability 14:59in SQL light which if you're not 15:01familiar is one of the most widely used 15:03kind of database engines out there and 15:06this is a really interesting story 15:07because at least by their accounting 15:09this is kind of one of the first 15:10instances in which an agent was able to 15:12find sort of a genuine vulnerability in 15:15the wild in a code base that is kind of 15:17like widely used and so in some ways 15:19it's almost kind of like a a real kind 15:21of hello world demonstration that we 15:23might one day be able to use these 15:24agents for uh identifying um real world 15:28vulnerabilities and making our systems 15:30um safer and so I guess maybe Chris I'll 15:33I'll kick it to you to kind of kick us 15:35off on this topic but you know I think 15:37the first thing I think a little bit 15:38about is is this the beginning of just 15:40kind of a new era like we will just 15:41start to see agents play a bigger and 15:43bigger role in making systems more 15:45robuster is this still kind of in the 15:47realm you think of like the toy project 15:48right like we're still going to be a few 15:50years off before we we live in that 15:52world no I I think we're already in that 15:55world I and there's a couple of things 15:57about the big sleep thing the first 15:59thing is if you give agents access to 16:01tools and then you get them to follow 16:03patterns um then the agents are going to 16:06do a pretty good job so if you think of 16:07cyber security you know go fix me this 16:10bug go identify this pattern go find me 16:14what ports are open and on a firewall 16:16these are all things that agents can do 16:18today now if we look at the big sleep 16:19one and I and I do want to caution this 16:21because when I read the paper there the 16:23thing that they did is they took an 16:26existing vulnerability that existed on 16:28on that code base and then they got the 16:30agent to go search the PRS and say hey 16:33go find me another vulnerability of this 16:36style that matches this pattern um that 16:39wouldn't have been patch yet and then it 16:40went and found that so as much as it's 16:44like by my understanding as much as um 16:47the agent discovered a vulnerability on 16:49its own at the same time it's kind of 16:51pattern matching and was prompted and 16:53directed to go and find a bug of that 16:56similarity and and that is completely 16:59within today's technology um you know 17:02agents and models are really good at 17:04pattern matching and if you give them 17:06access a large enough codebase by tools 17:09Etc you access to PRS and the commits 17:11they're they're going to be able to do 17:12that um are they quite at the stage of 17:16being able to find a whole new class of 17:18vulnerability that is completely 17:19undiscovered and not prompt and 17:21patterned in itself I don't know yet I 17:23think we're maybe a little bit off that 17:24but I don't think we're too far away 17:26from it yeah pretty interesting cter 17:28Maybe uh to bring it to you next I mean 17:30because you think a little bit about the 17:31kind of risks around all these 17:33Technologies uh you know it seems to me 17:36right like that like you're going to use 17:37this for security but also like the bad 17:39guys will get access to these agents in 17:41well and it seems like very 17:42straightforward to be like I I have this 17:44vulnerability find it elsewhere in this 17:46code base is also exactly the kind of 17:48same thing you need to do if you were 17:50going to sort of harm these systems um 17:53curious about how you see that kind of 17:54cat and mouse game playing out like does 17:55the defense have the advantage right now 17:57do you think the the offense is 17:58eventually going to have the advantage 18:00kind of just what that balance looks 18:01like um as these systems become more 18:03sophisticated yeah that's a very good 18:05point of course as big sleep or other 18:08similar system they're strengthening 18:09defense with AI agents so they're 18:12revolutionizing vulnerability testing 18:14allowing continuous autonomous scanning 18:17that adapts to new threats and this this 18:19is especially beneficial in complex 18:21systems or complex environments like for 18:23example Cloud infrastructures where 18:25we're doing all these manual monitoring 18:27is very inefficient 18:29and security teams could be empowered 18:32and act faster on these emerging 18:35vulnerabilities and reducing the attack 18:37window however at the same time there's 18:39also this threat of offensive AI so Aid 18:42driven security tools can also be a 18:44weapon in the wrong hands just as 18:46Defenders can use AI to preemptively 18:49catch vulnerabilities attackers could 18:51also use similar tools to identify 18:53exploits at scale so this creates this 18:56potential AI like he said arms rate in 18:59cyber security where the line between 19:01defense and offense is very thin yeah I 19:04think what's so interesting about it is 19:06it also suggests kind of eventually 19:08we're going to see a whole kind of dark 19:10criminal ecosystem which kind of M 19:12mirrors the the kind of one that we have 19:14publicly like that there will be 19:15basically like a a criminal Lambda Labs 19:19right where you can like kind of run all 19:20these agents um you know completely free 19:23and and for criminal purposes um and 19:25it'll be really interesting to see how 19:26that kind of ecosystem evolves because 19:28you know people who want to use these 19:29agents for bad purposes will sort of 19:31need the same infrastructure that you 19:33know the the people doing cyber security 19:35are are engaged in yeah so I think 19:37that's why maybe some ethical and 19:39Regulatory here challenges are will will 19:43need to be resolved you know with this 19:46rapid development of AI Bas security 19:49there is this call for framework to 19:51ensures also responsible users how do 19:53you protect these infrastructures and 19:56tools uh so government for example and 19:59government uh cyber cyber Security 20:01Experts they need to be tasked with 20:03creating also ethical guidelines and 20:04regulations to balance the benefits of 20:07things like big big sleep with its 20:10potential misuse also yeah um let me 20:13give you a client U perspective on this 20:15we do a lot of work with our clients on 20:17cyber security we have a whole Security 20:19Services team with an i Consulting it's 20:20been doing an exceptional job with 20:22clients we also partner very heavily 20:24with our partners like Palo Alto to do a 20:26lot of cyber security work with them and 20:28we leveraging generative AI models and 20:30AI models quite heavily in that 20:32partnership as well U there it's a 20:34two-way street it is AI helping drive 20:36better security and as the reverse how 20:39do you secure the AI models themselves 20:41right if you look at the three different 20:42steps that our clients go through the 20:44securing the actual data that went into 20:46the models securing the model itself 20:48from cyber attacks and then the usage 20:50itself how do you prevent misuse of the 20:52model when it's in production right so 20:54there's across all these three different 20:55buckets we've done quite a bit of work 20:57in creating AI models that prevent and 21:00detect and can can counter the serial 21:03imp attacks and things of that nature we 21:05had recently released our Granite series 21:07of models Granite 3.0 uh if there's a 21:10there are a lot of public benchmarks and 21:12we have some private IBM benchmarks as 21:14well where every model that we are 21:15putting into production we have the 21:16ability to go test them across all these 21:18different uh attack patterns and stuff 21:20right and uh if you look at that that 21:23small class of models which are roughly 21:252 to8 billion parameter models we do a 21:27really good job at across all those 21:30different seven eight different criteria 21:32the granite model scored higher than say 21:34the llama and the Mel and a few other 21:36models as well then on the on securing 21:39the actual usage every time you're 21:41talking to a model and you're you're 21:42bringing data out for the model both 21:44input and outputs get filtered so I'm 21:46much more confident in 21:482024 November when we put models in 21:51production there enough safety guard 21:53rails from IBM and other ecosystem 21:54partners that that we can start to 21:56address these fairly well yeah that's 21:58great and there's one subtlety here that 22:00I think is worth diving a little bit 22:01more into show but if you want to speak 22:02to it is you know with big sleep you're 22:04basically having like an agent like an 22:07AI model examine sort of traditional if 22:09you will software uh code um and it 22:12strikes me that there's a whole separate 22:14set of questions about how you could use 22:17models to analyze the security of models 22:19right yes um because I think obviously 22:21where all this goes is that like once 22:22you do security on agents it's the 22:24security of your security agent that 22:25becomes important curious if you can 22:27talk a little bit about like how the 22:28thinking around that is evolving because 22:30it feels like the pattern matching of oh 22:31here's a vulnerability and code that 22:33we're finding elsewhere looks a little 22:35bit different from how you might use a 22:36model to evaluate the security or safety 22:39of of a model yes and uh I've been 22:41really excited about the work the 22:43collectively the a community has done in 22:44the space um outside of Google we've had 22:47some amazing work done by Nvidia meta 22:50IBM research on creating these models 22:52that can detect vulnerabilities right so 22:53we do that at scale there's a pattern 22:56recognition on the logs that's coming 22:57out there is vulnerability on what are 22:59the corner cases you can now start to 23:01create infinite possible combinations of 23:03how you could break a particular model 23:05and you can stress test them in real 23:06time right so I think we we're doing a 23:09good job as a community on sharing those 23:11techniques as well a lot of the work in 23:12the space has been very open source so 23:15you can start to to to compare different 23:17models different benchmarks private and 23:19public that people are leveraging to 23:21test these vulnerabilities of software 23:22code um I think over time uh there's 23:25there's a recent paper that came on uh 23:27comparing even the llm judge how do you 23:30judge the llm judge right so there's a 23:31lot of this like starts to get thinking 23:33about very meta and and the there AI 23:36That's monitoring AI but I think we are 23:38just moving the the bar of what does a 23:41human do versus what does an AI do so if 23:43you think about the way we uh employ 23:45people into our organizations we would 23:47have somebody who's a graduate from an 23:49amazing school with multiple degrees 23:50just like a really nice llm and we're 23:53giving them some few short learning some 23:54examples during training saying that 23:55here's how we do this thing in our 23:57company then you'll give them access to 23:59all the other vulnerabilities and all 24:01the other things right they are in real 24:02time reading up on a new vulnerability 24:04that happens in a particular environment 24:06and then trying to think how will that 24:08impact their own code so we're starting 24:09to to Crunch through some of those steps 24:11that a human would have done and if you 24:13think about this as bring a new graduate 24:15hire from an institution like MIT or 24:18Stanford into your organization for 24:19cyber security that's the exact same 24:21pattern that we are following with LMS 24:22as well yeah that kind of human metaphor 24:24of how we train cyber Security Experts 24:26uh and applying that to the model is is 24:28interesting and I think lands on maybe 24:30the final question I had for this 24:31segment which is Chris if I can ask you 24:33to make another wild prediction for this 24:35episode um is you know it feels like the 24:38threshold the badge of honor if you're a 24:39security person is like you you 24:41disclosed a really novel kind of exploit 24:44at Defcon um and I guess I'm kind of 24:46curious if like you think that like 24:48agents will eventually pull that off and 24:49if so you have an over-under on like the 24:51year is it is it 2027 when we're going 24:54to have a billion Engineers or how far 24:56off is that in 2020 24:59this is my prediction AI agents will 25:03reveal the first human vulnerability in 25:07code and therefore they will say this 25:10person here is a human vulnerability and 25:13they're doing bad things so that's my 25:15prediction 2028 it's going to be the 25:17other way around AI agents predicting 25:19human vulnerabilities interesting yeah I 25:22would love if the agent finds out a way 25:24like this is the new method for social 25:26engineering would be actually in some 25:27ways like very perfect 25:28I think also what's going to be 25:29interesting is as AI finds our security 25:32flows faster than ever the real question 25:34is who's quicker Defenders patching them 25:37or attackers ready to exploit them no 25:39that' be really funny to see and the 25:41human vulnerability part Chris you just 25:43mentioned we're doing this for one of 25:44the big Latin American Banks right now 25:46where we're leveraging some social 25:47engineering techniques and stuff the 25:49emails that you create for social 25:50engineering attacks is just looks so 25:53plausible right llms are really good at 25:54creating convincing content and you can 25:57trick and let the whole like click 25:59baiting people to go uh into a into a 26:02rabbit hole that's working out really 26:03well but it's really nice some of our 26:05clients are saying that hey U I'm not 26:07quite sure about putting AI in 26:08production our security teams won't give 26:10us the green check let's go pilot llms 26:13for security team first if they're 26:15convinced and they put it to production 26:17then they don't have an excuse to to 26:18bottleneck the rest of the organizations 26:19it's been a good good method working 26:21with lawyers and cyber security teams in 26:23these large organizations yeah it's 26:25going to be so hard when you like try to 26:26log into work and it's like you've been 26:28locked out cuz you're just too 26:30gullible like we've assessed that you 26:32can't make it here just like okay it's 26:35coming 20128 you heard it here 26:37[Music] 26:41first for our final segment I want to 26:43talk a little bit about search gbt so it 26:46goes without saying that open AI is the 26:49the heavy in the industry the the big 26:51leader everybody's been waiting on their 26:52features and what they release and one 26:54thing that everybody's been waiting on 26:55for a long time is for them to finally 26:57get into to the search space um and long 27:00anticipated but it finally launched um 27:02and now open AI now has a search GPT 27:05feature um and this enters a market 27:07that's been kind of dominated and 27:09competed over by you know companies like 27:11perplexity and of course you know Google 27:13uh through Gemini really wants to get 27:14into this space as well and so this is a 27:17big move right the the big industry 27:18leader has finally kind of put its 27:20marker down for what it wants to do in 27:22Search and I know show you looked into 27:24this you know the question I always come 27:25to it is like does this mean that 27:27perplexity is doomed like is everybody 27:28doomed now that open AI is in the space 27:31um and kind of curious about what you 27:32think the effect on the Market's going 27:34to look like so I I recently posted on 27:36LinkedIn saying that uh after I've I've 27:39had access to GD search for a while u i 27:42I pay for a whole I'm very gullible in 27:45paying 20 bucks a month to try out all 27:47kinds of AI so I've been I've been a 27:49paid subcriber for a while and I was 27:50lucky enough to get access to it U I was 27:53comparing it the closest competitor 27:54would be something like Gemini search 27:57right and then it'll be things like 27:58perplexity right so I think if I did a 28:00side by side comparison I have like 13 28:03different areas of topics that I 28:05compared GT search versus uh Google 28:08Gemini and overall I don't think I'm 28:10going to be switching my search from 28:12perplexity and Google and Gemini over to 28:15uh gbt search quite yet and there are a 28:17few things that I found when I was 28:18comparing them one by one just to just 28:20summarize this have a whole article 28:22giving you visual side by sides but 28:24Google generally is a lot more visual 28:27they've learned learned from years of ux 28:29what's the best way to represent the 28:31information for the user right so for 28:32example if you're suggesting restaurants 28:34if I ask gbt search to find restaurants 28:37in a particular location versus Google 28:39Gemini Google Gemini understands that 28:41it's logical to put a map and pinpoint 28:44all the restaurants in the response that 28:45I'm giving you right so itance the right 28:48ux and people would want to go interact 28:49with the graphic and see which one is 28:50closest and so on so forth right 28:52similarly if you're talking about 28:53weather it makes sense and for last few 28:56years Google has had a really nice on 28:58the very top and tells you exactly what 28:59I looking for right the one thing that 29:02I'm I still need to uh that GP needs to 29:04address is they have have a 29:06prolification of different capabilities 29:08they're not quite combined into a single 29:09UI yet so as an example when I switch 29:12over to web search I lose the ability to 29:14upload any content I can't give it any 29:17attachments I can't use any function 29:18calling things of that nature that I'm 29:20very used to when I'm using my o1 29:22previews or my my four O's right versus 29:24in the Gemini world Google Gemini they 29:27they fig out what I'm looking for right 29:30so the simplest example would be if I'm 29:31standing in front of a monument or some 29:33place some Landmark I take a picture I 29:35said can you find me restaurants around 29:37this right now obviously Google Gemini 29:39will identify the place very high 29:41accuracy it will give me nice 29:42recommendations and it help me fine tune 29:45it chat gbt's gbt search cannot take uh 29:49attachment so it can't take any iary it 29:51can't do things like uh if I give you a 29:54document and say Here's my here are the 29:56people that I'm looking for go on link 29:58in and scrap scrape something for them 30:00it can't act it doesn't have access to 30:02function calling I can't give you 30:03documents right so there are certain 30:05things that are like absolutely missing 30:07uh on the GPT side I I think that the 30:10last the last piece that is going for 30:13Gemini which uh is is uh which is still 30:17why I favor Gemini Google is the 30:19connection to your personal data I've 30:21been a big Google like my email address 30:23is show with the Gmail I got that like 30:25when they were starting at the very very 30:27beginning right so we uh all my data my 30:30photos my calendars and stuff like that 30:32are inside of Gmail so when I ask about 30:34hey can I find restaurants near the uh 30:37near the hotel I'm staying in in Mexico 30:38it'll be able to go find that really 30:40quick it's very very personalized you 30:41can go search with my permission of 30:43course it can go and look into my emails 30:45and things of that nature that has a 30:47huge value add to me yeah that's so 30:49interesting is basically that you know I 30:51think way maybe one way of thinking 30:52about this competition is like how much 30:54is search about like the form of the 30:57results versus like the substance of the 30:59results right which is kind of show what 31:01you're saying is like oh when you ask 31:02for a restaurant it's great to have like 31:04the map and the pins and like all the 31:07stuff that Google has indexed even 31:09though the response might be like less 31:11conversationally well flavored than like 31:13what you might get out of perplexity or 31:15or something like that just to counter 31:16maybe some of the arguments that 31:19mentioned of course having that 31:21personalization is so important having 31:22access to all of that and I think Google 31:24has perfected many of these features 31:27given its long history with search but 31:29don't you see as GPT is acquiring also 31:33more 31:34multimodality uh features and as people 31:37more people are using uh chat GPT or 31:41search GPT that personalization will 31:44come uh along you know they they'll 31:47acquire more personal data they can 31:48customize also things so I think it's 31:51just maybe a catchup game here uh one 31:53thing also that I find nice in SE GPT 31:57that I still don't see in Google search 31:59is that interactive nature um the way 32:02they basically it's more conversational 32:05search so unlike traditional search 32:08where they give you like a bunch of 32:10links that you have to click through 32:12this this is making search more 32:14intuitive particularly for complex 32:16queries or ongoing project users might 32:19not longer need to click through a list 32:21of links as the model delivers 32:22synthesized responses so coner I will 32:25push back on that a bit if I may I think 32:27it's it's unfair it's apples or oranges 32:29if you're comparing gbt search with the 32:31classic Google search the right uh 32:34comparison would be Gemini search with 32:36Google right so Google's Gemini is 32:38multimodal like I said earlier I can 32:40take pictures and things of that nature 32:42it is personalized you can tap into your 32:43Google your Gmail and stuff if needed I 32:45can take I can take images and so on so 32:48forth right so I don't Google 32:50understands acknowledges that the Blue 32:52Link world is dying their Gemini Google 32:55search I think is an incredible product 32:57it works really really well and they're 32:58trying their best to make sure that 32:59within the conservative boundaries of 33:01what they can do being such a large 33:03company personalizing hyp personalizing 33:05and multimodality and things of that 33:06nature looking at very very long videos 33:08and summarizing it like things like that 33:10I think they have a very good mode but 33:13the the true comparison is not Google 33:15search Blue Links with GPT search like a 33:17lot of people in media are comparing the 33:19two together and I feel that it's it's 33:20unfair to Google I agree with you it's 33:22not a fair comparison yes and I think 33:26the question here are we moving towards 33:28this one model to rule them all scenario 33:31for search or it's going to be a 33:33competition so but we always had one 33:36more model to rule them all with Google 33:39because they had such a massive 95% Plus 33:42Market right so I I think people are is 33:45that shifting to the Google's gemini or 33:48open AI right now will have a place as 33:51it's also improving his search 33:53capabilities so I I think open AI is 33:57going to win this one out but maybe not 33:59for the reasons you think and so my 34:02experience with chat gbt with search in 34:05this case is it works as a true 34:07extension to the the conversation I was 34:10having anyway so I was having a good so 34:12maybe I'm looking at a particular paper 34:14on something I want something updated 34:17before without access to to the internet 34:20there it's only going to come back with 34:22uh a limited amount of information right 34:24with chat GPT with search it extends out 34:27it takes its knowledge plus the 34:29knowledge that it's got from the 34:30internet and then starts to give me back 34:31better answers and and for me that is 34:35the game changing part and I just found 34:37myself using chat GPT with search more 34:41naturally than I did before so rather 34:43than reaching out for Google to go 34:45answer that question and then mess 34:46around I'm just doing it within the 34:48conversation now if I then bring that in 34:51with the o1 capabilities as that starts 34:53to get releasing and as they start to 34:55combine the modalities the fact is you 34:57know uh you know open ai's been leading 35:01on the modalities on this for a while uh 35:03they're aheed at a game with the o1 35:05models Etc making it more agentic when 35:07they bring all that together um I think 35:10Google's got a lot of work to do there 35:11are they going to go after true search 35:13Etc no but if this is a compar 35:15comparison between Gemini and and the 01 35:19models with search capabilities and 35:21tools as it stands today I I think open 35:24AI is winning that one and and I feel 35:26that today from experience I'm having 35:28and and and the fact is there are 35:31millions of people using chat gbt today 35:33and there's maybe 12 people on chit 35:35that's using Gemini toar today so so I 35:40think that's that's that's where uh 35:43that's where that's my feeling yeah I 35:45think there's a very interesting 35:46question here a little bit about like 35:47it's a debate over what we think the 35:49commodity asset is and what do we think 35:51is the Irreplaceable asset or the hard 35:53to reduplicate ABS asset I think show 35:55bit if I don't want to put words in your 35:56mouth but show bit your your position 35:57seems to be all of this data all of this 36:00kind of incumbent Advantage is the hard 36:03to replace thing and I think what Chris 36:05is saying is like well actually getting 36:07the data is not the hard part it's this 36:09initial additional analysis layer which 36:12is going to be the really unique 36:13differentiator I don't know maybe that's 36:15the right way so there's no doubt that 36:17Google is under logge pressure 36:19perplexity has just shown how well they 36:21work and like I'm Pro user for a very 36:22long timey amazing work right so I think 36:25yeah generally speaking yes they have a 36:27lot of pressure on getting this right 36:29it's a hundred billion dollar problem 36:31for them to solve so they're putting 36:32everything that they can behind it right 36:34so they they had to make sure they nail 36:36the conversational search part and more 36:38more personalized I think the things 36:40that are going in favor of Google are 36:42the fact that they have the world's data 36:43to train on in YouTube and search and 36:46they have like Decades of how people the 36:48the patterns that people follow to get 36:50to the right answer when they're 36:51planning a trip things of that nature 36:52they do have a lot that they can tap 36:54into that other competitors like open I 36:56do not have access to today right so 36:58over time they'll try to catch up with 36:59each other uh Google will always have a 37:01lot of fire behind them to go fix this 37:03to get this the right way but I'm just 37:05the fact that my personal data is 37:07accessible to Google I think that may 37:09change at at some point but in the 37:11current state it is more relevant for me 37:14to have an answer that's hyper 37:15personalized to me and the way I do 37:17things right the fact that I'm asking 37:19you to set an iary in Italy it should 37:21know that I'm landing at 2 p.m. and not 37:24start my itary at 6:00 a.m. right so 37:26that that fun fundamental part of me 37:29having to tell a model say guys just 37:31understand what is important to me first 37:33and you know that the airport is X hours 37:34away so take all of that information 37:36into consideration and I'm thinking 37:38about this from a very Enterprise 37:40perspective as well right for us our 37:41clients are more focused on I've all 37:43this repository of manufacturing 37:45documents and warranty documents and 37:46stuff like that and then I have all of 37:48the other data sets I need to be able to 37:49search against those with high accuracy 37:51and the same experience I'm getting with 37:52chib or search or with Gemini I need to 37:55bring that into my employees to get 37:57unlock the value and it's really nice to 37:59see that meta is starting to get into 38:01this game as well there's a lot of rumor 38:02over the week about meta coming up with 38:04his own search because now they're 38:05incrementally making progress towards 38:07that space as well so I'm really excited 38:09about the future of what happens with 38:11getting information to show in the 38:12moment that I need that's hyper 38:14personalized to the way I consume 38:16information and what's in my emails 38:18things of that nature and I agree with 38:20that shet but you know what I don't want 38:22have Google having exclusive access to 38:24my information right do you know what I 38:25actually want an open EOS system in 38:28marketplace where I can plug into the 38:30agents here go access to my Gmail go 38:33access to this Etc and as opposed to 38:36going well Google's already got this 38:37information and it can train its models 38:39and do whatever it wants with my data 38:41and nobody else can play in the system 38:42so open ecosystem is where I am so yes I 38:45agree but it's got to be open yeah there 38:47is this potential uh for a centralized 38:50AI search model to emerge potentially 38:52monopolizing search you know while this 38:55could bring you know consistency and 38:56ease of fuse it also risk creating you 38:58know this information bottleneck so I I 39:01definitely agree with Chris that having 39:03an open system would be better because 39:06if one model provides small search 39:08answers it might centralize information 39:10flow reduce diversity information 39:12sources and also shape public knowledge 39:15in ways we really don't yet understand 39:17great well uh that's all the time we 39:19have for today uh it's great show but 39:21that you mentioned that meta thing 39:22because that was the other part I wanted 39:23to get into so we will definitely have 39:24that on a future episode of mixture of 39:26experts but unfortunately we are out of 39:28time today so thank you for joining us 39:30if you enjoyed what you heard you can 39:32get us on Apple podcast Spotify and 39:34podcast platforms everywhere show bit 39:36kowar Chris thanks as always appreciate 39:38you joining us