How did you get connected with System76?

One of the things I've always been really passionate about is the grassroots, open source community. I've been doing that stuff for a lot of years. I did a stint at the Linux foundation. I spent years getting to know open source and really like working with the grassroots community, working with enthusiasts and tinkerers and home labbers and makers with a lot of this technology. And then they turn around and they bring it into the workplace. 

And so for the work that I've been doing around arm, native automotive, robotics, all these kinds of things, I know how important the developer community is. And I knew System76, you occupy a really interesting place in that ecosystem. I started over a year ago, reaching out to System76, talking about getting something going. So Carl, Jeremy and folks got some ampere gear in house. You were using it in house for like nine months to develop Pop!_OS for Arm64. You were doing some raspberry PI stuff, like developing, building, testing arm software, mostly for the PI community.

And it’s at the point which you all decided that there is something there. And we got really serious about what can we do together and what can we do in a big way. So that's my journey with y'all. 

What do you do at Ampere?

I had worked with arm developing a reference platform for software defined vehicles and autonomous driving. And we needed this kind of multi core, many core silicon for this program. And Ampere was the obvious fit. So I started developing products using Ampere, and then Ampere asked me to come join the company. 

Now, I'm the Head of Edge. The easiest way to think of that is I lead and drive everything we do that's not in the data center, not in the cloud. And so what we do around developer desktops, space, autonomous vehicles, 5g content distribution networks, like medical imaging, all of these kinds of topics. We’re putting dense, compute, efficient AI sustainable infrastructure out there in the world.

What were your initial thoughts on Thelio Astra?

I don't know if everyone realizes the system76 products in general and things like Thelio in particular, these are not just built, they're not just assembled: these things are designed and thought about and engineered and tested and simulated. And you are doing this airflow fluid dynamics, like hotspot simulation, and you really put a lot of thought into this, and it's kind of amazing! I've spent some time at the factory there, me and my buddy Ken Peterson here at Ampere, and it's really incredible! It's sheet aluminum and aluminum billet enters the factory at one end and it leaves the factory as a beautiful finished product at the other end. And to have that happening, made in the USA, that's a really big deal for me. That's so incredible!

There's a lot of craftsmanship that goes into this and quality. And for anyone who's listening to this podcast, it's no mystery, no surprise that you have some serious open source and Linux and firmware and desktop development chops. 

Something near and dear to my heart is the whole rust topic. And so what you're doing with cosmic, what Jeremy is leading with Redox, redux, redos the, is incredibly important, and this will really resonate. This is important stuff for the community I live in. Rust is a hot topic within automotive. It's what everyone is trying to get to, and you guys are already there. 

What companies do you predict would be using this pre-built solution?

You've got the home labbers, and people are quite adept at hardware, and they love to build things just because it's fun. There's also people who are armed software developers. They just want to get about their doing what they want to do, and they're building them out of necessity. There's a big pent up demand for these things. There will be people that continue to build these things and contribute work on firmware and all these other things. And that's an important part of the ecosystem. And honestly, we can learn and get a lot of insights from that. I ran a home lab arm builder contest where we could take a look and reward people that are doing really interesting things. 

The big demand you look at, it's already public, that Tier IV, which is a famous Japanese autonomous driving company, they're backed by Isuzu, Suzuki, Mitsubishi, people like that. They're doing stuff with Kawasaki, all of these. A lot of autonomous busses, a lot of autonomous movement of goods, like in factory complexes. So they already use your product. There's also, you know, there's a Japanese automaker, big Japanese automaker, who commissioned the development of this product for their autonomous driving team. You see a big demand and uptake in a few areas. So really, anyone who's doing arm software development, anyone who's into product runtime is arm. And the automaker is a good example of that. 

Why Automakers?

The vehicles are in mid flight moving their electronic compute architecture from having hundreds of little cheap ecus. When they talk about software defined vehicles, they really mean cloud native cars. In a software defined vehicle, what you have is instead of a hundreds of these little computers running a single piece of software, you have a few, or ultimately one computer that is larger, more powerful, multicore, many cores with complicated, complex software. Virtualize, containerize, or run things like QNX, rtAs, and automotive grade Linux, cockpit and Android automotive infotainment, all on one computer in the car.

It’s a heterogeneous computer with mixed workloads. You need this freedom of interference, and all of that's arm based. But you can't develop and test your software on that kind of hardware. You'd lose your mind. So most vehicle computers are like Raspberry PI class devices,  but how many hours to build Linux on a Raspberry PI? If you had to develop on that for a living, you'd go nuts. It would be too slow. So you've got to be able to develop and test at scale and deliver at scale. And this is a real challenge for the automakers. Using Ampere, using things like Thelio Astra on the desktop and system 76 Starling in the rack to be able to develop and test and deliver software at scale, being able to get the same silicon that automakers use today on clouds and put that on your desk- that's a really powerful thing. It's an incredibly productivity improving thing.

Why not develop in the cloud?

People say, I'll just do it all in the cloud. But there are some topics where doing it on prem has huge benefits. You're testing different kinds of hardware and cameras and sensors and driveby wire and things. You can't do that in the cloud. You can stub it out, you can do 90 something percent of your test, but you actually, at some point you need physical hardware to do these things. If you're doing digital cockpits, infotainment and instrument cluster, there's real value in having a dozen virtual head units running on your desktop with Nvidia GPUs and being able to render it, capture it, stream it, share it with everyone in the office. That's huge. And I think that's where a lot of the uptake is going to be. All of these things can be done on Thieleo Astra.

Advantages of Ampere CPUs

There's people that kind of ascribe certain values to it. And for the cloud, it's really a matter of rack density and efficiency. To make your cloud business work better and customers happier, and be more productive and profitable, and hit all your carbon green goals, you want more cores per rack and to use less power. So it's a density thing. Historically, a lot of the x 86 is all about scale up. Can I make the cores bigger, hotter, faster, more powerful? With arm, it's kind of more scaled out. If I can fit a large number of energy efficient cores per rack, that's good for the business, good for the environment, good for the bottom line. If these cores are fixed frequency, then they deliver predictable performance. If they have large private caches, then I don't have any. The hyper threading impacts and I can get good performance out of each of those cores. And then if I can put lots of those cores per socket and put lots of sockets per rack, that's the density. So here's the thing. For desktops, this is how this translates because it's so energy efficient.

You can take 128 cores, put it into a desktop, and at 2.6 GHz fully loaded, 124 watts. That makes it easy to cool, it makes it good on your home power bill. It means that it makes it really easy to make it quiet. So now I can have Thelio. I can get Threadripper class performance out of a Thelio, but it's super quiet and not using much power. Pick a number. Let's say I want it to be 500 watts. If my cpu is using less power, then that means I can have more power for all the other things. So I can put hotter GPU's, more gpu's, and still keep within that same thermal and power limit. That's a really big thing. Also, there are folks where they get sticker shock that CPU's in the cloud with GPU's is hella expensive. So there is a lot of flight, not just in the home lab or in individual developers, but corporations and R + D teams and autonomous driving companies who end up shifting these things to the desktop and to racks in the office because they can rack and stack Ampere CPUs with Nvidia GPUs all day long for a fraction what it costs to do that in the cloud. 

Why Thelio Astra?

There is something to be said where if I need to physically touch the hardware or I need GPUs, there's not much that can beat that. Being able to do things like take a Thelio Astra and spin up 32 virtual four core mobile phone Socs or automotive ecus car computers and be running tests in parallel, that has a huge impact on your ability to do more tests faster. Because you can parallelize it. And if you can do more testing faster, that's more turns of the wheel, more complete testing per day. So I move from once a week to once a night per pull request. I can do the full regression testing and that's just going to be awesome for all of these people. So incredibly excited to be doing this with you all. And it's going to have a real impact on automotive quality, mobile quality.

And, you know, for vehicles, this is stuff, you know, ultimately, like, it saves lives, right? It gets better products to market faster. You're going to help accelerate autonomous driving and democratize it so that more people can get into autonomous driving without having to front $100 million in data center GPU. So, yeah, it's genius.

To listen to the audio version of this interview, check out the podcast episode at: https://system76.transistor.fm/12