Civo: The public cloud powered by Talos Linux
Data Center Bare Metal
Global
Public Cloud
Civo is a public cloud provider focused on challenging the narrative of what a hyperscaler can offer and deliver. They launched with a target of being developer-focused and delivering a Kubernetes-focused public cloud and now offer cluster launch times of under 90 seconds.
Challenge
- Minimizing data center intervention
- Provisioning of full Kubernetes clusters in minutes
- Avoiding overhead and configuration drift
Environment
- Bare metal servers
Impact
- No more data center visits
- No more configuration drift
Why Sidero and Omni
- API-managed
- Aligned with the Kubernetes operator pattern
- Immutable, declarative, minimal OS
Building a reliable infrastructure meant to scale
The process of getting servers up and running can take months of intensive work, from ordering hardware and waiting for delivery to unboxing, racking, cabling, installing the operating system, setting up switches and routers, and more. While Civo removes this work for its customers, the team must perform all of these steps on a massive scale. Building a cloud requires you to examine data center choice, hardware, and connectivity in incredible depth. When Civo set out to build their infrastructure, they outlined several goals.
- They did not want their team to ever have to visit the data center again. Data center visits are time-consuming, expensive, and can present tough work for the specialist.
- They wanted to reduce the time to build from months to minutes, so they could get up and running fast.
- They wanted to provide their internal teams with “it just works” cloud management, ensuring that both engineers and end-customers had a positive experience.
Civo wanted to create a single interface to provision switches, routers, and compute hardware. They opted to use bare PXE servers to provision with cloud-init for OS install, and DHCP options for networking hardware. For their operating system, Civo initially looked to CoreOS, as it was focused on running Kubernetes. However, once CoreOS was acquired by RedHat, Civo found it was no longer suitable. They evaluated OpenStack, OpenShift, and CloudStack, but found that none of these solutions supported their vision for a modern cloud native infrastructure. Instead, these options imposed a large amount of overhead that shouldn’t be necessary to get Kubernetes running. Civo evaluated Ubuntu and Alpine but found that these came with excess packages that were not relevant to Kubernetes.
Civo chose to use Ubuntu. Issues arose quickly. Though they used Ansible to manage the tenant workload and address configuration drift, they found themselves having to create more and more operators to manage the infrastructure. Within two weeks, the system was rife with small changes, as engineers had gone in to make quick fixes. The team knew this configuration drift would not be manageable as the infrastructure grew. Civo was building for a massive scale, and they needed a solution to support their goals.
Drop-in, drift-free, and designed for Kubernetes OS
“The fact that we could just put operators on top of it, we could interact with it with an API to drive things like node reboots, and even all the way through to upgrades and configuration changes, really fit in with the operator pattern that we had running at the moment.”
Dinesh Majrekar, CTO, Civo
Civo discovered Talos Linux and “absolutely loved it.” The team found Talos Linux inherently solved many of their problems and saw it as a modern version of CoreOS, able to quietly perform its job and stay out of the way while also ensuring security. The team also liked that Talos Linux is API-managed, including for node reboots and upgrades, making it natively aligned with the operator paradigm of Kubernetes and therefore ideal for environments using Operators to automate infrastructure. Talos Linux functions as a drop-in replacement for their Ubuntu setup and is now a drop-in replacement for the PXE-based build system, sending Talos kernel flags for configuration.
Civo gives its users the choice between Talos Linux and K3s, which was their initial offering at launch, for provisioning Kubernetes clusters. They are the first provider to perform the full automation of a complete Talos Linux cluster with the press of a button. Civo users now need only 90 seconds to launch a new cluster.
From power-on to customer-ready in 20 minutes
Civo now has one region fully built on Talos Linux and will migrate all other existing data center infrastructures from Ubuntu to Talos Linux. Going forward, Talos Linux will be used across all new regions and replace K3s as the default tenant offering.
With Talos Linux, the Civo team has achieved its goal of never having to go to a data center again. Within 20 minutes of a server arriving on site and connecting to the network, the region is up and serving customers. They can simply turn on the new hardware, and it self-registers, builds, and configures itself, and is made available. They no longer have to worry about configuration drift, as Talos Linux is immutable and declarative, eliminating the possibility of unwanted changes and ensuring consistency across Civo’s entire infrastructure. The team also enjoys an OS free from unnecessary packages or irrelevant overhead, as Talos Linux is built specifically for Kubernetes and includes only what is required to run Kubernetes.
This article is a summary of the talk Civo gave at TalosCon 2023. Watch the full talk below. All the talks of TalosCon are available here: TalosCon 2023 Kubernetes talks playlist.