Now we don't deploy this to private companies except some gov projects. But we are engaging existing IaaS and dedicated servers providers. Later hope we would fight with Broadcom and IBM :)
We previously ran a services company that built over 100 private clouds, then decided to turn that experience into a product with an aggressive level of automation to let DevOps, SRE, and platform teams hit a 1:300 ratio per developer (now dev + AI agent teams).
We are currently deployed across 12 datacenters and raising our seed round this month.
Regarding been curious, today I have created my first explainable video with Remotion +claude so if you want to see open edit results check https://promo-review-shalb-promo.stage.kube-dc.com/
Would be glad if you comment. Thanks!
Looking forward to see how your product develops. Would love to work with you in future.
From my hands-on engineering POV I often set this maybe not as a hard requirement but definitely a goal, and the places where it's not possible I find a little painful.
In your case where is it becoming something that doesn't make sense for you?
The other issue is permissions management. Git is great for auditability but it doesn't have the fine-grained permissions management you'd typically require from a "cloud" provider.
The end result is that we've created a "private cloud console" which has AWS-like IAM with fine-grained permissions and RBAC. All of the actions in the console talk to Github.com and open pull requests to repositories which once merged will publish a new OCI artifact. FluxCD pulls the latest artifact and Crossplane does the logic of updating the state of resources. Some (external/non-k8s) resources don't fit into this model very well if they don't have a Crossplane provider like our NetApp arrays where S3 is hosted.
Same things with dynamic changes just for test something, you need or commit temp change and commit back old value again or just pause controller from updating during your tests.
But the bright side of this is the fact that your configuration completly traceble and you have whole IaC backup for your infra in unrbreakable place (git).
For now we fully perform all updates and all configuration with Flux and this was a good decision I can retrospect.
How have you found the stability of KubeVirt through all of this? Are you using KubeVirt directly or a wrapper? We tried Harvester and it is amazing on day 1. Day 2 is a horrid nightmare of broken upgrade timesink. Granted we started at Harvester 1.4.0 in 2024 but have a couple clusters stuck on 1.5.0 right now and have VM failures regularly. Longhorn was also in use for VM disks and it was horrendously unstable even with dedicated 2x100Gb for each node.
We've moved everything to Proxmox+NetApp now and it's rock solid. Dedicated NICs + switches for corosync since that was the cause of an outage once.