I want to share few words about me. I'm CTO of FastVPS Eesti OU with ~8 years of overall experience with containers and OpenVZ. I'm author of following tools for OpenVZ:
- Ploop userspace (user mode implementation of ploop filesystem)
- Fast incremental difference algorithm for ploop images.
- Author of script for TC based shaper for OpenVZ
From the operational side we have about few hundreds of physical servers with OpenVZ (VZ6 based) with pretty huge density and I'm trying to keep my hands dirty and dig into details of any issues.
What you should expect following technical issues when you are working with OpenVZ:
- Very often and repetitive kernel faults for different reasons even on top of top-notch hardware with ECC, redundant power and battery backed RAID's
- Huge disk space (up 30-40% in some cases) overuse if you decide to use ploop filesystem (that's true for VZ7 too).
- Pretty often storage faults with complete data corruption if you are using ploop. You could read more details about it here.
- If you want to get security updates in timely manner you should not rely on OpenVZ team. You should use some external toolkit for applying patches to running kernel (Ksplice, Kpatch or Kernel Care).
- You should expect a lot of issues already fixed in recent kernels because OpenVZ project are relying on already outdated RHEL 6 or RHEL7 (based on top of 3.10 but current version is 4.6) kernel. You definitely know about huge backport work of Red Hat but these kernels are really outdated and you could get significant speedup with update to recent vanilla kernel (as Oracle or Ubuntu already doing). But please keep in mind if you are running OpenVZ (VZ7) you are running already outdated toolkit.
- You should expect significant changes without backward compatibility in next major release of OpenVZ (for more details please read more details about prlctl and vzctl). In some nice moment you should rewrite all your integration code from billing, configuration management and support control subsystems.
- Very poor QA and issues with pretty obvious things.
- Almost each upcoming systemd update in container may require new kernel from OpenVZ / Virtuozzo team. Actually, this updates could break your running kernel.
If we are speaking about overall experience about project I could share my ten cents below:
- You should not expect any openness in the decision making of OpenVZ project development. That's still true for day-to-day issues (bug priority) and for helicopter view (strategic planning). That's still commercial company and they will not discuss roadmaps with the community. If they decide to do something they will do it even if community hate their decisions. You should check "simfs deprecation" conversation in OpenVZ Users maillist for details.
- You should have very experienced system administrator who should dig into details of each issue and system instability.
- OpenVZ (VZ6, VZ7)
- Linux upstream containers (managed by LXC or Docker)
Unfortunately, Linux upstream containers could not run whole OS properly they are pretty insecure in compare with OpenVZ. But if you want to isolate your own applications I want to recommend to use Docker because they have a lot of benefits and haven't almost all OpenVZ issues.
But if you want to isolate whole operating systems (for making VPS service) I want to recommend KVM technology. But KVM has significant overhead in compare with containers! Actually, it's true only partially because Intel have been doing a lot of things for improving performance of hardware based isolation and recent CPU's (Xeon E3, E5) has lightning fast KVM implementation! But please be careful and do not use RHEL 7 (do you remember? It's already outdated and you could host more VM's per host with recent kernel!) and look at vanilla kernel, Oracle kernel or some other project which offer recent and stable kernel.
Finally, I really like container isolation and I will be happy when all mentioned issues will be fixed ;)
Thanks for attention!
