Kubernetes is an amazing orchestration service. It will make sure all your services are up, restart ones that appear to be failing, and organize your logs and diagnostics. It has taken the industry by storm and has become a common link between all the cloud providers. If you know Kubernetes, you can transition relatively easily between Azure’s AKS, Amazon’s EKS, and Google’s GKE.
Kubernetes是一项了不起的编排服务。 它将确保您的所有服务都已启动，重新启动似乎出现故障的服务，并整理日志和诊断信息。 它席卷了整个行业，并已成为所有云提供商之间的通用链接。 如果您了解Kubernetes，则可以在Azure的AKS，亚马逊的EKS和Google的GKE之间相对轻松地进行过渡。
If you’re a developer, it holds the promise of making the development and production deployments so similar that bugs that would previously only be caught in production can be found early on in the development cycle. And it makes fully automated CI/CD almost close enough to touch without miles of handwritten scripts.
如果您是一名开发人员，那么它有望使开发和生产部署如此相似，从而可以在开发周期的早期发现以前只能在生产中捕获的错误。 而且，它使全自动CI / CD几乎可以接近，而无需数英里的手写脚本。
Alas, it is not all things to all people, and when given a choice, the Kubernetes team will always favor the production orchestration side and developers get left behind. Fully automated CI/CD is still an arduous task, and every development shop still requires DevOps to write those miles of handwritten scripts. In this article, I will try to enumerate where Kubernetes falls short in a development environment.
las，这并不是所有人都能解决的所有问题，如果有选择的话，Kubernetes团队将始终偏向生产流程方面，而开发人员会落在后面。 全自动CI / CD仍然是一项艰巨的任务，每个开发机构仍然需要DevOps来编写那么长的手写脚本。 在本文中，我将尝试枚举Kubernetes在开发环境中的不足之处。
1. Docker中的Docker (1. Docker in Docker)
This should strike fear in the hearts of everyone who has needed to deal with it. Docker is not suited to run inside itself, and Kubernetes orchestrates Docker containers. So if your pipeline needs Docker to build an image and your pipeline is run under Kubernetes, you need to run Docker inside of Docker, which gives me a headache just thinking about.
这应该在需要处理它的每个人的心中引起恐惧。 Docker不适合在其内部运行，Kubernetes精心设计Docker容器。 因此，如果您的管道需要Docker来构建映像并且管道在Kubernetes下运行，则需要在Docker内部运行Docker，这让我头疼不已。
There are two solutions to this problem. The first is to hijack the Docker socket of the outside host. This makes your Docker container running inside more of a sibling container than a child container. Let me explain. Docker is divided into two parts: the command-line interface and the Docker machine. The two communicate through a well-known socket, so when you type in a Docker command, the command-line program sends a message via the socket to the Docker machine, which does most of the work. So if you issue a Docker command-line instruction from inside a container, it will send the message to the Docker machine of the outside host. The drawback to this solution is that the inner container needs to be privileged to access the Docker socket of the outside host.
有两个解决方案。 首先是劫持外部主机的Docker套接字。 这使您的Docker容器在子容器中运行的空间更多于同级容器。 让我解释。 Docker分为两部分：命令行界面和Docker计算机。 两者通过一个众所周知的套接字进行通信，因此，当您键入Docker命令时，命令行程序会通过套接字将消息发送到Docker计算机，这完成了大部分工作。 因此，如果您从容器内部发出Docker命令行指令，它将把消息发送到外部主机的Docker机器。 该解决方案的缺点是，需要授予内部容器访问外部主机的Docker套接字的特权。
The second solution is to emulate Docker via another program. There are a plethora of these non-Docker emulation programs, such as Buildah or Kaniko. But Docker actually handles a lot of tasks: building images, dealing with the container registry, and running images. The two aforementioned programs only deal with building and pushing to container registries. Other programs can be used for more access to container registries.
第二种解决方案是通过另一个程序模拟Docker。 这些非Docker仿真程序很多，例如Buildah或Kaniko 。 但是Docker实际上处理许多任务：构建映像，处理容器注册表以及运行映像。 上面提到的两个程序仅涉及构建和推送到容器注册表。 其他程序可用于对容器注册表进行更多访问。
So while there are solutions for eliminating Docker from the build process, no one does Docker like Docker.
因此，尽管有一些解决方案可以从构建过程中消除Docker，但没有人像Docker一样拥有Docker。
2. Kubernetes描述符文件是静态的 (2. Kubernetes Descriptor Files Are Static)
The people at Kubernetes made a conscious decision to not allow outside influences when it comes to descriptor files. No environment variables, no command-line variables. What’s in the descriptor file is what’s running. You can use the command line to modify already-running services to update the image version in a controlled fashion or other changes, but those changes aren’t reflected in the original descriptor files.
Kubernetes的人们做出了一个有意识的决定，即在描述文件时，不允许外界的影响。 没有环境变量，没有命令行变量。 描述符文件中的内容是运行中的内容。 您可以使用命令行来修改已经运行的服务，以受控方式或其他更改来更新映像版本，但是这些更改不会反映在原始描述符文件中。
Other than using the command line to change descriptions in very controlled ways (and in production, you probably only want to change things in very controlled ways), the only workaround is templating. Of course, this defeats the purpose of static descriptor files in the first place.
除了使用命令行以受控制的方式更改描述(在生产中，您可能只想以受控制的方式更改内容)之外，唯一的解决方法是模板化。 当然，这首先破坏了静态描述符文件的目的。
Helm charts are the main way to templatize the descriptor files. And they actually solve a lot of other problems like coordinating changes between a multitude of services. But it is also very easy to get carried away with Helm charts and make your deployments a complicated mess, which is what we’re trying to avoid. Basically, they are job security for the DevOps team.
舵图是模板化描述符文件的主要方法。 实际上，它们解决了许多其他问题，例如协调多个服务之间的更改。 但是，也很容易迷失Helm图表，并使您的部署变得复杂，这是我们要避免的事情。 基本上，它们是DevOps团队的工作保障。
Strict adherence to static descriptor files makes it very hard to use the same descriptors in a development and production environment. So our environments start to diverge, and you must spend time and effort trying to re-converge. So the “it works in dev, so we know it’s going to work in prod” dream evaporates.
由于严格遵守静态描述符文件，因此在开发和生产环境中很难使用相同的描述符。 因此，我们的环境开始出现分歧，您必须花费时间和精力尝试重新融合。 因此，“它可以在开发人员中工作，因此我们知道它将在产品中工作”的梦想就烟消云散了。
3.开发环境中的准备情况检查 (3. Readiness Checks in a Development Environment)
A developer finally merges their code and celebrates by going out for a few drinks. However, back in Kubernetes, the service fails to start up. So Kubernetes obediently restarts it and the service fails again. Now the node is going to be using all of its CPU restarting a broken service. Sure, you could write a little program that un-deploys services that have restarted once a minute for the last hour or make your script deploying the new image smart enough to back out and restore the last image. But that just adds more lines of script for something that was supposed to reduce complexity. Different strategies are needed for development and production, and those differences add to the divergence that we wanted to avoid.
开发人员最终合并他们的代码，并出去喝杯酒庆祝。 但是，回到Kubernetes，该服务无法启动。 因此Kubernetes乖乖地重新启动它，并且服务再次失败。 现在，该节点将使用其所有CPU重新启动中断的服务。 当然，您可以编写一个小程序来取消部署在过去一小时内每分钟重新启动一次的服务，或者使您的脚本足够智能地部署新映像以退出并还原最后一个映像。 但这只会增加原本应该减少复杂性的脚本行。 开发和生产需要不同的策略，这些差异加剧了我们想要避免的差异。
4.容器内部和外部的不一致 (4. Inconsistencies Inside and Outside the Container)
Sometimes, we need to run a program locally so we can test code before we commit. In order to do that, we need all the external dependencies so the program can operate without dummying up the databases, message queues, or other services. It would be nice to use the services as they are running in Kubernetes, but problems arise when some services have subtle differences running inside the cluster or don’t have proper external ports for access from outside the cluster.
有时，我们需要在本地运行程序，以便在提交之前可以测试代码。 为此，我们需要所有外部依赖项，以便程序可以运行而无需虚拟数据库，消息队列或其他服务。 使用服务在Kubernetes中运行时会很好，但是当某些服务在集群内部运行时存在细微差异或没有适当的外部端口供集群外部访问时，就会出现问题。
I’m thinking specifically about services that can have multiple instances and have leader elections. When you connect to one instance, it returns the IP address of the chosen leader. Since the network inside the cluster is a different subnet than the external network, the IP address won’t work. Often, very convoluted configuration needs to take place just to have consistent access from both inside and outside the cluster — even if we know that there’s only a single instance running inside the cluster. This isn’t an issue with Kubernetes per se, but it still is an impediment to taking advantage of container orchestration in a development environment.
我正在专门考虑可以具有多个实例并具有领导者选举的服务。 当您连接到一个实例时，它将返回所选领导者的IP地址。 由于群集内部的网络与外部网络是不同的子网，因此IP地址将不起作用。 通常，为了使集群内部和外部都具有一致的访问权限，就需要进行非常复杂的配置-即使我们知道集群内部只有一个实例正在运行。 这不是Kubernetes本身的问题，但它仍然是在开发环境中利用容器编排的障碍。
5.图像标签被认为是不可变的 (5. Image Tags Are Considered Immutable)
In Docker, image tags are by their nature mutable. There are some situations where you want to change them and other situations where you don’t want to change them, but you have the choice and Docker acts accordingly. If your tag is a semantic version, you probably never want to change it. It should always belong to the same image. But if your tag is an abstract condition such as stable or test, you might want to bump those as an image winds its way through the CI/CD pipeline. Once a new image is built, you might want to remove the test tag from the previous image that had it and move it to the new image. Once a new image passes a battery of tests, you might want to move the stable tag. Then your CI/CD pipeline knows which image to deploy at each stage and/or namespace.
在Docker中，图像标签本质上是可变的。 在某些情况下，您想要更改它们，而在其他情况下，您不想更改它们，但是您可以选择，并且Docker会相应地采取行动。 如果您的标签是语义版本，则您可能永远不想更改它。 它应该始终属于同一图像。 但是，如果您的标签是诸如stable或test类的抽象条件，则可能需要在图像绕过CI / CD管道的过程中碰碰它们。 构建新映像后，您可能希望从具有该映像的先前映像中删除test标签，然后将其移动到新映像。 新图像通过一系列测试后，您可能希望移动stable标签。 然后，您的CI / CD管道知道在每个阶段和/或名称空间部署哪个映像。
The problem is Kubernetes considers tags immutable. If an image tagged stable is already pulled to a node, it will never check if the image is changed. This appears to be a long-running debate in the Kubernetes community. However, there will probably be no change in Kubernetes — only external tools that will help you achieve this.
问题是Kubernetes认为标签是不可变的。 如果标记为stable的图像已被拉到节点，则它将永远不会检查图像是否已更改。 这似乎是Kubernetes社区中长期存在的辩论。 但是，Kubernetes可能不会有任何变化，只有外部工具可以帮助您实现这一目标。
结论 (Conclusion)
I still like the idea of using container orchestration in a development environment and to help automate the CI/CD processes. There is just too much benefit to using Kubernetes to run your pipelines and services that are being tested.
我仍然喜欢在开发环境中使用容器编排并帮助自动化CI / CD流程的想法。 使用Kubernetes运行正在测试的管道和服务有太多好处。
But let this be a fair warning to those who think they can achieve this without compromises or complexity: Kubernetes in a development environment is not the panacea that some people would have you think it is.
但是，对于那些认为自己可以在不妥协或复杂的情况下实现这一目标的人来说，这是一个公平的警告：开发环境中的Kubernetes并不是某些人认为的万能药。