Posted on Jul 24 • Originally published at raftt.io In recent years, Kubernetes has emerged as the de facto standard for container orchestration, providing an efficient and scalable platform for deploying and managing applications. As developers strive to streamline their workflows and optimize productivity, an emerging trend is developing applications directly on top of the Kubernetes Cluster itself. In this blog post, we will explore the benefits of developing right on top of your Kubernetes cluster, and dive into the challenges that make this a difficult reality to achieve.Starting with the benefits - By developing directly on your Kubernetes cluster, you eliminate the need for a separate development environment or local setup. This streamlines the development process, reducing the overhead of replicating the cluster setup locally, and therefore the time spent on setting up and maintaining multiple environments.In a complex project there may be multiple flavors of production environments along with a host of staging, test, and preview environments. Having to maintain yet another environment type (which is used by the entire dev team!) is a huge time and attention sink for both developers and devops.Catch and resolve issues early on, and reduce the risk of surprises during deployment by using the same tools, libraries, and configurations as the production environment. Developing on top of your Kubernetes cluster ensures that your development environment is aligned with the production environment. This consistency eliminates the "works on my machine" problem, where code behaves differently in different environments.Developing on top of your Kubernetes cluster gives you direct access to cluster services and features. You can easily integrate with monitoring tools, logging systems, service meshes, and other cluster-specific resources. This allows you to develop applications that are tightly coupled with the underlying infrastructure, leveraging the full capabilities of Kubernetes and maximizing the efficiency of your applications.Kubernetes provides powerful features for collaboration, allowing multiple developers to work simultaneously on the same cluster. With development happening directly on the cluster, team members can easily share links to environments (using Kubernetes Ingress objects). This enhances teamwork and promotes knowledge sharing, as developers can observe and learn from each other's work.Developing directly on your Kubernetes cluster simplifies the testing process. Since you are working on the same environment where your application will run, you can easily reproduce and analyze issues that might arise during deployment. This reduces the time spent on recreating issues locally and provides a more accurate representation of how the application behaves in the production environment.Unfortunately, there are a couple of significant challenges that make developing on top of a cluster difficult - Containers are designed to be immutable, meaning they can not be modified once built. While this immutability promotes scalability and consistency, it poses a big challenge for developers - it is no longer possible to make code changes and see fast feedback! Instead, developers rely on their CI pipeline to deploy new images, which extends the feedback cycle from seconds to 30+ minutes.Long cycles leads to bundling many changes together, which in turn causes each cycle’s success rate to drop, and inevitably dev speed is slowed significantly.To make matters worse, it is difficult to set up proper debugging for code running in a Kubernetes cluster. All of these mean that practically - developers cease debugging once their env is running on Kubernetes.Kubernetes is a highly sophisticated and complex system. While some developers have a strong grasp of the basic Kubernetes concepts, the learning curve can be steep and it can slow down developers significantly. In most organizations, access to Kubernetes clusters (even those intended for development) is not available to all developers. For good reason too - it is easy to make mistakes and cause damage that takes time and resources to repair, and the teams maintaining the clusters are wise to take proactive measures to minimize the chance of that. The larger the organization, and the more complex the deployment, the higher the chance of limitations being in place for its clusters.Managing the cost of a Kubernetes cluster is an art in itself. Significantly increasing the number of environments due to their use in development can increase costs significantly. Managing the large number of dev environments can add a lot of overhead to already overextended DevOps or platform teams. Existing tooling intended to provide visibility into a limited number of environments may not scale well to the higher usage. And differences in configuration between environment types can cause overhead and false-positives.Luckily, there are all kinds of tools that can be used to manage and develop on dev environments on Kubernetes. Everything from Spinnaker to ArgoCD and FluxCD for orchestration, and various solutions for the developer experience layer.We use our own product (Raftt), which handles all of this complexity [and more]. Raftt adapts the Kubernetes environment for development, enabling out of the box hot-reloading of code and debugging, connected directly to your local machine and IDE. Raftt’s optional cluster-level controller handles environment provisioning and lifecycle, minimizing maintenance and additional costs.You are left with a great development experience (including code hot reloading, debugging, collaboration…) that is easy to manage and deploy. You can try it yourself - deploy Raftt for development on your Kubernetes cluster, with our demo project or with your own code.Templates let you quickly answer FAQs or store snippets for re-use. Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink. Hide child comments as well Confirm For further actions, you may consider blocking this person and/or reporting abuse Ajisafe Oluwapelumi - May 12 Rob Holland - Jun 15 Hussein Alamutu - Jun 15 Gabor Szabo - Jun 15 Once suspended, gsaraf will not be able to comment or publish posts until their suspension is removed. Once unsuspended, gsaraf will be able to comment and publish posts again. Once unpublished, all posts by gsaraf will become hidden and only accessible to themselves. If gsaraf is not suspended, they can still re-publish their posts from their dashboard. Note: Once unpublished, this post will become invisible to the public and only accessible to Gahl Saraf. They can still re-publish the post if they are not suspended. Thanks for keeping DEV Community safe. Here is what you can do to flag gsaraf: gsaraf consistently posts content that violates DEV Community's code of conduct because it is harassing, offensive or spammy. Unflagging gsaraf will restore default visibility to their posts. DEV Community — A constructive and inclusive social network for software developers. With you every step of your journey. Built on Forem — the open source software that powers DEV and other inclusive communities.Made with love and Ruby on Rails. DEV Community © 2016 - 2023. We're a place where coders share, stay up-to-date and grow their careers.