When it comes to Devops, the Continuous Integration and Continuous Delivery (CI/CD) of an Application is what matters most. Everything else is just noise. But for developers, setting up a CI/CD pipeline often means dealing with an annoying amount of tools and processes, struggling with GUIs, and taking on a barrage of custom scripts and endless Jenkins jobs.
Addressing these issues ahead of time with a proper CI/CD setup can do wonders for your applications and teams, especially in a complex microservice architecture. On my personal blog, I’ve been writing a lot about microservices, Kubernetes, and JHipster. And since I work for a DevOps company, I think it’s important to take a deeper look at DevOps for microservices. You can check out the original post here.
Creating a Microservice Stack Using JHipster
For this post, I’m going to create a a full-fledged microservice stack using JHipster and then deploy it to Google Kubernetes Engine. The Google Cloud Platform (GCP) infrastructure will be provisioned with Terraform and then all of this will be orchestrated using the XebiaLabs DevOps Platform.
If you’re a DevOps engineer interested in a more declarative approach to DevOps, then this post is for you. And if you’re an enterprise developer or architect looking for a CI/CD setup that meets your company’s compliance and security requirements, then this post is also for you.
Prerequisites
To proceed, you need to install JHipster if you haven’t already. You can follow the set up instructions here.
You’ll also need version 8.6 of the XebiaLabs Command Line Interface. You can get the installation instructions here.
The Technology
Why JHipster
JHipster is the leading rapid application development platform for the JVM ecosystem. With JHipster, you can create monolith or microservice applications in just minutes using Spring Boot and Angular/React/VueJS. JHipster provides a nice DSL to define applications, entities, and deployment options. Check out my previous post about creating microservices with JHipster for more details. We will be using JHipster to scaffold our microservice applications.
Why Terraform
HashiCorp Terraform is an open source Infrastructure-as-Code tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned. Terraform is a powerful tool with a nice CLI for provisioning and tearing down infrastructure in a variety of cloud and native environments. We will be using Terraform to provision our Google Cloud infrastructure.
Why XebiaLabs
XebiaLabs enables enterprises to do Continuous Delivery at scale for cloud, container, and legacy environments. As a developer with his fair share of DevOps experience, I can confidently say that XebiaLabs has one of the most sophisticated release orchestration and deployment automation tools I have ever seen. The XebiaLabs DevOps Platform is particularly powerful for complex enterprise applications that have a lot of process, compliance, and security requirements. For the techies, like me, who would prefer everything in code––even their wedding invitations––XebiaLabs provides a function that we call DevOps as Code. DevOps as Code is a YAML-based schema for defining deployments and release processes as declarative code.
For this demo, we will be using both XL Release and XL Deploy, along with the DevOps as Code features.
XebiaLabs DevOps as Code
The DevOps as Code offering is made of three main components––Blueprints, XL YAML files, and the XL Command Line Interface.
Blueprints are templates, written using our in-house scaffolding mechanism, built on top of Go templating language. We use blueprints to provide starter templates for the DevOps as Code feature. Blueprints are similar to JHipster in that they help to scaffold configurations for provisioning and DevOps, so you can focus on getting work done rather than writing boilerplate code.
The XL YAML files provide a comprehensive schema to define release pipelines for XL Release and deployment configurations for XL Deploy. They can be used to completely replace the UI based on the configurations of both products. This lets us iterate faster and provides a more natural way of defining the DevOps pipeline right from a IDE/Editor.
Finally, the XL CLI helps to put everything together, so it can be used to run blueprints and to apply the XL YAML files to the respective products and more.
Creating the Microservice Stack
We can now create our microservice application using the JHipster Domain Language (JDL) to model our applications. In a previous post, I showed how to create a full stack microservice architecture using JHipster and JDL. For this exercise, we will use the same application.
Let’s look at the application architecture…
To recap, we need to create a JDL file––app.jdl––and copy the below content into it.
application {
config {
baseName store,
applicationType gateway,
packageName com.jhipster.demo.store,
serviceDiscoveryType eureka,
authenticationType jwt,
prodDatabaseType mysql,
cacheProvider hazelcast,
buildTool gradle,
clientFramework react,
useSass true,
testFrameworks [protractor]
}
entities *
}
application {
config {
baseName invoice,
applicationType microservice,
packageName com.jhipster.demo.invoice,
serviceDiscoveryType eureka,
authenticationType jwt,
prodDatabaseType mysql,
buildTool gradle,
serverPort 8081,
skipUserManagement true
}
entities Invoice, Shipment
}
application {
config {
baseName notification,
applicationType microservice,
packageName com.jhipster.demo.notification,
serviceDiscoveryType eureka,
authenticationType jwt,
databaseType mongodb,
prodDatabaseType mongodb,
devDatabaseType mongodb,
cacheProvider no,
enableHibernateCache false,
buildTool gradle,
serverPort 8082,
skipUserManagement true
}
entities Notification
}
/* Entities for Store Gateway */
/** Product sold by the Online store */
entity Product {
name String required
description String
price BigDecimal required min(0)
size Size required
image ImageBlob
}
enum Size {
S, M, L, XL, XXL
}
entity ProductCategory {
name String required
description String
}
entity Customer {
firstName String required
lastName String required
gender Gender required
email String required pattern(/^[^@\s]+@[^@\s]+\.[^@\s]+$/)
phone String required
addressLine1 String required
addressLine2 String
city String required
country String required
}
enum Gender {
MALE, FEMALE, OTHER
}
entity ProductOrder {
placedDate Instant required
status OrderStatus required
code String required
invoiceId Long
}
enum OrderStatus {
COMPLETED, PENDING, CANCELLED
}
entity OrderItem {
quantity Integer required min(0)
totalPrice BigDecimal required min(0)
status OrderItemStatus required
}
enum OrderItemStatus {
AVAILABLE, OUT_OF_STOCK, BACK_ORDER
}
relationship OneToOne {
Customer{user(login) required} to User
}
relationship ManyToOne {
OrderItem{product(name) required} to Product
}
relationship OneToMany {
Customer{order} to ProductOrder{customer(email) required},
ProductOrder{orderItem} to OrderItem{order(code) required} ,
ProductCategory{product} to Product{productCategory(name)}
}
service Product, ProductCategory, Customer, ProductOrder, OrderItem with serviceClass
paginate Product, Customer, ProductOrder, OrderItem with pagination
/* Entities for Invoice microservice */
entity Invoice {
code String required
date Instant required
details String
status InvoiceStatus required
paymentMethod PaymentMethod required
paymentDate Instant required
paymentAmount BigDecimal required
}
enum InvoiceStatus {
PAID, ISSUED, CANCELLED
}
entity Shipment {
trackingCode String
date Instant required
details String
}
enum PaymentMethod {
CREDIT_CARD, CASH_ON_DELIVERY, PAYPAL
}
relationship OneToMany {
Invoice{shipment} to Shipment{invoice(code) required}
}
service Invoice, Shipment with serviceClass
paginate Invoice, Shipment with pagination
microservice Invoice, Shipment with invoice
/* Entities for notification microservice */
entity Notification {
date Instant required
details String
sentDate Instant required
format NotificationType required
userId Long required
productId Long required
}
enum NotificationType {
EMAIL, SMS, PARCEL
}
microservice Notification with notification
Next, we’ll add the Kubernetes deployment description to the JDL so we can generate Kubernetes manifests for our applications. Make sure to use your own dockerRepositoryName.
deployment {
deploymentType kubernetes
appsFolders [store, invoice, notification]
dockerRepositoryName "deepu105"
kubernetesNamespace xl-demo
}
We’ll then create a directory called ecommerce in your preferred location in the file system and navigate into it. Save the above JDL as app.jdl in the directory and run the JHipster import-jdl command. This could take a few minutes, especially the NPM install step.
$ mkdir ecommerce && cd ecommerce $ jhipster import-jdl app.jdl --skip-git
Once the JHipster process is complete, our store gateway, invoice service, and notification service are ready for use. The Kubernetes manifests have also been created alongside.
You can deploy the application locally using Docker. If you haven’t done that before I strongly suggest doing so to get an idea of how the application works locally on your machine.
Now that our application is ready we will set up a Jenkins CI build. In the spirit of a true microservice, we will add a Jenkins job to each of the services.
We’ll then navigate into each of the store, invoice, and notification folders, run the JHipster CI/CD command, and follow the instructions. Choose the answers below for all the services. Use your own docker login name for “Organization Name for the Docker registry.”
$ cd store $ jhipster ci-cd Welcome to the JHipster CI/CD Sub-Generator ? What CI/CD pipeline do you want to generate? Jenkins pipeline ? Would you like to perform the build in a Docker container ? No ? Would you like to send build status to GitLab ? No ? What tasks/integrations do you want to include ? Build and publish a *Docker* image ? *Docker*: what is the URL of the Docker registry ? https://registry.hub.docker.com ? *Docker*: what is the Jenkins Credentials ID for the Docker registry ? docker-login ? *Docker*: what is the Organization Name for the Docker registry ? deepu105 create Jenkinsfile create src/main/docker/jenkins.yml create src/main/resources/idea.gdsl create src/main/docker/docker-registry.yml INFO! Congratulations, JHipster execution is complete!
JHipster expects each project to live in its own Git repository, but for this demo, we will follow a monorepo approach. So, we will need to adjust the generated Jenkinsfile to work with this. The change is simple––in each of the generated Jenkinsfiles, we need to run a CD command before executing any other commands, and we need to adjust the paths in the copy commands. For example, here is the modified Jenkinsfile for the store service with the changes highlighted. Some unrelated code is truncated for brevity.
...
node {
...
stage('clean') {
sh "cd store && chmod +x gradlew"
sh "cd store && ./gradlew clean --no-daemon"
}
stage('npm install') {
sh "cd store && ./gradlew npm_install ..."
}
stage('backend tests') {
try {
sh "cd store && ./gradlew test ..."
} ...
}
stage('frontend tests') {
try {
sh "cd store && ./gradlew npm_run_test-ci ..."
} ...
}
stage('packaging') {
sh "cd store && ./gradlew bootWar ..."
...
}
def dockerImage
stage('build docker') {
sh "cp -R store/src/main/docker store/build/"
sh "cp store/build/libs/*.war store/build/docker/"
dockerImage = docker.build('docker-login/store', 'store/build/docker')
}
...
}
Creating the CI/CD pipeline with XebiaLabs Blueprints
Now that our application is ready, it’s time to set up the CI/CD pipeline. We will use the XebiaLabs blueprint feature to scaffold this. At the root of our applications––the ecommerce directory––run the following command:
$ xl blueprint -b gcp/microservice-ecommerce
This will prompt you to answer a bunch of questions. The proper answers are in bold––for most of them you can use the default values provided.
? What is the name of the application? e-commerce-ms ? What is the GCP project ID? e-commerce-devops-playground ? What is the username for Kubernetes cluster? admin ? What is the password for Kubernetes cluster? (minimum 16 characters) ************************* ? Select the GCP region: europe-west1 ? Do you want to provision a new GCP GKE cluster? Yes ? What is the name of the cluster? e-commerce-ms ? What is the name of the namespace? This should be same as the one used in Kubernetes files. xl-demo ? Do you want to enable CI/CD integration with Jenkins? Yes ? XL Deploy URL for XL Release (This will only be used in the XL Release template) http://xl-deploy:4516 ? Confirm to generate blueprint files? Yes
Note: If you have an existing Google Project ID you can use that and select the Google Cloud Platform region that is best suited for you. If you do not yet have an ID you create one through the Google Cloud GUI or using the command below.
$ gcloud projects create e-commerce-devops-playground \ — organization [YOUR_ORG_ID] \ — set-as-default
$gcloud beta billing projects link e-commerce-devops-playground \ — billing-account [YOUR_BILLING_ACCOUNT_ID]
The value of YOUR_ORG_ID and YOUR_BILLING_ACCOUNT_ID can be found by running these commands:
$ gcloud organizations list $ gcloud beta billing accounts list
This will generate the XebiaLabs YAML files under the xebialabs folder, Terraform provisioning scripts under a terraform folder, and a docker-compose file under the docker folder. It will also create a xebialabs.yaml file at the root for execution.
DevOps as Code
Let’s have a quick look at the XebiaLabs YAML files generated:
xld-infra-env.yaml: This defines the GCP infrastructure and the environment to be used by XL Deploy to provision the Terraform infrastructure and to deploy our application.
xld-kubernetes-*-app.yaml: These files define the XL Deploy deployment packages for each of our microservice applications.
xld-terraform-apps.yaml: This file defines the XL Deploy deployment packages for the Terraform provisioning scripts that create our Kubernetes cluster and the namespace.
xlr-pipeline-ci-cd.yaml: This file defines the entire provisioning and release orchestration pipeline for XL Release. If you prefer, you can also use separate XL Release pipelines for each. XL Release will skip the provisioning steps if the infrastructure doesn’t have any changes.
xlr-pipeline-destroy.yaml: This file defines a pipeline to de-provision and un-deploy everything.
Infrastructure as Code using Terraform
We also generated some Terraform scripts to provision our infrastructure:
gke: This module defines the GKE cluster.
vpc: This module defines the VPC, Subnets, and Firewall rules for the cluster.
main.tf: This is the main module that ties everything together.
backend.tf: This is the backend module that defines a Google Cloud Storage backend for storing the Terraform state.
Deploying to Google Cloud Platform using XL Release
Now, let’s start our CI/CD journey.
Google Cloud Environment Preparation
Before we jump in we need to set up a Google Cloud project. Detailed steps for doing this are described in the Prerequisites section of xebialabs/USAGE.md, which is generated along with the blueprint in the previous step.
We need to make sure that the gcloud CLI is setup and authenticated. We then need to refer to the xebialabs/USAGE.md file to set up the Google Cloud Project, a service account, and the back-end storage for the tfstate file in Cloud Storage.
From this step, we will have downloaded a account.json file. We will copy that file into the terraform folder that was generated when we executed the blueprint in the previous step.
Setting up XL Release, XL Deploy, and Jenkins with Docker
We also need to run an instance of XL Release, XL Deploy, Jenkins, and Terraform. We’ll use the preconfigured docker-compose file generated by the blueprint to quickly run these tools.
Navigate to the docker folder generated by the blueprint.
$ cd ecommerce/docker
Before we start running the containers we need to specify a few environment variables for our Jenkins setup.
$ export GITHUB_USER=$ export DOCKER_USER= $ export DOCKER_PASS=
We then need to ensure that Jenkins can build our application. In the provided docker-compose setup there are preconfigured Jenkins jobs for building the application. It looks for a GitHub repository https://github.com/${GITHUB_USER}/e-commerce-microservice.git with a gke-blueprint branch to check out the application. Please create the e-commerce-microservice repository under your GitHub user, commit everything we have generated so far, and push it to that repository. If you can’t use that name update the generated docker/jenkins/jenkins.yaml file to reflect the new repository/branch name you want to use. Follow these steps to initiate and push to the Git repository.
$ cd ecommerce $ git init $ git remote add upstream https://github.com/$GITHUB_USER/e-commerce-microservice.git $ git add --all $ git commit -am "your message here" $ git checkout -b gke-blueprint $ git push upstream gke-blueprint
Now we can start the Docker containers by running the command below.
$ docker-compose up -d
This will spin up our containers in daemon mode so that we don’t accidentally kill them by ctrl+c.
We’ll visit the following URLs to ensure that all services are running correctly––we should be able to see the UI for the services. It could take a few minutes for all services to be up and running.
XL Release: localhost:5516
XL Deploy: localhost:4516
Jenkins: localhost:8080
Note: The username and password is “admin” for all these services. You can change them in the docker-compose files if you like.
Applying the XebiaLabs DevOps as Code YAML
Now that everything is ready, let’s use the generated XL YAML files. We’ll apply them using the following command from the project root.
$ xl apply -f xebialabs.yaml
This will apply all the XL Release and XL Deploy configurations and will print out information regarding what was done.
XL Deploy Deployments
We’ll take a look at what was created in XL Deploy via the explorer tab. From there we’ll expand the entries in Applications, Environments, and Infrastructure and play around to see how the Terraform and Kubernetes deployments are mapped. Visit the XL Deploy documentation to understand the concepts and the GUI.
XL Release Pipeline
We’ll then take a look at what was created in XL Release via the design tab, clicking on the “e-commerce-ms” folder (you may have used a different name during blueprint generation). Now click on “e-commerce-ms-ci-cd” from the list to see the pipeline template. Visit the XL Release documentation to understand the concepts and the GUI.
Running the Release Pipeline
Now we can run our release. In the XL Release pipeline template, click on the “new release” button, fill in a name for the release and click “create.” In the resulting pipeline click “start release” and then “start.”
The release will start by provisioning a GKE cluster, then will build our applications using Jenkins, which will build and push Docker images to Docker registry. XL Release will then orchestrate the Kubernetes deployment to the cluster that we created, run a smoke test, and present the URL for verification. You can click on each task to see more details and to see the log outputs. The release should take about 15 to 20 minutes if everything goes well.
Microservice Gateway
By clicking on the URL presented after the release, we’ll see the dapper looking guy below greeting you. Explore the application and play with the different services. You can mark the final task of the release as complete once you have verified everything. This will complete the release.
Google Cloud Console
We need to look at the Google Cloud Console to see our GKE cluster, and at the node and workloads respectively.
We can see from the shots above that all of the services are deployed and are running successfully.
Teardown and De-Provisioning with XL Release
The blueprint has also generated a handy teardown template so that we can un-deploy and de-provision everything we created. We can do so by visiting the design tab in XL Release and clicking on the “e-commerce-ms” folder. We can then click on “e-commerce-ms-destroy” from the list to see the pipeline template.
This template will un-deploy the applications with Kubernetes and will de-provision the GKE infrastructure with Terraform. To run this, we need to click on the “new release” button, fill in a name for the release, and click “create.” In the resulting pipeline click “start release” and then “start.”
Conclusion
Doing DevOps for microservices is not easy. And simple orchestration tools fall short when it comes to orchestrating complex microservices-based deployment pipelines. Of course, we can write our own orchestration scripts with a script runner like Jenkins, but scripts don’t scale when it comes to big teams and enterprise environments. This is where the XebiaLabs DevOps Platform shines, taking care of all the annoying bits and pieces of DevOps, like process, quality, and security assurances.
As I highlighted earlier, if you have simple apps, this may not be for you. But if you are on a process-oriented team that has to care about security, quality, and all of the other boring stuff that big companies and enterprises care about, then the XebiaLabs DevOps Platform is the perfect fit.
The post DevOps as Code: The Continuous Delivery of Microservices appeared first on XebiaLabs.
