Go back to Blogs
Automate your Deployment with Jenkins Pipeline for Continuous Delivery
Introduction
Deploying applications efficiently and reliably is a crucial step for development teams, especially when working with complex environments or multiple stages of deployment, such as dev, staging, and prod. Jenkins, a popular open-source automation server, has become a staple in continuous integration (CI) and continuous deployment (CD) pipelines, helping teams streamline and automate the deployment process. In this article, we’ll explore Jenkins deployment, its architecture, best practices, and how to set up a deployment pipeline.
What is Jenkins?
Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks related to building, testing, and delivering or deploying software. Jenkins can be installed through native system packages, Docker, or even run standalone by any machine with a Java Runtime Environment (JRE) installed.
What is a Jenkins Pipeline?
Jenkins Pipeline is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins. A continuous delivery pipeline is an automated expression of your process for getting software from version control right through to your users and customers.
Jenkins Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines “as code”. The definition of a Jenkins Pipeline is written into a text file (called a Jenkinsfile) which in turn can be committed to a project’s source control repository. This is the foundation of “Pipeline-as-code”; treating the CD pipeline as a part of the application to be versioned and reviewed like any other code.
The figure below illustrates Jenkins Pipeline steps in DEV, STAGING AND PROD deployment
Why Pipeline?
Jenkins is, fundamentally, an automation engine which supports a number of automation patterns. Pipeline adds a powerful set of automation tools onto Jenkins, supporting use cases that span from simple continuous integration to comprehensive CD pipelines. By modeling a series of related tasks, users can take advantage of the many features of Pipeline:
- Code: Pipelines are implemented in code and typically checked into source control, giving teams the ability to edit, review, and iterate upon their delivery pipeline.
- Durable: Pipelines can survive both planned and unplanned restarts of the Jenkins controller.
- Pausable: Pipelines can optionally stop and wait for human input or approval before continuing the Pipeline run.
- Versatile: Pipelines support complex real-world CD requirements, including the ability to fork/join, loop, and perform work in parallel.
- Extensible: The Pipeline plugin supports custom extensions to its DSL and multiple options for integration with other plugins.
Why Jenkins for Deployment?
Jenkins provides a robust platform for automating deployments, which allows developers and DevOps engineers to:
- Reduce the risk of human error.
- Deploy quickly and reliably.
- Roll back easily if something goes wrong.
- Use plugins for managing different stages of deployment across multiple environments.
- Gain visibility and control over deployment pipelines with logging and reporting features.
Key Components of Jenkins Deployment
- Jobs and Pipelines: Jenkins jobs are at the core of deployment. You can use Freestyle projects for simple tasks or Jenkins Pipeline projects for more complex, customizable workflows. Jenkins pipelines can be scripted with Groovy in the “Pipeline as Code” approach, which makes defining stages, tasks, and parameters flexible and manageable in code.
- Stages: Stages in a Jenkins pipeline represent different phases in the deployment process (e.g., build, test, and deploy). Stages allow parallelization and help visualize the progress of each phase in the Jenkins UI.
- Nodes and Executors: Jenkins nodes (or agents) run on separate machines to perform specific deployment tasks. Executors handle job execution and distribute workloads across nodes, enabling parallel deployments and reducing bottlenecks.
- Plugins: Jenkins has an extensive plugin library that enhances its deployment capabilities. Some essential plugins for deployment include GitHub/GitLab for source control integration, Docker for containerized applications, Kubernetes, and AWS/GCP for cloud deployments.
- Credentials Management: Jenkins allows you to securely manage credentials, such as SSH keys and API tokens, needed for deployment tasks. It keeps these sensitive details encrypted, and jobs can reference them without exposing the actual values.
Setting up a Jenkins Deployment Pipeline
Configure Jenkins and Install Plugins
- Install Jenkins: Jenkins is typically run as a standalone application in its own process. Jenkins can be installed on a dedicated server / platform with a version of JDK supported by Jenkins or it can be used as a cloud-based Jenkins, depending on your infrastructure. More details on how to set up Jenkins in various platforms is provided in the official Jenkins documentation.
- Install Required Plugins: For a deployment pipeline, consider plugins for source control (GitHub/GitLab), build tools (Maven/Gradle), cloud deployment (AWS, Azure), and notifications (Slack, email).
Set Up Source Control Integration
Configure Jenkins to connect with your source code repository. If using GitHub, install the GitHub plugin and create a Jenkins job that triggers on code changes. You can set up webhooks in GitHub to automatically notify Jenkins when a new code push occurs, initiating the deployment pipeline.
Define the Jenkinsfile
A Jenkinsfile is a text file that defines the steps of your CI/CD pipeline in code, making the process reproducible and maintainable. Here’s is an example of Jenkinsfile for a simple deployment pipeline in dev, staging and prod env in AWS cloud service (e.g EC2 instance):
pipeline {
agent any
environment {
// Path to your SSH key
SSH_KEY_PATH = "~/.ssh/your-ec2-key.pem"
// The package to deploy (tar.gz, zip, etc.)
DEPLOY_PACKAGE = "your-package.tar.gz"
// Path to the deploy script
DEPLOY_SCRIPT = "./deploy.sh"
}
stages {
stage('Deploy to Dev - Feature or Master') {
when {
anyOf {
// Deploy when changes are pushed to feature branches
branch pattern: "feature/.*"
// Deploy automatically to dev when changes are merged into master
branch 'master'
}
}
steps {
script {
echo "Deploying to dev environment..."
sh """
${DEPLOY_SCRIPT} ${DEPLOY_PACKAGE} dev-ec2-ip /var/www/dev dev
"""
}
}
}
stage('Approval for Staging') {
when {
// Approval for staging only after merging to master
branch 'master'
}
steps {
input message: "Approve deployment to staging?"
}
}
stage('Deploy to Staging') {
when {
// Deploy to staging only after approval for master branch
branch 'master'
}
steps {
script {
echo "Deploying to staging environment..."
sh """
${DEPLOY_SCRIPT} ${DEPLOY_PACKAGE} staging-ec2-ip /var/www/staging staging
"""
}
}
}
stage('Approval for Prod') {
when {
// Approval for prod only after deploying to staging
branch 'master'
}
steps {
input message: "Approve deployment to production?"
}
}
stage('Deploy to Prod') {
when {
// Deploy to prod only after approval for master branch
branch 'master'
}
steps {
script {
echo "Deploying to production environment..."
sh """
${DEPLOY_SCRIPT} ${DEPLOY_PACKAGE} prod-ec2-ip /var/www/prod prod
"""
}
}
}
}
post {
success {
echo "Deployment pipeline completed successfully!"
}
failure {
echo "Deployment failed. Check logs for details."
}
}
}
The script that handles the deployment to an AWS (EC2) instance by transferring the deployment package, extracting it, and restarting the service as necessary could be given in deploy.sh file as illustrated below:
Note: Replace APP_SERVICE, APP_PORT, EC2_IP and DEPLOY_PATH
#!/bin/bash
# Exit script on error
set -e
# Arguments
# The package to deploy (e.g., your app's tar.gz or zip file)
DEPLOY_PACKAGE=$1
# The EC2 instance IP address
EC2_IP=$2
# Path on EC2 where the app should be deployed
DEPLOY_PATH=$3
# The environment (dev, staging, prod)
ENVIRONMENT=$4
# The app service to restart
APP_SERVICE=$5
# The app po
echo "Starting deployment to $ENVIRONMENT environment..."
# Step 1: Transfer the deploy package to the EC2 instance
echo "Transferring package to EC2 instance..."
scp -i $SSH_KEY_PATH $DEPLOY_PACKAGE ec2-user@$EC2_IP:$DEPLOY_PATH
# Step 2: SSH into the EC2 instance and perform deployment tasks
echo "Connecting to EC2 instance..."
ssh -i $SSH_KEY_PATH ec2-user@$EC2_IP << EOF
echo "Connected to EC2 instance: $EC2_IP"
# Step 3: Extract the package (if it's a tarball or zip)
echo "Extracting the package..."
tar -xzvf $DEPLOY_PATH/$DEPLOY_PACKAGE -C $DEPLOY_PATH
# Step 4: (Optional) Restart the application or relevant services
echo "Restarting services..."
sudo systemctl restart $APP_SERVICE # Replace with your actual service name
# Step 5: Clean up (optional)
echo "Cleaning up old files..."
rm -f $DEPLOY_PATH/$DEPLOY_PACKAGE
# Step 6: (Optional) Confirm deployment by checking app status
echo "Checking application status..."
# Replace with your actual app's health check URL
curl http://localhost:$APP_PORT/health
EOF
echo "Deployment to $ENVIRONMENT environment completed successfully!"
How It Works?
- For Feature Branches: When you push a feature branch (e.g., feature/NGN-your-feature), the Jenkinsfile checks if the branch matches the pattern “feature/.*”. If it does, it will deploy automatically to dev without requiring any approval.
- For Master Branch: When you merge your feature branch into master, the deployment to dev will also be triggered automatically.
Triggering Events
- Push to Feature Branch
- The Deploy to Dev – Feature or Master stage runs when a push to any branch matching the pattern feature/.* occurs. This ensures that dev is updated with the latest feature changes.
- Merge into Master:
- Once a feature branch is merged into master, the same stage will run for dev, deploying the latest code to dev.
- Approval Deployment / Release in STAGING and PROD:
- The staging and prod deployments only occur after approval (as defined in the Jenkinsfile), so they will not trigger automatically when pushing a feature branch or merging to master. These steps require manual approval as specified in the pipeline.
If you push your code to the feature branch feature/NGN-my-new-feature, Jenkins will automatically trigger the deployment pipeline and deploy to dev.
Similarly, if you merge feature/NGN-my-new-feature into master, Jenkins will again automatically deploy to dev.
Setting Up Deployment Environments
Based on your infrastructure, you may deploy to:
- On-Premise Servers: Use SSH plugins to deploy to internal servers or run deployment scripts.
- Cloud Services (AWS, Azure, GCP): Jenkins offers plugins and integrations to deploy directly to cloud services.
- Kubernetes: Jenkins supports Kubernetes integration for managing containerized applications. You can deploy applications to a Kubernetes cluster from Jenkins using Kubernetes-specific plugins.
Configure Notifications and Alerts
Configuring notifications helps monitor deployment status. You can set up Slack, email, or other messaging integrations to get alerts for pipeline failures, successes, or other critical updates.
Jenkins Deployment Best Practices
- Use Declarative Pipelines: Declarative syntax in Jenkinsfile is straightforward and easier to maintain than scripted pipelines.
- Version Control the Jenkinsfile: Keeping the Jenkinsfile in version control with the project code makes it easier to track changes and roll back when necessary.
- Implement Environment-Based Deployments: Configure separate stages or jobs for each environment (e.g., Dev, QA, Production) to ensure code quality and prevent accidental production deployment.
- Automate Rollbacks: Incorporate rollback mechanisms in case a deployment fails. Jenkins can run rollback scripts or restore from the previous stable build.
- Run Smoke Tests Post-Deployment: After deployment, automate smoke tests to validate that the application is functioning correctly.
- Use Secure Credentials Management: Use Jenkins’s credentials store or integrate with a third-party vault to keep sensitive data secure.
Troubleshooting Common Jenkins Deployment Issues
- Build Failures: Check logs for errors in the Jenkins console, verify environment configurations, and ensure all dependencies are installed on agents.
- Permission Issues: Ensure Jenkins has appropriate permissions on servers and cloud environments it deploys to, and verify access credentials.
- Network Connectivity: Check that Jenkins agents can communicate with the master and that all necessary endpoints are reachable.
Conclusion
Jenkins is an exceptionally powerful tool for automating deployments, making it an essential component in modern DevOps and CI/CD workflows. By adhering to best practices and fine-tuning your Jenkins configurations, you can create robust, scalable, and maintainable deployment pipelines that minimize errors and maximize efficiency. Whether you’re deploying to on-premise servers, cloud infrastructure, or containerized environments, Jenkins offers the flexibility and extensibility required to adapt to diverse deployment scenarios, ensuring a seamless and streamlined software delivery process.