Shallow Git Clone On Jenkins Pipeline
Overview
On a beautiful day, your Jenkins can't clone from the Git repository like every day before.
The error may appear GitException with status code 128
Due to the error, you cannot clone the repository, run the test, commit the result by Jenkins.
How to deal with it?
You should first determine what the cause of the error is. In my case, the current working repository contains a lot of commit history and there will be more in the future.
There is a simple solution called "shallow clone", it is a feature of git to reduce server load. These clone truncate the commit history to reduce the clone size and copy only recent revisions. It’s fast clone, save time and disk space.
You just use the –depth option. For example
git clone --depth [depth] --branch [branch] [remote-url] [directory]
Apply shallow git clone to Jenkins Pipelines
Assuming you have a Jenkins Pipeline to clone the complete repository as follows, the pipeline works fine before the above error appears
node {
stage('Download source code from Git') {
// the below will clone your repo completely and will be checked out to master branch by default
git credentialsId: 'auto-credentials', url: 'https://gitlab.shiftasia.com/thuanpham/jenkins-docker-sample.git', branch: 'master'
}
stage('Run Test') {
// download docker image with specific version
docker.image('python:3.6.8-alpine').inside {
// check Python version
sh 'python --version'
sh 'pip list'
withEnv(["HOME=${env.WORKSPACE}"]) {
// install libraries required for testing
sh 'pip install -r requirements.txt --user'
// check libraries installed
sh 'pip list'
// execute test and update test result to input.csv
sh 'python -m pytest -n 2 --dist=each'
}
}
}
stage('Upload test result to Git') {
// save local changes to stash
sh 'git stash -u'
// pull the latest commit if any
withCredentials([usernamePassword(credentialsId: 'auto-credentials', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) {
sh "git pull https://'$USERNAME:$PASSWORD'@gitlab.shiftasia.com/thuanpham/jenkins-docker-sample.git master"
}
// get local changes from stash
sh 'git stash apply'
sh 'git add input.csv'
sh '''git commit -m "Commit from Jenkins $(date +"%d-%m-%Y")" '''
withCredentials([usernamePassword(credentialsId: 'auto-credentials', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) {
sh "git push https://'$USERNAME:$PASSWORD'@gitlab.shiftasia.com/thuanpham/jenkins-docker-sample.git master"
}
}
}
In the first stage, you need to update as follows to change complete clone to shallow clone
And the remaining stages, please keep them as they are.
stage('Download source code from Git') {
// clean workspace if the current workspace is not empty
//cleanWs()
// check txt file in current directory existing or not
def exitCode = sh script: 'find . -name "*.txt" | egrep .'
boolean exists = exitCode == 0
// if the file is existing
if (existing) {
// pull the latest commits if the current repo is shallow clone
withCredentials([usernamePassword(credentialsId: 'auto-credentials', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) {
sh "git fetch --depth 1"
sh "git reset --hard origin/master"
sh "git fetch -dfx"
}
} else {
// shallow clone the repo without project folder to the workspace
// the command works only when the current workspace is empty. Otherwise, try to run cleanWs()
withCredentials([usernamePassword(credentialsId: 'auto-credentials', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')]) {
sh "git clone --depth 1 --branch master https://'$USERNAME:$PASSWORD'@gitlab.shiftasia.com/thuanpham/jenkins-docker-sample.git ."
}
}
}
ref: how-to-update-a-git-shallow-clone
Not only the error has been resolved, but it has also significantly improved the time to clone the repository on Jenkins. The solution works fine in my case.
Just note that the current repo is shallow clone. If you want to convert it to complete/full clone, please refer to the link how-to-convert-a-git-shallow-clone-to-a-full-clone
That's all for the example, thanks for reading.
Enjoy it!