Introduction
In a DevOps-driven environment, real-time insights into CI/CD pipelines are crucial for efficient development and deployment workflows. While GitLab provides default email notifications for pipeline success or failure, they often lack detailed information—making it harder to troubleshoot failures or track deployment progress efficiently.
To overcome this limitation, I implemented customized email notifications within the GitLab CI/CD pipeline using an SMTP server*. These emails include commit details, logs, and deployment links for both **success and failure cases**, improving traceability and issue resolution.*
The Challenge: Lack of Contextual Notifications
By default, GitLab’s email notifications only provide basic status updates*, which are not always enough for debugging or tracking deployments effectively. The primary issues were:*
✅ No detailed success notifications*: Missing commit details, timestamps, and deployment URLs. ✅ **Failure emails lacked debugging information*: No attached logs to diagnose what went wrong. ✅ *No structured reporting**: Manually checking the GitLab UI for logs and debugging was time-consuming.*
To bridge this gap, I automated detailed, structured email reports that provide the necessary context for both successful and failed deployments
Solution: Automating Email Notifications in GitLab CI/CD
I configured the GitLab CI/CD pipeline to send detailed email reports using an SMTP server*. These emails contain:*
📌 Success notifications with commit details, deployment links, and attached logs. 📌 Failure notifications with commit information, error logs, and failure reasons for easy debugging. 📌 Automated log extraction using GitLab APIs and built-in CI/CD environment variables.
Key Components Implemented
✅ SMTP integration for sending emails programmatically. ✅ GitLab API usage to fetch commit details and timestamps. ✅ Dynamic log extraction to attach relevant pipeline execution logs.
Pipeline Configuration: Sending Emails via SMTP
notify_on_success:
stage: success
needs:
- build-job
- deploy-job
script:
# Install dependencies
- apt-get update -y && apt-get install -y curl jq mailutils ssmtp
# Save the success message to a file
- |
SUCCESS_FILE="pipeline_success_message.txt"
BUILD_LOG_FILE="build_job_logs.txt"
DEPLOY_LOG_FILE="deploy_job_logs.txt"
# Initialize the success file
echo "Pipeline Success Details:" > $SUCCESS_FILE
echo "Pipeline: $CI_PROJECT_PATH" >> $SUCCESS_FILE
echo "Branch: $CI_COMMIT_REF_NAME" >> $SUCCESS_FILE
echo "Commit: $CI_COMMIT_SHA" >> $SUCCESS_FILE
# Fetch committer's name and commit timestamp using GitLab API
COMMIT_INFO=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
"https://devgit.craftmyapp.in/api/v4/projects/$CI_PROJECT_ID/repository/commits/$CI_COMMIT_SHA")
COMMITTER_NAME=$(echo "$COMMIT_INFO" | jq -r '.committer_name // "null"')
COMMITTER_EMAIL=$(echo "$COMMIT_INFO" | jq -r '.committer_email // "null"')
COMMIT_TIME=$(echo "$COMMIT_INFO" | jq -r '.committed_date // "null"')
if [[ "$COMMITTER_NAME" != "null" && "$COMMIT_TIME" != "null" ]]; then
IST_COMMIT_TIME=$(date -d "$COMMIT_TIME + 5 hours 30 minutes" '+%Y-%m-%d %H:%M:%S IST')
echo "Committer: $COMMITTER_NAME" >> $SUCCESS_FILE
echo "Commit Time (IST): $IST_COMMIT_TIME" >> $SUCCESS_FILE
else
echo "Committer: Unable to fetch (check permissions or API response)" >> $SUCCESS_FILE
echo "Commit Time: Unable to fetch (check permissions or API response)" >> $SUCCESS_FILE
fi
# Determine the correct appsettings file based on branch name
BRANCH_NAME=$(echo "$CI_COMMIT_REF_NAME" | tr '[:upper:]' '[:lower:]')
APPSETTINGS_DIR="webapp/Admin"
case "$BRANCH_NAME" in
production)
SELECTED_FILE="$APPSETTINGS_DIR/appsettings.Production.json"
;;
generated)
SELECTED_FILE="$APPSETTINGS_DIR/appsettings.Generated.json"
;;
qa)
SELECTED_FILE="$APPSETTINGS_DIR/appsettings.Qa.json"
;;
develop|development)
SELECTED_FILE="$APPSETTINGS_DIR/appsettings.Development.json"
;;
*)
SELECTED_FILE="$APPSETTINGS_DIR/appsettings.json"
;;
esac
echo "Selected appsettings file: $SELECTED_FILE"
if [[ -f "$SELECTED_FILE" ]]; then
BASEURL=$(jq -r '.ApiSettings.baseURL' "$SELECTED_FILE")
echo "Deployment URL: $BASEURL" >> $SUCCESS_FILE
else
echo "Deployment URL: Not found (appsettings file missing)" >> $SUCCESS_FILE
fi
# Fetch and save Build Job Logs
BUILD_JOB_ID=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
"https://devgit.craftmyapp.in/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs?scope[]=success" | jq -r '.[] | select(.name=="build-job") | .id')
if [[ -n "$BUILD_JOB_ID" ]]; then
curl --silent --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
"https://devgit.craftmyapp.in/api/v4/projects/$CI_PROJECT_ID/jobs/$BUILD_JOB_ID/trace" > $BUILD_LOG_FILE
echo "Build Job Logs saved to $BUILD_LOG_FILE" >> $SUCCESS_FILE
else
echo "Build Job Logs not available." > $BUILD_LOG_FILE
echo "Build Job Logs not available." >> $SUCCESS_FILE
fi
# Fetch and save Deploy Job Logs
DEPLOY_JOB_ID=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
"https://devgit.craftmyapp.in/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs?scope[]=success" | jq -r '.[] | select(.name=="deploy-job") | .id')
if [[ -n "$DEPLOY_JOB_ID" ]]; then
curl --silent --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
"https://devgit.craftmyapp.in/api/v4/projects/$CI_PROJECT_ID/jobs/$DEPLOY_JOB_ID/trace" > $DEPLOY_LOG_FILE
echo "Deploy Job Logs saved to $DEPLOY_LOG_FILE" >> $SUCCESS_FILE
else
echo "Deploy Job Logs not available." > $DEPLOY_LOG_FILE
echo "Deploy Job Logs not available." >> $SUCCESS_FILE
fi
# Configure mail command (ssmtp or other MTA)
- |
echo "mailhub=$SMTP_SERVER:587
AuthUser=$SMTP_USER
AuthPass=$SMTP_PASS
UseSTARTTLS=YES" > /etc/ssmtp/ssmtp.conf
# Send the email
- |
if [[ "$COMMITTER_EMAIL" != "null" ]]; then
echo "Sending email to: $COMMITTER_EMAIL"
mail -s "Pipeline Success Notification for $CI_PROJECT_PATH" \
-A $BUILD_LOG_FILE -A $DEPLOY_LOG_FILE \
"$COMMITTER_EMAIL" < $SUCCESS_FILE
else
echo "No committer email found. Skipping email."
fi
when: on_success
only:
- CICD
notify_on_failure:
stage: failure
script:
# Install dependencies
- apt-get update -y && apt-get install -y curl jq mailutils ssmtp
# Fetch the last failed job and prepare the email content
- |
OUTPUT_FILE="pipeline_failure_details.txt"
FAILED_JOB_LOG_FILE="last_failed_job_log.txt"
ERROR_SNIPPET_FILE="error_snippet_log.txt"
echo "Fetching the last failed job..."
FAILED_JOBS=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
"https://devgit.craftmyapp.in/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs?scope[]=failed")
LAST_FAILED_JOB=$(echo "$FAILED_JOBS" | jq 'last // empty')
if [[ -z "$LAST_FAILED_JOB" ]]; then
echo "No failed jobs found."
exit 1
fi
# Extract job details
JOB_ID=$(echo "$LAST_FAILED_JOB" | jq -r '.id')
JOB_NAME=$(echo "$LAST_FAILED_JOB" | jq -r '.name')
JOB_FAILURE_REASON=$(echo "$LAST_FAILED_JOB" | jq -r '.failure_reason // "Unknown failure"')
# Fetch committer's name and commit timestamp using GitLab API
COMMIT_INFO=$(curl --silent --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
"https://devgit.craftmyapp.in/api/v4/projects/$CI_PROJECT_ID/repository/commits/$CI_COMMIT_SHA")
COMMITTER_NAME=$(echo "$COMMIT_INFO" | jq -r '.committer_name // "null"')
COMMITTER_EMAIL=$(echo "$COMMIT_INFO" | jq -r '.committer_email // "null"')
COMMIT_TIME=$(echo "$COMMIT_INFO" | jq -r '.committed_date // "null"')
if [[ "$COMMITTER_NAME" != "null" && "$COMMIT_TIME" != "null" ]]; then
IST_COMMIT_TIME=$(date -d "$COMMIT_TIME + 5 hours 30 minutes" '+%Y-%m-%d %H:%M:%S IST')
else
IST_COMMIT_TIME="Unable to fetch (check permissions or API response)"
fi
# Save the failure details to an output file
echo "Pipeline Failure Details:" > $OUTPUT_FILE
echo "Pipeline: $CI_PROJECT_PATH" >> $OUTPUT_FILE
echo "Branch: $CI_COMMIT_REF_NAME" >> $OUTPUT_FILE
echo "Commit: $CI_COMMIT_SHA" >> $OUTPUT_FILE
echo "Committer: $COMMITTER_NAME" >> $OUTPUT_FILE
echo "Commit Time (IST): $IST_COMMIT_TIME" >> $OUTPUT_FILE
echo "Job Name: $JOB_NAME" >> $OUTPUT_FILE
echo "Failure Reason: $JOB_FAILURE_REASON" >> $OUTPUT_FILE
# Fetch the log for the last failed job and save it to a separate file
curl --silent --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
"https://devgit.craftmyapp.in/api/v4/projects/$CI_PROJECT_ID/jobs/$JOB_ID/trace" > $FAILED_JOB_LOG_FILE
echo "Last Failed Job Log saved to $FAILED_JOB_LOG_FILE."
# Extract error snippet and save it to a separate file
command_line=$(grep -nF "\$" $FAILED_JOB_LOG_FILE | tail -n 1 | cut -d: -f1)
error_line=$(grep -nF "ERROR" $FAILED_JOB_LOG_FILE | tail -n 1 | cut -d: -f1)
if [[ -n "$command_line" && -n "$error_line" ]]; then
sed -n "${command_line},${error_line}p" $FAILED_JOB_LOG_FILE > $ERROR_SNIPPET_FILE
echo "Error snippet saved to $ERROR_SNIPPET_FILE."
else
echo "Error snippet could not be extracted." > $ERROR_SNIPPET_FILE
fi
# Configure mail command (ssmtp or other MTA)
- |
echo "mailhub=$SMTP_SERVER:587
AuthUser=$SMTP_USER
AuthPass=$SMTP_PASS
UseSTARTTLS=YES" > /etc/ssmtp/ssmtp.conf
# Send the email with attachments
- |
if [[ "$COMMITTER_EMAIL" != "null" ]]; then
echo "Sending email to: $COMMITTER_EMAIL"
mail -s "Pipeline Failure Notification for $CI_PROJECT_PATH" \
-A $FAILED_JOB_LOG_FILE \
-A $ERROR_SNIPPET_FILE \
"$COMMITTER_EMAIL" < $OUTPUT_FILE
else
echo "No committer email found. Skipping email."
fi
when: on_failure
only:
- CICD
SMTP Server Configuration
To enable email delivery, I configured an SMTP server with GitLab CI/CD using environment variables:
env:
SMTP_SERVER: "smtp.yourserver.com"
SMTP_PORT: "587"
SMTP_USERNAME: "your-email@example.com"
SMTP_PASSWORD: "your-secure-password"
This ensures secure email delivery with proper authentication.
Benefits & Impact
✅ Faster Debugging*: Immediate access to logs and errors via email, reducing time spent on manual debugging. ✅ **Better Deployment Tracking*: Success emails include deployment URLs for quick access. ✅ *Proactive Issue Resolution*: Developers receive *instant failure alerts** with log attachments.*
By implementing detailed CI/CD email notifications*, our team significantly improved **deployment traceability and issue resolution efficiency**. 🚀*
Final Thoughts
Automating detailed email notifications in GitLab CI/CD has enhanced our DevOps workflow, ensuring developers stay informed about deployments and failures in real-time*. If you’re looking to improve your **CI/CD pipeline visibility*, integrating *SMTP-based email reports** can be a game-changer. 💡*
🔗 Have you automated notifications in your CI/CD pipeline? Share your experience in the comments! 👇
#DevOps #GitLab #CICD #SMTP #Automation #SoftwareEngineering #ContinuousIntegration #ContinuousDeployment