Prompt Details
Model
(claude-5-sonnet)
Token size
2,623
Example input
[pipeline_description]: GitHub Actions pipeline: lint → test → build Docker image → manual approval → deploy to production
[tech_stack]: Node.js backend, deployed to AWS ECS
[team_and_release_goal]: 6 engineers, aiming for daily deploys, currently deploying twice a week
Example output
═══════════════════════════
STEP 0 — PIPELINE CONTEXT & RISK PROFILE
═══════════════════════════
| Dimension | Classification | Rationale | Evidence Type |
|-----------|----------------|-----------|----------------|
| Pipeline maturity | Standard (CI tool) | Named tool (GitHub Actions) with a defined ordered sequence of jobs: lint → test → build → approval → deploy | STATED |
| Deployment frequency goal | Daily | Team explicitly states the target is daily deploys, currently at twice a week | STATED |
| Rollback capability | Manual | No rollback stage or automated recovery path is described anywhere in the pipeline sequence; ECS deployments without a stated circuit breaker typically require manual task-definition revert | INFERRED |
| Supply chain exposure | Med (OSS deps + container base images) | Node.js backend implies npm dependency tree; "build Docker image" implies a base image pulled from a registry — both are common OSS supply chain entry points | INFERRED |
| Primary failure risk — top 2, ranked | 1) Slow feedback (manual approval bottleneck) 2) Security in pipeline (no scan stage) | Manual approval sits directly before prod deploy with no stated automation criteria; no SAST/dependency/secret scan appears anywhere in the five listed stages | INFERRED |
Failure mode priority map:
| Failure Mode | Priority | Evidence Type | Conf |
|-------------|----------|---------------|------|
| Secrets in plaintext in pipeline config or logs | MED | INFERRED | 55 |
| No rollback path from production | HIGH | INFERRED | 70 |
| Tests that always pass (flaky green) | MED | INFERRED | 50 |
| Build artifacts not pinned — supply chain risk | MED | INFERRED | 60 |
| No environment promotion gates | HIGH | INFERRED | 75 |
| Missing observability between stages | MED | INFERRED | 55 |
═══════════════════════════
STEP 1 — PIPELINE STAGE AUDIT
═══════════════════════════
| Stage | Present? | Quality | Key Gaps | Severity | Evidence Type | Conf |
|-------|----------|---------|----------|----------|---------------|------|
| Source control integration (branch strategy, PR gates) | PARTIAL | Fair | GitHub confirmed as VCS via GitHub Actions, but no stated branch protection or required PR review before merge to main | 🟡 | INFERRED | 60 |
| Build (reproducibility, caching, artifact versioning) | YES | Fair | No stated image tagging strategy, layer caching, or pinned build args; risk of tag overwrites (e.g. `latest`) | 🟡 | INFERRED | 65 |
| Unit test gate (coverage threshold, fast feedback <5min) | YES | Fair | No coverage threshold or runtime stated; fast-feedback claim can't be confirmed | 🟡 | INFERRED | 55 |
| Integration test gate (environment, data, isolation) | NO | N/A — stage absent | No integration/e2e stage between unit test and Docker build; ECS-specific runtime/config issues won't surface until production | 🔴 | INFERRED | 70 |
| Security scan (SAST, dependency audit, secret detection) | NO | N/A — stage absent | None of the five listed stages performs SAST, dependency audit, or secret scanning; vulnerable code/deps can reach the built image | 🔴 | INFERRED | 80 |
| Artifact signing & provenance | NO | N/A — stage absent | No image signing (e.g. cosign) or SBOM generation between build and deploy | 🟡 | INFERRED | 70 |
| Environment promotion (dev → staging → prod gates) | NO | N/A — stage absent | Sequence goes build → manual approval → prod directly; no staging/pre-prod environment described | 🔴 | INFERRED | 75 |
| Deployment strategy (blue/green, canary, rolling) | UNCLEAR | N/A — not specified | No deployment strategy named for the ECS service update; default rolling behavior unconfirmed | 🟡 | INFERRED | 60 |
| Smoke test post-deploy | NO | N/A — stage absent | Manual approval occurs before deploy, not after; no post-deploy verification step exists | 🔴 | INFERRED | 70 |
| Rollback mechanism | NO | N/A — stage absent | No automated rollback stage or ECS deployment circuit breaker described | 🔴 | INFERRED | 75 |
| Observability & alerting on pipeline health | NO | N/A — stage absent | No mention of failure notifications, dashboards, or DORA-style metrics tracking | 🟡 | INFERRED | 60 |
═══════════════════════════
STEP 2 — SECURITY IN THE PIPELINE (SUPPLY CHAIN)
═══════════════════════════
| Attack Vector | Exposed? | Mechanism | Mitigation | Severity | Evidence Type |
|--------------|----------|-----------|------------|----------|---------------|
| Secrets in env vars or logs | UNCLEAR | AWS credentials for ECS deploy are necessarily present somewhere in the workflow; handling method (static keys vs. federated) not stated | Use GitHub OIDC to assume an AWS IAM role; ensure secrets are masked in Action logs | 🟡 | INFERRED |
| Unpinned base images or dependencies | YES | "Build Docker image" stage with no stated base-image pinning or lockfile enforcement (e.g. `npm ci` vs `npm install`) | Pin base image to a digest; use `npm ci` against a committed lockfile | 🟡 | INFERRED |
| Unsigned artifacts promoted to production | YES | Image moves from build → manual approval → deploy with no signing or verification step described | Sign images with cosign post-build; verify signature before ECS task update | 🟡 | INFERRED |
| Pipeline-as-code with no peer review | UNCLEAR | Workflow YAML lives in the repo (implied by GitHub Actions) but review requirements on `.github/workflows` changes aren't stated | Require CODEOWNERS approval on workflow file changes | 🟢 | INFERRED |
| Third-party actions/plugins without version pinning | YES | Typical GH Actions pipelines (checkout, docker build-push, etc.) reference marketplace actions by floating tag; pinning not confirmed | Pin all third-party actions to full commit SHA | 🟡 | INFERRED |
| OIDC / identity federation missing | UNCLEAR | Deploy stage requires AWS auth for ECS; OIDC vs. long-lived key usage not stated | Configure GitHub OIDC trust with a scoped AWS IAM role, remove static keys if present | 🟡 | INFERRED |
═══════════════════════════
STEP 3 — ASSUMPTION LEDGER & SPEED ANALYSIS
═══════════════════════════
Assumption ledger:
| Assumption | Where It Enters | If False — Pipeline Behavior |
|------------|----------------|------------------------------|
| Manual approval is a human review gate (GitHub environment protection rule), not an automated check | Deploy stage; Step 0 risk ranking | If it's actually an automated gate, the "slow feedback" bottleneck analysis is wrong and velocity constraint lies elsewhere |
| No separate staging/dev ECS cluster exists beyond production | Step 1 environment-promotion row; Step 0 | If a staging cluster exists but was omitted, environment-promotion severity drops from 🔴 to 🟡/🟢 |
| "Test" stage means unit tests only, not integration/e2e | Step 1 rows 3–4 | If integration coverage is already included, the standalone integration-test-gap finding is invalid |
| AWS deploy credentials are static Actions secrets, not OIDC-federated | Step 2 rows 1 and 6 | If OIDC is already configured, both rows shift from 🟡 to 🟢 |
Pipeline speed analysis:
| Stage | Estimated Duration | Parallelizable? | Caching Opportunity | Evidence Type |
|-------|------------------|----------------|--------------------|----|
| Source control integration | N/A — not an executed stage, it's a repo/config gate | N/A — no execution to parallelize | N/A — not applicable | INFERRED |
| Build (Docker image) | ~3–5 min | NO — must follow test | YES — Docker layer caching / GH Actions cache for npm install layer | INFERRED |
| Unit test gate | ~2–4 min | YES — test suites can shard across runners | YES — cache `node_modules` keyed on lockfile hash | INFERRED |
| Integration test gate | N/A — stage absent, no execution exists | N/A — stage not present | N/A — stage not present | INFERRED |
| Security scan | N/A — stage absent | N/A — stage not present | N/A — stage not present | INFERRED |
| Artifact signing & provenance | N/A — stage absent | N/A — stage not present | N/A — stage not present | INFERRED |
| Environment promotion | N/A — stage absent | N/A — stage not present | N/A — stage not present | INFERRED |
| Deployment strategy execution | ~2–5 min (ECS service update + task replacement) | NO — sequential rollout for availability | NO | INFERRED |
| Smoke test post-deploy | N/A — stage absent | N/A — stage not present | N/A — stage not present | INFERRED |
| Rollback mechanism | N/A — stage absent, no automated execution path | N/A — not present | N/A — not present | INFERRED |
| Observability & alerting | N/A — stage absent | N/A — not present | N/A — not present | INFERRED |
Total estimated pipeline duration: Automated stages (lint + test + build + deploy) likely total roughly 8–16 minutes of machine execution time. The manual approval step adds an unbounded, human-dependent wait on top of that.
Target vs. current gap: The goal is daily deploys; current cadence is twice a week. Since the automated stages sum to well under 20 minutes, the manual approval wait is the most probable dominant bottleneck (INFERRED) — not raw pipeline execution speed. Closing the gap likely requires reducing reliance on blanket manual approval rather than optimizing lint/test/build runtime.
═══════════════════════════
STEP 4 — INPUT ROBUSTNESS SCENARIOS
═══════════════════════════
| Condition | First Break Point | Predicted Behavior | Recovery Path | Conf |
|-----------|------------------|--------------------|--------------|------|
| Flaky test causes false failure | Unit test gate | Pipeline fails, blocking build/deploy for that commit; no auto-retry logic is described | Engineer manually re-runs the workflow or pushes a new commit | 55 |
| Bad deploy reaches production | Deploy to production stage (post-approval) | With no smoke test or rollback mechanism described, the broken deploy stays live until someone manually notices | On-call engineer manually reverts to the previous ECS task definition via console/CLI | 70 |
| Dependency registry goes down mid-build | Build stage (npm install inside Docker build) | Build step fails on network/registry error; pipeline halts before approval, so no bad deploy reaches prod (fails safe) | Retry the build once the registry recovers; no mirror/offline cache fallback is described | 60 |
| Secret rotated but not updated in pipeline | Deploy to production stage (AWS auth) | Deployment authentication fails; workflow errors out before ECS update, but already-running prod tasks are unaffected | Manually update the GitHub Actions secret with the new credential and re-run the workflow | 65 |
═══════════════════════════
STEP 5 — PRIORITIZED IMPROVEMENTS
═══════════════════════════
| Priority | Finding | Specific Action | Effort | Impact | Evidence Type |
|----------|---------|----------------|--------|--------|---------------|
| 🔴 P1 | No environment promotion gate before production | Add a staging ECS service receiving the same image, with automated checks run there before the production approval step | Med | High | INFERRED |
| 🔴 P1 | No rollback mechanism | Enable ECS deployment circuit breaker with automatic rollback tied to health-check failures | Low | High | INFERRED |
| 🔴 P1 | No security scan stage | Add `npm audit --audit-level=high` and a container image scan (Trivy or ECR scanning) as a required job before build completes | Low | High | INFERRED |
| 🔴 P1 | No post-deploy smoke test | Add a smoke test job hitting a health-check endpoint after ECS stabilizes, failing (and triggering rollback) on error | Low | High | INFERRED |
| 🟡 P2 | Manual approval likely the primary bottleneck against the daily-deploy goal | Automate gates (tests + smoke checks) for low-risk changes; reserve manual approval for flagged high-risk changes only | Med | High | INFERRED |
| 🟡 P2 | Unpinned third-party actions and Docker base images | Pin marketplace actions to commit SHA and base image to a digest; enable Dependabot | Low | Med | INFERRED |
| 🟡 P2 | No artifact signing | Sign Docker images with cosign post-build; verify signature before ECS deploy | Med | Med | INFERRED |
| 🟡 P2 | Unclear AWS credential handling | Migrate deploy job from static access keys (if used) to GitHub OIDC federated role assumption | Med | Med | INFERRED |
| 🟢 P3 | No pipeline observability/alerting | Add Slack/email notification on workflow failure and a dashboard tracking deploy frequency and lead time | Low | Med | INFERRED |
| 🟢 P3 | Branch/PR gate strategy unconfirmed | Enforce required PR review + status checks on main via GitHub branch protection rules | Low | Low | INFERRED |
═══════════════════════════
STEP 6 — PIPELINE INTEGRITY CHECK, CONFIDENCE & OPEN QUESTIONS
═══════════════════════════
| Check | Status | Note |
|-------|--------|------|
| Security scan stage covers secrets, dependencies, and SAST | ❌ | No security scan stage exists anywhere in the described sequence |
| Rollback mechanism is explicit and tested | ❌ | No rollback stage or automated recovery path is mentioned |
| Every stage has a pass/fail gate (no silent skips) | ⚠️ | Lint/test/build presumably gate on failure, but the manual approval step's rejection behavior isn't described |
| Pipeline duration analysis addresses the stated release frequency goal | ✅ | Speed analysis identifies manual approval as the likely bottleneck against the daily-deploy target |
| Supply chain attack vectors are all assessed | ✅ | All 6 vectors in Step 2 evaluated against available information |
| All INFERRED findings are labeled | ✅ | Every row across all tables carries an explicit STATED/INFERRED tag |
AUDIT CONFIDENCE: 62 — Most structural gaps (security scan, staging, rollback) are inferred from stage omission rather than a confirmed absence, so the real configuration could differ from what's assessed here.
Biggest unknown: Whether a staging/pre-production ECS environment exists but was simply left out of the description — its presence would materially lower both the environment-promotion and rollback risk ratings.
STATED vs. VERIFY breakdown:
- STATED: GitHub Actions is the CI/CD tool; stage order is lint → test → build Docker image → manual approval → deploy to production; backend is Node.js; deployment target is AWS ECS; team size is 6 engineers; release goal is daily deploys; current cadence is twice a week.
- VERIFY: Whether any security/dependency/secret scan exists in the actual workflow; whether AWS credentials use OIDC or static keys; whether a staging/dev ECS environment exists; whether any rollback mechanism (ECS circuit breaker or manual runbook) is configured; whether third-party Actions and the Docker base image are version-pinned; whether smoke tests run post-deploy; what specifically causes the current twice-weekly cadence (approval wait time vs. another bottleneck); branch protection/PR review requirements on main.
Get this prompt + 9 more with Select$14 your first month, then $19/mo$14 first month, then $19/mo
View SelectBy purchasing this prompt, you agree to our terms of service
CLAUDE-5-SONNET
⚙️A senior DevOps audit of your CI/CD pipeline — security, reliability, speed, and observability in one pass. Classifies pipeline maturity and rollback capability, then scans for plaintext secrets, missing rollback paths, flaky 'always-green' tests, unpinned build artifacts, and missing promotion gates. Every finding is severity-tagged with a confidence score and a prioritized fix list. STATED vs INFERRED labeling throughout — no vague 'best practices' filler.🚦
...more
Added over 1 month ago
