2024-11-10 6 min read

Secrets Sprawl: Stopping Credential Leaks in Your Codebase

Credentials in code aren't just sloppy—they're a direct path to compromise. Learn how secrets end up in repos and proven tactics to remove them.

Secrets Sprawl: Stopping Credential Leaks in Your Codebase

Your developer just committed a database password to GitHub. Your CI/CD pipeline shipped an API key in a Docker image. A contractor left AWS credentials in a commented-out section three months ago. These aren't hypotheticals—they happen daily across teams of all sizes, and each one is a live vulnerability waiting to be discovered.

Secrets sprawl is the silent killer of application security. Unlike a single catastrophic breach, credential leaks accumulate quietly across repositories, branches, and artifacts. By the time you notice, unauthorized access may already be underway.

How Secrets End Up in Code

Credential exposure rarely happens by malice. It's friction meeting convenience:

Development Workflows

Developers hardcode secrets to move fast locally, intending to remove them before committing. That intention often doesn't survive a late-night push:

python
# Bad: credentials in source
db_password = "MySecurePass123!"
api_key = "sk_live_51234567890abcdef"
connection = psycopg2.connect(
    host="prod-db.internal",
    password=db_password,
    user="app_user"
)

Configuration Files

Config files tracked in git—especially those copied across environments—often contain production secrets meant for

code
.env
files or secret managers:

bash
# .env checked in by accident
DATABASE_URL=postgresql://user:prod_password@db.prod.internal:5432/app
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Historical Commits

Even if you delete a secret from the current branch, it lives in git history. A shallow clone doesn't guarantee safety. Attackers routinely scan public repositories for exposed credentials, and private repositories are breached more often than teams assume.

Build Artifacts

Secrets baked into Docker images, compiled binaries, or Lambda packages persist across deployments. A single exposed image tag can compromise entire environments.

Detection and Cleanup

Scan Your History Now

Start with a complete audit. Tools like

code
git-secrets
and
code
TruffleHog
scan repositories for common patterns:

bash
# Install and configure git-secrets
git clone https://github.com/awslabs/git-secrets.git
cd git-secrets && make install
git secrets --install

# Scan your entire history
git secrets --scan --all

# Or use TruffleHog for entropy-based detection
pip install truffleHog
truffleHog filesystem /path/to/repo --json

Remove Secrets from History

If you find credentials, you must rewrite history—not just delete from the current commit. Tools like

code
BFG Repo-Cleaner
are faster than
code
git filter-branch
:

bash
# Install BFG
brew install bfg  # or download from https://rtyley.github.io/bfg-repo-cleaner/

# Remove all occurrences of a specific secret
bfg --replace-text secrets.txt my-repo.git

# Or remove files that should never have existed
bfg --delete-files passwords.txt my-repo.git

# Force-push cleaned history
cd my-repo && git reflog expire --expire=now --all && git gc --prune=now
git push origin --force --all

Warn your team: force-pushes break existing clones. Everyone must re-fetch.

Prevent Future Leaks

Detection and cleanup are band-aids. Prevention is the actual fix:

  • Pre-commit hooks: Use
    code
    detect-secrets
    or
    code
    git-secrets
    in your CI/CD pipeline to reject commits containing credentials
  • Secret managers: Store all secrets in HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault—never in code or config files
  • Scanning in CI: Run TruffleHog on every pull request before merge
  • Rotation: Any leaked credential should be rotated immediately, regardless of whether it's active

Moving Forward

Secrets sprawl compounds over time. A codebase without a prevention strategy will leak credentials again. At LavaPi, we've seen teams rotate from detecting one exposed key to discovering dozens hidden across years of history. The fix is straightforward: scan now, remove completely, and enforce prevention at the CI/CD gate.

Start with a single repository. One audit, one cleanup, one integrated secret-scanning tool in your pipeline. That foundation matters more than perfection across your entire estate.

Share
LP

LavaPi Team

Digital Engineering Company

All articles