> For the complete documentation index, see [llms.txt](https://whitepaper.litho.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://whitepaper.litho.ai/docs/developers/ci-cd-guide.md). # CI/CD Guide > ⚠️ **Aspirational / roadmap doc.** Describes the K8s + ArgoCD future state. **Current production is EC2 + Docker Compose** via `deploy-simple.yaml` (SSM-over-OIDC to bastion, then `docker compose up`). For the live deployment flow see [`docs/governance/deployment-approvals.md`](/docs/governance/deployment-approvals.md) and [`docs/governance/release-process.md`](/docs/governance/release-process.md). The `clusters/` + `infra/k8s/` directories this guide references were removed on 2026-05-12 — they had been unused boilerplate since project inception. This guide describes the Lithosphere CI/CD pipeline, covering the 4-environment deployment strategy, GitHub Actions workflow structure, ArgoCD GitOps deployment, and the environment comparison matrix. It is derived from the architectural decisions documented in ADR-001 (Environment Strategy) and ADR-002 (Technology Stack). *** ## 4-Environment Strategy Lithosphere uses a 4-environment strategy aligned with blockchain development best practices: | Environment | Purpose | Persistence | Cluster | Trigger | | ----------- | -------------------------------------------- | ----------------------- | ---------------------- | ----------------------------------- | | **Local** | Developer workstation testing and debugging | Ephemeral (per session) | Docker Compose | Manual (`docker-compose up`) | | **Devnet** | CI/CD automated testing and PR validation | Ephemeral (24h max) | Kubernetes (shared) | Push to `main`, Pull Request opened | | **Staging** | Persistent testnet, integration testing, UAT | Persistent | Kubernetes (dedicated) | Merge to `release/*` branch | | **Mainnet** | Production deployment, real user traffic | Persistent | Kubernetes (dedicated) | Git tag `v*.*.*` + manual approval | *** ## Environment Details ### Local Environment The local environment runs entirely on the developer's workstation using Docker Compose. **Capabilities:** * Hot-reload for API and frontend changes * Contract deployment with instant mining via Hardhat Network * Mainnet state forking for realistic testing (configurable via `FORK_URL`) * Full observability stack (Grafana, Prometheus) available locally **Configuration:** ```bash # docker/local/.env FORK_URL=https://mainnet.lithosphere.network/rpc FORK_BLOCK_NUMBER=latest LOCAL_RPC_PORT=8545 POSTGRES_HOST=localhost REDIS_HOST=localhost ``` **Quick start:** ```bash # Start all core services docker compose up -d # Start with monitoring stack docker compose -f docker-compose.yaml -f docker-compose.monitoring.yaml up -d ``` ### Devnet Environment The devnet is an ephemeral environment created automatically during CI runs. * Seeded with deterministic test fixtures. * No approval required for deployment. * Auto-cleanup after 24 hours. * Shared Kubernetes namespace: `lithosphere-devnet`. * Maximum 2 concurrent devnet deployments. **ArgoCD Configuration:** ```yaml # clusters/devnet/argocd/application.yaml spec: syncPolicy: automated: prune: true selfHeal: true source: targetRevision: main ``` ### Staging Environment The staging environment is a persistent testnet used for integration testing, UAT, and partner testing. * Chain data is preserved across deployments. * Automatic deployment with smoke tests on merge to `release/*` branches. * PostgreSQL: Daily snapshots retained 30 days. * Chain data: Weekly snapshots retained 90 days. * Indexer: Continuous sync with testnet. **Promotion criteria to reach staging:** * All CI tests passing * Security scan (Slither) clean * Docker image signed * Contract artifacts verified ### Mainnet Environment The mainnet environment is the production deployment with the highest security requirements. * Triggered only by a Git tag matching `v*.*.*` (semantic versioning). * **Manual approval required** from CODEOWNERS before deployment. * Multi-AZ deployment across 3 availability zones. * Database: RDS Multi-AZ with read replicas. * 3 Lithosphere nodes behind a load balancer. * CDN (CloudFront) for frontend assets. **Rollback Strategy:** * ArgoCD automatic rollback on health check failure. * Manual rollback via `argocd app rollback`. * Database point-in-time recovery (PITR) enabled. *** ## Promotion Flow Code changes flow through environments in a strict promotion pipeline: ``` ┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ LOCAL │────────>│ DEVNET │────────>│ STAGING │────────>│ MAINNET │ │ │ push │ │ merge │ │ tag │ │ │ Docker │ to │ K8s │ to │ K8s │ v*.* │ K8s │ │ Compose │ main │ shared │ release│ dedicated │ dedicated └─────────┘ └─────────┘ └─────────┘ └─────────┘ + manual approval ``` 1. **Local to Devnet**: Push to `main` branch or open a Pull Request. CI builds and tests run automatically, and artifacts are synced to the devnet cluster. 2. **Devnet to Staging**: Merge to a `release/*` branch. Automated smoke tests run after deployment. 3. **Staging to Mainnet**: Create a semantic version Git tag (`v1.2.3`). A CODEOWNER must approve before ArgoCD syncs to the mainnet cluster. *** ## GitHub Actions Workflow Structure The CI/CD pipeline is implemented as a collection of GitHub Actions workflows: ``` .github/workflows/ ├── ci.yaml # Lint, test, security scan (all branches) ├── build.yaml # Docker build & push (main branch) ├── contracts.yaml # Compile, test, audit contracts ├── deploy-devnet.yaml # Auto-deploy to devnet ├── deploy-staging.yaml # Deploy to staging (release branches) └── deploy-mainnet.yaml # Deploy to mainnet (tags, with approval) ``` ### CI Pipeline (ci.yaml) Runs on every push and pull request: ```yaml # Simplified ci.yaml structure on: push: branches: [main, 'release/*'] pull_request: branches: [main] jobs: lint-and-test: steps: - checkout - setup-node (Node.js 20 LTS) - pnpm install (with cache) - pnpm lint - pnpm typecheck - pnpm test security-scan: steps: - slither analysis - trivy container scan ``` ### Contract Pipeline (contracts.yaml) Compiles, tests, and audits smart contracts: ```yaml contract-audit: steps: - hardhat compile - hardhat test --parallel - forge test --fuzz-runs 10000 - slither . --sarif output.sarif - mythril analyze contracts/*.sol ``` ### Build Pipeline (build.yaml) Builds and pushes Docker images on merges to `main`: ```yaml on: push: branches: [main] jobs: build-and-push: steps: - docker/build-push-action (multi-platform) - cosign sign (image signing) - syft generate SBOM ``` *** ## ArgoCD GitOps Deployment Lithosphere uses ArgoCD for GitOps-based deployment to Kubernetes clusters. The Application-of-Applications pattern manages all environments: ```yaml # Application-of-Applications pattern apiVersion: argoproj.io/v1alpha1 kind: ApplicationSet metadata: name: lithosphere-environments spec: generators: - list: elements: - env: devnet autoSync: true - env: staging autoSync: true - env: mainnet autoSync: false # Manual approval required ``` **ArgoCD Sync Policies by Environment:** | Environment | Auto-Sync | Self-Heal | Prune | | ----------- | --------- | --------- | ----- | | Devnet | Yes | Yes | Yes | | Staging | Yes | Yes | Yes | | Mainnet | **No** | No | No | ### Kubernetes Configuration Structure ``` clusters/ ├── devnet/ │ └── argocd/application.yaml ├── staging/ │ └── argocd/application.yaml └── mainnet/ └── argocd/application.yaml ``` Environment-specific configuration is managed through Kustomize overlays to minimize drift between environments. *** ## Environment Comparison Matrix | Aspect | Local | Devnet | Staging | Mainnet | | --------------- | ------------- | -------------- | -------------- | --------------------- | | **Trigger** | Manual | Push/PR | Release branch | Git tag `v*` | | **Approval** | None | None | None | CODEOWNER | | **Persistence** | Session | 24h max | Permanent | Permanent | | **Data** | Mock/Fork | Fixtures | Testnet | Production | | **HA** | No | No | Yes | Yes (Multi-AZ) | | **Secrets** | `.env` file | GitHub Secrets | Vault | Vault | | **Monitoring** | Local Grafana | Shared Grafana | Dedicated | Dedicated + PagerDuty | | **Cost** | $0 | Low | Medium | High | *** ## Further Reading * [Hardhat Example](/docs/developers/examples/hardhat-example.md) -- Smart contract development and deployment * [Foundry Example](/docs/developers/examples/foundry-example.md) -- Fuzz testing with Foundry * [Troubleshooting](/docs/developers/troubleshooting.md) -- Common deployment issues and solutions --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://whitepaper.litho.ai/docs/developers/ci-cd-guide.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.