Merge pull request #122 from sirius0xdev/refactor/repo-reorganization-plan
docs: repository reorganization migration plan
This commit is contained in:
sirius0xdev
GitHub
commit
b957b3c704
1 changed files with 203 additions and 0 deletions
203
MIGRATION_PLAN.md
Normal file
203
MIGRATION_PLAN.md
Normal file
|
|
@ -0,0 +1,203 @@
|
|||
# Repository Reorganization Plan
|
||||
|
||||
## Goal
|
||||
|
||||
Enforce the rule: **gcloud-lab = Kubernetes manifests only. No application code.**
|
||||
|
||||
Application code (source, Dockerfiles, CI/CD workflows, Helm charts, deployment scripts) belongs in `hermes-projects/`. gcloud-lab should contain only K8s manifests, Terraform infrastructure modules, cluster configs, and infra controller configs.
|
||||
|
||||
---
|
||||
|
||||
## Current State
|
||||
|
||||
### gcloud-lab/ — Full Directory Audit
|
||||
|
||||
```
|
||||
gcloud-lab/
|
||||
├── apps/
|
||||
│ ├── base/
|
||||
│ │ ├── customer1/ [KEEP] K8s manifests (deployments, services, configmaps, secrets)
|
||||
│ │ ├── monitoring/ [KEEP] K8s manifests (dashboards)
|
||||
│ │ └── osint-dashboard/ [KEEP] Helm chart (templates, values.yaml, Chart.yaml)
|
||||
│ ├── staging/
|
||||
│ │ ├── customer1/ [KEEP] K8s overlay (kustomization.yaml)
|
||||
│ │ └── osint-dashboard/ [KEEP] K8s overlay (kustomization.yaml)
|
||||
│ └── vwap-monitor/ [MOVE] App code (Dockerfile, app/, deploy/)
|
||||
├── clusters/ [KEEP] Cluster configs (devops-lab/*.yaml, flux-system/)
|
||||
├── infrastructure/ [KEEP] Infra controllers, gatewayapi, gpus, tailnet
|
||||
├── misc/ [KEEP] Terraform snippets + K8s YAML
|
||||
├── modules/ [KEEP] Terraform modules (gke.tf, nodepool.tf, etc.)
|
||||
├── scripts/ [KEEP] Setup scripts
|
||||
├── .github/workflows/
|
||||
│ ├── osint-dashboard-infra.yml [KEEP] Infra deployment workflow
|
||||
│ └── trade-dashboard.yml [MOVE] CI for trade-dashboard app → hermes-projects/
|
||||
├── .devcontainer.json [KEEP] Dev environment config
|
||||
├── .sops.yaml [KEEP] SOPS encryption config
|
||||
├── .terraform.lock.hcl [KEEP] Terraform lock
|
||||
├── .gitignore [KEEP] Git ignore rules
|
||||
├── mise.toml [KEEP] Tool version management
|
||||
├── README.md [KEEP] (will be updated)
|
||||
├── infra-tailnet.yaml [KEEP] Tailscale infra config
|
||||
├── tailscale-0auth.yaml [KEEP] Tailscale config
|
||||
├── rays-new-deployment.yaml [REVIEW] Orphan K8s deployment YAML — move to apps/base/
|
||||
├── trade-dashboard/ [MOVE] Full FastAPI app → hermes-projects/trade-dashboard/
|
||||
├── trading-platform/ [MOVE] Duplicate/deploy configs → hermes-projects/trading-platform/
|
||||
├── trading-scripts/ [MOVE] Application code → hermes-projects/trading-scripts/
|
||||
├── analyses/ [REMOVE] Research artifacts (not code, not infra)
|
||||
└── plans/ [REMOVE] Planning docs (not code, not infra)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Items to Move / Remove
|
||||
|
||||
### 1. `trade-dashboard/` → hermes-projects/trade-dashboard/
|
||||
|
||||
**What it is:** Full FastAPI application (not K8s manifests)
|
||||
- `Dockerfile` — build config for the app
|
||||
- `app/` — Python source code (main.py, models.py, schemas.py, database.py, requirements.txt, static/)
|
||||
- `alembic/` — database migration scripts (env.py, versions/)
|
||||
- `alembic.ini` — alembic config
|
||||
|
||||
**Also move:** `.github/workflows/trade-dashboard.yml` (CI workflow for this app)
|
||||
|
||||
**Already in gcloud-lab:** `apps/base/customer1/trade-dashboard/` — these are the K8s manifests for trade-dashboard (deployment.yaml, service.yaml, configmap.yaml, kustomization.yaml). **KEEP these** — they belong here.
|
||||
|
||||
### 2. `trading-platform/` → hermes-projects/trading-platform/
|
||||
|
||||
**What it is:** Duplicate/alternative deployment configs that overlap with hermes-projects/trading-platform/
|
||||
|
||||
Contents:
|
||||
- `.github/workflows/` — 3 CI/CD workflows (build-push.yml, build-test.yml, deploy.yml)
|
||||
- `README.md` — project readme
|
||||
- `deploy/` — deployment configs:
|
||||
- `ci-cd/` — additional CI workflows
|
||||
- `docker-compose/` — docker-compose.dev.yml
|
||||
- `dockerfiles/` — Dockerfiles (api-gateway, dashboard, data-service, execute-service, news-service)
|
||||
- `helm/` — Helm charts (api-gateway, dashboard, data-service, execute-service, news-service)
|
||||
- `k8s/` — raw K8s manifests (deployments, services, hpa, cert-manager, ingress)
|
||||
- `mtls/` — mTLS README
|
||||
- `scripts/` — deploy.sh, generate-mtls-certs.sh
|
||||
- `dockerfiles/` — Dockerfiles (dashboard, data-service, execute-service, news-service)
|
||||
- `helm/` — Helm chart with templates (trading-platform chart, values.yaml, secrets)
|
||||
|
||||
**Already in gcloud-lab:** `apps/base/customer1/trading-platform/` — these are the K8s manifests. **KEEP these** — they belong here.
|
||||
|
||||
**Already in hermes-projects:** `hermes-projects/trading-platform/` — source code exists here (dashboard, data-service, execute-service, news-service, data_infrastructure). The gcloud-lab trading-platform/ deploy/dockerfiles/helm content should be MERGED into hermes-projects/trading-platform/.
|
||||
|
||||
**Decision needed:** The `trading-platform/` in gcloud-lab has BOTH deploy configs (dockerfiles, helm, k8s manifests) AND CI workflows. The K8s manifests in `deploy/k8s/base/` are similar but NOT identical to what's in `apps/base/customer1/trading-platform/`. Need to decide which is authoritative.
|
||||
|
||||
### 3. `trading-scripts/` → hermes-projects/trading-scripts/
|
||||
|
||||
**What it is:** Application code (Python trading scripts)
|
||||
- `market_data.py` — market data script
|
||||
- `orb-monitor/` — monitoring tool (monitor.py, config.yaml)
|
||||
- `README.md`, `ROADMAP.md` — documentation
|
||||
|
||||
### 4. `apps/vwap-monitor/` → hermes-projects/vwap-monitor/
|
||||
|
||||
**What it is:** Application code with a Dockerfile
|
||||
- `Dockerfile` — build config
|
||||
- `app/` — source code (scanner.py, requirements.txt)
|
||||
- `deploy/` — deployment config (deployment.yaml, kustomization.yaml, secret.yaml, config.env)
|
||||
|
||||
**Note:** The `deploy/` subdirectory contains K8s manifests. These should be moved BACK into gcloud-lab as `apps/base/customer1/vwap-monitor/`. The app code (Dockerfile + app/) goes to hermes-projects.
|
||||
|
||||
### 5. `analyses/` → REMOVE from gcloud-lab
|
||||
|
||||
**What it is:** Research/analysis markdown documents
|
||||
- `telegram-webhook-container-analysis.md`
|
||||
- `telegram-webhook-failure-analysis.md`
|
||||
|
||||
These are one-time research artifacts, not infrastructure config. Remove from gcloud-lab entirely.
|
||||
|
||||
### 6. `plans/` → REMOVE from gcloud-lab
|
||||
|
||||
**What it is:** Planning/strategy markdown documents
|
||||
- `2026-04-25-openclaw-brain-v1.1.md`
|
||||
- `AI_ARCHITECTURE.md`
|
||||
- `models-to-try.md`
|
||||
|
||||
These are planning docs, not infrastructure config. Remove from gcloud-lab entirely.
|
||||
|
||||
### 7. `rays-new-deployment.yaml` → REVIEW
|
||||
|
||||
**What it is:** A standalone K8s deployment YAML at repo root.
|
||||
|
||||
**Action:** Move to `apps/base/customer1/hermes-agent/` (appears related to hermes-agent/rays deployment based on filename). Already similar files exist in that directory.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Migration Plan (Ordered by PR)
|
||||
|
||||
### PR 1: This Plan (docs only)
|
||||
- Add `MIGRATION_PLAN.md` (this file)
|
||||
- Update `README.md` to document the new structure
|
||||
|
||||
### PR 2: Remove planning/research docs
|
||||
- Delete `analyses/` directory
|
||||
- Delete `plans/` directory
|
||||
- Low risk, no dependencies
|
||||
|
||||
### PR 3: Move trade-dashboard app to hermes-projects
|
||||
- Move `trade-dashboard/` → hermes-projects/trade-dashboard/
|
||||
- Move `.github/workflows/trade-dashboard.yml` → hermes-projects/.github/workflows/
|
||||
- K8s manifests in `apps/base/customer1/trade-dashboard/` stay in place
|
||||
- Verify image references in K8s manifests still point to correct registry
|
||||
|
||||
### PR 4: Move trading-platform deploy configs to hermes-projects
|
||||
- Move `trading-platform/` → merge with hermes-projects/trading-platform/
|
||||
- CI workflows → hermes-projects/trading-platform/.github/workflows/
|
||||
- Dockerfiles → hermes-projects/trading-platform/dockerfiles/
|
||||
- Helm charts → hermes-projects/trading-platform/helm/
|
||||
- K8s manifests from `trading-platform/deploy/k8s/` → reconcile with `apps/base/customer1/trading-platform/`
|
||||
- **Decision needed:** Which K8s manifests are authoritative? The ones in gcloud-lab/apps/ or trading-platform/deploy/k8s/?
|
||||
|
||||
### PR 5: Move trading-scripts to hermes-projects
|
||||
- Move `trading-scripts/` → hermes-projects/trading-scripts/
|
||||
- Simple move, no K8s manifest reconciliation needed
|
||||
|
||||
### PR 6: Split vwap-monitor (app → hermes-projects, K8s → gcloud-lab)
|
||||
- Move `apps/vwap-monitor/app/` + `apps/vwap-monitor/Dockerfile` → hermes-projects/vwap-monitor/
|
||||
- Move `apps/vwap-monitor/deploy/` K8s manifests → `apps/base/customer1/vwap-monitor/`
|
||||
- Update image references in K8s manifests
|
||||
|
||||
---
|
||||
|
||||
## Items That Stay in gcloud-lab (No Changes)
|
||||
|
||||
| Path | Reason |
|
||||
|------|--------|
|
||||
| `apps/base/customer1/` | K8s manifests (kustomize structure) |
|
||||
| `apps/base/monitoring/` | K8s manifests (dashboards) |
|
||||
| `apps/base/osint-dashboard/` | Helm chart for infra |
|
||||
| `apps/staging/` | K8s overlays |
|
||||
| `clusters/` | Cluster configs, flux-system |
|
||||
| `infrastructure/` | Controllers, gatewayapi, gpus, tailnet |
|
||||
| `misc/` | Terraform snippets + K8s YAML |
|
||||
| `modules/` | Terraform modules |
|
||||
| `scripts/` | Setup scripts |
|
||||
| Root config files | .sops.yaml, .devcontainer.json, mise.toml, .gitignore, .terraform.lock.hcl |
|
||||
| `infra-tailnet.yaml` | Tailscale infra config |
|
||||
| `tailscale-0auth.yaml` | Tailscale config |
|
||||
|
||||
---
|
||||
|
||||
## K8s Manifest Reference Check
|
||||
|
||||
After moves, verify these image references still resolve:
|
||||
|
||||
| K8s Manifest | Image Reference |
|
||||
|--------------|----------------|
|
||||
| `apps/base/customer1/trade-dashboard/deployment.yaml` | Check image tag matches hermes-projects build |
|
||||
| `apps/base/customer1/trading-platform/*/deployment.yaml` | Check image tags match hermes-projects build |
|
||||
| `apps/base/customer1/hermes-agent/deployment.yaml` | N/A (already correct) |
|
||||
| `apps/base/customer1/siriusdevops-site/deployment.yaml` | N/A (already correct) |
|
||||
|
||||
---
|
||||
|
||||
## Decisions Needed Before Proceeding
|
||||
|
||||
1. **trading-platform K8s manifest authority:** `trading-platform/deploy/k8s/base/` vs `apps/base/customer1/trading-platform/` — which is the source of truth?
|
||||
2. **analyses/ and plans/:** Delete entirely, or archive somewhere else?
|
||||
3. **rays-new-deployment.yaml:** Move to `apps/base/customer1/hermes-agent/` or delete?
|
||||
Loading…
Add table
Reference in a new issue