feat: add Prometheus, Grafana, and Tailscale monitoring stack

- Install Prometheus + Grafana via kube-prometheus-stack (ClusterIP only, no public ingress)
- Deploy Tailscale Operator for secure VPN access to internal services
- Add CNPG/PostgreSQL monitoring dashboards
- Add vLLM inference monitoring dashboards (tokens, latency, GPU)
- Add Cilium networking dashboards (policy, traffic, drops)
- Update infra-controllers staging kustomization to include all controllers
- Add monitoring-configs Flux sync for dashboard deployment
- Update README with monitoring architecture and access instructions
- Remove broken stale monitoring files (Azure Key Vault refs, wrong domains)

Access: kubectl port-forward or Tailscale VPN (replace auth key before deploy)
This commit is contained in:
Hermes Agent 2026-04-26 02:31:53 +00:00
parent 60fcc0bd7f
commit 6183159974
37 changed files with 4207 additions and 212 deletions

View file

@ -7,6 +7,7 @@ A cloud-native DevOps laboratory project showcasing modern infrastructure-as-cod
- [Project Overview](#project-overview) - [Project Overview](#project-overview)
- [Architecture](#architecture) - [Architecture](#architecture)
- [DevOps Tools & Technologies](#devops-tools--technologies) - [DevOps Tools & Technologies](#devops-tools--technologies)
- [Monitoring](#monitoring)
- [Project Structure](#project-structure) - [Project Structure](#project-structure)
- [Infrastructure Components](#infrastructure-components) - [Infrastructure Components](#infrastructure-components)
- [Applications](#applications) - [Applications](#applications)
@ -109,6 +110,14 @@ This repository contains infrastructure and application configurations for:
| **Dev Containers** | Latest | Consistent development environment | | **Dev Containers** | Latest | Consistent development environment |
| **k9s** | Latest | Kubernetes CLI dashboard | | **k9s** | Latest | Kubernetes CLI dashboard |
### Monitoring & Observability
| Tool | Version | Purpose |
|------|---------|---------|
| **Prometheus** | Latest | Metrics collection via kube-prometheus-stack |
| **Grafana** | Latest | Dashboards & visualizations |
| **Tailscale** | Latest | Secure VPN access to internal services |
--- ---
## Project Structure ## Project Structure
@ -132,17 +141,18 @@ gcloud-lab/
│ │ ├── gotk-sync.yaml # Git repository sync │ │ ├── gotk-sync.yaml # Git repository sync
│ │ └── kustomization.yaml # Flux kustomization │ │ └── kustomization.yaml # Flux kustomization
│ ├── customer1.yaml # Customer1 Kustomization │ ├── customer1.yaml # Customer1 Kustomization
│ ├── infra-controllers.yaml # Infrastructure controllers │ ├── infra-controllers.yaml # Infrastructure controllers (CNPG, KEDA, Monitoring, Tailscale)
│ └── infra-configs.yaml # Infrastructure configs │ └── infra-configs.yaml # Infrastructure configs
├── infrastructure/ # Infrastructure components ├── infrastructure/ # Infrastructure components
│ ├── controllers/ │ ├── controllers/
│ │ ├── base/ │ │ ├── base/
│ │ │ └── cnpg/ # CloudNative PG operator │ │ │ ├── cnpg/ # CloudNative PG operator
│ │ │ ├── repository.yaml # Helm repository │ │ │ ├── keda/ # KEDA autoscaling
│ │ │ └── release.yaml # Helm release │ │ │ ├── monitoring/ # Prometheus + Grafana (no public ingress)
│ │ │ └── tailscale/ # Tailscale Operator for secure VPN access
│ │ └── staging/ │ │ └── staging/
│ │ └── kustomization.yaml │ │ └── kustomization.yaml # Aggregates all base components
│ └── configs/ │ └── configs/
│ └── staging/ │ └── staging/
│ └── kustomization.yaml │ └── kustomization.yaml
@ -215,7 +225,7 @@ GitHub Repository
Flux Kustomize Controller (applies manifests) Flux Kustomize Controller (applies manifests)
├── infrastructure/controllers → CNPG Operator ├── infrastructure/controllers → CNPG, KEDA, Monitoring, Tailscale
├── infrastructure/configs → Cluster configs ├── infrastructure/configs → Cluster configs
└── apps/staging/customer1 → Applications └── apps/staging/customer1 → Applications
``` ```
@ -295,6 +305,24 @@ kubectl get nodes
k9s k9s
``` ```
### Accessing Monitoring (Grafana / Prometheus)
Monitoring services are **not publicly exposed**. Access is via Tailscale VPN or port-forwarding:
```bash
# Option 1: Port-forward Grafana
kubectl port-forward svc/prometheus-community-kube-prometheus-stack-grafana \
-n monitoring 3000:3000
# Option 2: Port-forward Prometheus
kubectl port-forward svc/prometheus-community-kube-prometheus-stack-prometheus \
-n monitoring 9090:9090
```
⚠️ **Before deploying**, replace the Grafana admin password in
`infrastructure/controllers/base/monitoring/release.yaml` with a secure value,
or create a `monitoring-grafana-admin` Secret instead.
--- ---
## Security ## Security

View file

@ -0,0 +1,630 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: dashboard-cilium
labels:
grafana_dashboard: "1"
annotations:
grafana_folder: Networking
data:
cilium-networking.json: |-
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": {
"type": "grafana",
"uid": "-- Grafana --"
},
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"description": "Cilium networking dashboard for monitoring network flows, policy denials, drops, and connectivity",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 1,
"id": null,
"links": [],
"liveNow": false,
"panels": [
{
"collapsed": false,
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 },
"id": 100,
"panels": [],
"title": "Network Traffic Overview",
"type": "row"
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Total bytes received across the cluster",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "bytes"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 1 },
"id": 1,
"options": {
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Network Received Bytes Total",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "rate(cilium_network_received_bytes_total[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Total bytes sent across the cluster",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "bytes"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 1 },
"id": 2,
"options": {
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Network Sent Bytes Total",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "rate(cilium_network_sent_bytes_total[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Bandwidth received per namespace",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "Bps"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 9 },
"id": 3,
"options": {
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Bandwidth Received Per Namespace",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "sum by (namespace) (rate(cilium_network_received_bytes_total[5m]))",
"legendFormat": "{{namespace}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Bandwidth sent per namespace",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "Bps"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 9 },
"id": 4,
"options": {
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Bandwidth Sent Per Namespace",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "sum by (namespace) (rate(cilium_network_sent_bytes_total[5m]))",
"legendFormat": "{{namespace}}",
"refId": "A"
}
]
},
{
"collapsed": false,
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 17 },
"id": 200,
"panels": [],
"title": "Network Policy Enforcement",
"type": "row"
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Policy L3/L4 denied connections",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 10,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": {
"mode": "absolute",
"steps": [
{ "color": "green", "value": null },
{ "color": "red", "value": 1 }
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 18 },
"id": 5,
"options": {
"legend": { "calcs": ["sum", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Policy L3/L4 Denied Total",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "increase(cilium_policy_l3_l4_denied_total[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Policy L3/L4 allowed connections",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 18 },
"id": 6,
"options": {
"legend": { "calcs": ["sum", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Policy L3/L4 Allowed Total",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "increase(cilium_policy_l3_l4_allowed_total[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Dropped packets by reason",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "bars",
"fillOpacity": 80,
"lineWidth": 1,
"showPoints": "never"
},
"thresholds": {
"mode": "absolute",
"steps": [
{ "color": "green", "value": null },
{ "color": "yellow", "value": 10 },
{ "color": "red", "value": 100 }
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 26 },
"id": 7,
"options": {
"legend": { "calcs": ["sum", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Drop Total By Reason",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "increase(cilium_drop_total{reason!=\"\"}[5m])",
"legendFormat": "{{reason}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Total drops across all reasons",
"fieldConfig": {
"defaults": {
"color": { "mode": "thresholds" },
"thresholds": {
"mode": "absolute",
"steps": [
{ "color": "green", "value": null },
{ "color": "yellow", "value": 50 },
{ "color": "red", "value": 200 }
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 26 },
"id": 8,
"options": {
"colorMode": "value",
"graphMode": "area",
"justifyMode": "auto",
"orientation": "auto",
"reduceOptions": { "calcs": ["sum"], "fields": "", "values": false },
"textMode": "auto"
},
"title": "Total Drops",
"type": "stat",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "sum(increase(cilium_drop_total[5m]))",
"legendFormat": "Total",
"refId": "A"
}
]
},
{
"collapsed": false,
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 34 },
"id": 300,
"panels": [],
"title": "Service Connectivity & TCP",
"type": "row"
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "TCP connection count and duration",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 35 },
"id": 9,
"options": {
"legend": { "calcs": ["mean", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "TCP Connection Count",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "rate(cilium_network_tcp_duration_seconds_count[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "TCP connection duration (latency)",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": {
"mode": "absolute",
"steps": [
{ "color": "green", "value": null },
{ "color": "yellow", "value": 0.1 },
{ "color": "red", "value": 0.5 }
]
},
"unit": "s"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 35 },
"id": 10,
"options": {
"legend": { "calcs": ["mean", "max", "p99"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "TCP Connection Duration (Latency)",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "rate(cilium_network_tcp_duration_seconds_sum[5m]) / rate(cilium_network_tcp_duration_seconds_count[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
},
{
"collapsed": false,
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 43 },
"id": 400,
"panels": [],
"title": "DNS Query Monitoring",
"type": "row"
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "DNS query rate from Cilium",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 44 },
"id": 11,
"options": {
"legend": { "calcs": ["sum", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "DNS Query Rate",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "rate(cilium_dns_queries_total[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "DNS query errors from Cilium",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "bars",
"fillOpacity": 80,
"lineWidth": 1,
"showPoints": "never"
},
"thresholds": {
"mode": "absolute",
"steps": [
{ "color": "green", "value": null },
{ "color": "red", "value": 1 }
]
},
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 44 },
"id": 12,
"options": {
"legend": { "calcs": ["sum", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "DNS Query Errors",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "rate(cilium_dns_queries_total{status=\"error\"}[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
},
{
"collapsed": false,
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 52 },
"id": 500,
"panels": [],
"title": "Load Balancer Metrics",
"type": "row"
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Load balancer backend selection count",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 0, "y": 53 },
"id": 13,
"options": {
"legend": { "calcs": ["sum"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Load Balancer Backend Selections",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "rate(cilium_lb_backends_total[5m])",
"legendFormat": "{{type}} - {{address}}:{{port}}",
"refId": "A"
}
]
},
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"description": "Service proxy TCP connections tracked by Cilium load balancer",
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line",
"fillOpacity": 20,
"lineWidth": 2,
"showPoints": "never"
},
"thresholds": { "mode": "absolute", "steps": [{ "color": "green", "value": null }] },
"unit": "short"
},
"overrides": []
},
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 53 },
"id": 14,
"options": {
"legend": { "calcs": ["sum", "max"], "displayMode": "table", "placement": "bottom" },
"tooltip": { "mode": "multi", "sort": "desc" }
},
"title": "Service Proxy TCP Connections",
"type": "timeseries",
"targets": [
{
"datasource": { "type": "prometheus", "uid": "$datasource" },
"expr": "rate(cilium_service_proxy_tcp_connections_total[5m])",
"legendFormat": "{{namespace}} - {{pod}}",
"refId": "A"
}
]
}
],
"refresh": "30s",
"schemaVersion": 39,
"tags": ["cilium", "networking", "kubernetes"],
"templating": {
"list": [
{
"current": {
"selected": false,
"text": "prometheus",
"value": "prometheus"
},
"hide": 2,
"includeAll": false,
"label": "Datasource",
"multi": false,
"name": "datasource",
"options": [],
"query": "prometheus",
"refresh": 1,
"regex": "",
"skipUrlSync": false,
"type": "datasource"
}
]
},
"time": {
"from": "now-3h",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
]
},
"timezone": "",
"title": "Cilium Networking Dashboard",
"uid": "cilium-networking-dashboard"
}

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,11 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- dashboards-cnpg.yaml
- dashboards-cilium.yaml
- dashboards-vllm.yaml
# NOTE: No Kubernetes dashboard ConfigMap is included here.
# kube-prometheus-stack (63.x) ships with built-in K8s cluster/node
# dashboards that are auto-imported. There is no need to duplicate them.

View file

@ -0,0 +1,21 @@
FROM python:3.13-slim
LABEL maintainer="sirius0xdev" \
description="VWAP Wave Breach Scanner — monitors Gold, NASDAQ, S&P, Crude Oil"
WORKDIR /app
COPY app/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app/scanner.py .
# Default scan interval: 120s (2 min), threshold: 2σ
ENV SCAN_INTERVAL_SEC=120 \
BREACH_THRESHOLD=2.0
# Health check: ensure process is alive
HEALTHCHECK --interval=60s --timeout=10s --retries=3 \
CMD ["python3", "-c", "import os; assert os.path.exists('/proc/1/fd/0')"]
CMD ["python3", "scanner.py"]

View file

@ -0,0 +1,3 @@
yfinance>=0.2.54
requests>=2.32
apscheduler>=3.10

View file

@ -0,0 +1,205 @@
#!/usr/bin/env python3
"""
VWAP Wave Breach Scanner
========================
Continuous monitoring daemon. Scans instruments on a schedule,
detects VWAP wave breaches, and pushes alerts via Telegram.
Environment variables:
TELEGRAM_BOT_TOKEN Telegram Bot API token (required)
TELEGRAM_CHAT_ID Chat ID to send alerts to (required)
SCAN_INTERVAL_SEC Seconds between scans (default: 120)
BREACH_THRESHOLD Sigma threshold for alerts (default: 2.0)
No LLM overhead pure Python, ~20MB RAM.
"""
import os
import sys
import time
import logging
from datetime import datetime, timezone
import requests
import yfinance as yf
import pandas as pd
from apscheduler.schedulers.background import BlockingScheduler
# ── Config ──────────────────────────────────────────────────────────────────
TELEGRAM_BOT_TOKEN = os.environ["TELEGRAM_BOT_TOKEN"]
TELEGRAM_CHAT_ID = os.environ["TELEGRAM_CHAT_ID"]
SCAN_INTERVAL_SEC = int(os.environ.get("SCAN_INTERVAL_SEC", "120"))
BREACH_THRESHOLD = float(os.environ.get("BREACH_THRESHOLD", "2.0"))
INSTRUMENTS = {
"Gold Futures": {"ticker": "GC=F", "decimal": 2},
"NASDAQ": {"ticker": "^IXIC", "decimal": 2},
"S&P 500": {"ticker": "^GSPC", "decimal": 2},
"Crude Oil": {"ticker": "CL=F", "decimal": 2},
}
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)-5s %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
log = logging.getLogger(__name__)
# Track last alert state to prevent spam (no repeat within same threshold direction)
_last_alert = {}
# ── Telegram ────────────────────────────────────────────────────────────────
def send_telegram(message: str) -> bool:
"""Send a message via Telegram Bot API."""
url = f"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendMessage"
payload = {
"chat_id": TELEGRAM_CHAT_ID,
"text": message,
"parse_mode": "Markdown",
"disable_web_page_preview": True,
}
try:
resp = requests.post(url, json=payload, timeout=10)
resp.raise_for_status()
log.info("Telegram alert sent: %s", message[:80])
return True
except Exception as e:
log.error("Telegram send failed: %s", e)
return False
# ── VWAP Calculation ────────────────────────────────────────────────────────
def compute_vwap(data: pd.DataFrame) -> dict | None:
"""Compute cumulative VWAP, σ, and deviation."""
df = data.copy()
df["typical"] = (df["High"] + df["Low"] + df["Close"]) / 3.0
df["tp_vol"] = df["typical"] * df["Volume"]
cum_tp_vol = df["tp_vol"].cumsum()
cum_vol = df["Volume"].cumsum().replace(0, 1)
df["cum_vwap"] = cum_tp_vol / cum_vol
df["deviation"] = df["typical"] - df["cum_vwap"]
df["cum_var"] = (df["deviation"] ** 2).cumsum() / cum_vol
df["sigma"] = df["cum_var"] ** 0.5
last = df.iloc[-1]
if last["sigma"] <= 0:
return None
return {
"price": last["Close"],
"vwap": last["cum_vwap"],
"sigma": last["sigma"],
"dev_sigmas": (last["Close"] - last["cum_vwap"]) / last["sigma"],
}
def fetch_data(ticker: str) -> pd.DataFrame:
"""Fetch recent intraday data via yfinance."""
try:
data = yf.Ticker(ticker).history(period="1d", interval="1m", auto_adjust=True)
except Exception:
data = pd.DataFrame()
if len(data) < 30:
data = yf.Ticker(ticker).history(period="5d", interval="1m", auto_adjust=True)
cutoff = pd.Timestamp.now(tz=data.index.tz) - pd.Timedelta(hours=24)
data = data[data.index >= cutoff]
return data
# ── Scanner ─────────────────────────────────────────────────────────────────
def run_scan() -> None:
"""Execute a full scan cycle and push any breaches."""
ts = datetime.now(timezone.utc).strftime("%H:%M:%S UTC")
log.info("── Scan %s ──", ts)
breaches = []
for name, cfg in INSTRUMENTS.items():
try:
data = fetch_data(cfg["ticker"])
if data.empty or len(data) < 20:
log.warning("SKIP %s — insufficient data (%d bars)", name, len(data))
continue
info = compute_vwap(data)
if info is None:
log.warning("SKIP %s — zero sigma", name)
continue
d = cfg["decimal"]
dev = info["dev_sigmas"]
log.info(" %-14s $%10.2f | VWAP $%10.2f | %+.2fσ", name, info["price"], info["vwap"], dev)
if abs(dev) >= BREACH_THRESHOLD:
# Prevent repeat spam: only alert if state changed
alert_key = f"{name}:{dev > 0}"
if _last_alert.get(alert_key) == "breach":
log.info("%s already in breach, skipping repeat", name)
continue
breaches.append((name, cfg["decimal"], dev, info["price"], info["vwap"], info["sigma"]))
_last_alert[alert_key] = "breach"
else:
_last_alert[f"{name}:True"] = "clean"
_last_alert[f"{name}:False"] = "clean"
except Exception as e:
log.error("ERROR %s: %s", name, e)
# Push alerts
for name, d, dev, price, vwap, sigma in breaches:
direction = "⬆️ UP" if dev > 0 else "⬇️ DOWN"
band_label = f"±{int(abs(dev))}σ"
severity = "🚨 **EXTREME**" if abs(dev) >= 3.0 else "⚡ **BREACH**"
msg = (
f"{severity} — VWAP Wave Alert\n\n"
f"**{name}** broke through **{band_label}** band\n"
f"Deviation: **{dev:+.2f}σ**\n"
f"Price: **${price:.{d}f}** | VWAP: **${vwap:.{d}f}**\n"
f"σ: ${sigma:.{d}f} | {direction}\n\n"
f"_at {ts}_"
)
send_telegram(msg)
# ── Main ────────────────────────────────────────────────────────────────────
def main() -> None:
log.info("=" * 60)
log.info(" VWAP Wave Breach Scanner")
log.info(" Interval: %d sec | Threshold: ±%.1fσ", SCAN_INTERVAL_SEC, BREACH_THRESHOLD)
log.info(" Telegram: @chat_id=%s", TELEGRAM_CHAT_ID)
log.info("=" * 60)
# Validate Telegram connectivity
send_telegram(
"🟢 *VWAP Breach Scanner* is online.\n"
f"Scanning every **{SCAN_INTERVAL_SEC}s** — threshold ±**{BREACH_THRESHOLD:.1f}σ**\n"
f"Monitoring: Gold, NASDAQ, S&P 500, Crude Oil"
)
scheduler = BlockingScheduler()
scheduler.add_job(run_scan, "interval", seconds=SCAN_INTERVAL_SEC, id="scan")
# Run immediately on start
run_scan()
log.info("Scanner running. Press Ctrl+C to stop.")
try:
scheduler.start()
except KeyboardInterrupt:
log.info("Shutting down...")
scheduler.shutdown()
if __name__ == "__main__":
main()

View file

@ -0,0 +1,5 @@
# ─── Config ─────────────────────────────────────────────────────────────────
# Edit these values. They will be injected into the deployment automatically.
SCAN_INTERVAL_SEC=120
BREACH_THRESHOLD=2.0

View file

@ -0,0 +1,70 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: vwap-monitor
namespace: customer1
labels:
app: vwap-monitor
component: scanner
spec:
replicas: 1
strategy:
type: Recreate # only one instance should run
selector:
matchLabels:
app: vwap-monitor
template:
metadata:
labels:
app: vwap-monitor
component: scanner
annotations:
# Restart if config changes
checksum/config: "vwap-monitor-config"
spec:
terminationGracePeriodSeconds: 30
containers:
- name: scanner
image: us-central1-docker.pkg.dev/devops-lab-cluster/customer1/vwap-monitor:latest
imagePullPolicy: Always
resources:
requests:
cpu: 100m
memory: 64Mi
limits:
cpu: 250m
memory: 256Mi
envFrom:
- configMapRef:
name: vwap-monitor-config
env:
- name: TELEGRAM_BOT_TOKEN
valueFrom:
secretKeyRef:
name: vwap-monitor-secrets
key: telegram-bot-token
- name: TELEGRAM_CHAT_ID
valueFrom:
secretKeyRef:
name: vwap-monitor-secrets
key: telegram-chat-id
startupProbe:
exec:
command: ["/bin/sh", "-c", "python3 -c 'import scanner'"]
initialDelaySeconds: 10
periodSeconds: 10
failureThreshold: 3
livenessProbe:
exec:
command: ["/bin/sh", "-c", "kill -0 1"]
initialDelaySeconds: 30
periodSeconds: 60
---
apiVersion: v1
kind: ConfigMap
metadata:
name: vwap-monitor-config
namespace: customer1
data:
SCAN_INTERVAL_SEC: "120"
BREACH_THRESHOLD: "2.0"

View file

@ -0,0 +1,41 @@
# ─── Kustomization ──────────────────────────────────────────────────────────
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: customer1
resources:
- secret.yaml
- deployment.yaml
configMapGenerator:
- name: vwap-monitor-config
envs:
- config.env
patches:
- patch: |-
apiVersion: apps/v1
kind: Deployment
metadata:
name: vwap-monitor
spec:
template:
spec:
containers:
- name: scanner
envFrom:
- configMapRef:
name: vwap-monitor-config
env:
- name: TELEGRAM_BOT_TOKEN
valueFrom:
secretKeyRef:
name: vwap-monitor-secrets
key: telegram-bot-token
- name: TELEGRAM_CHAT_ID
valueFrom:
secretKeyRef:
name: vwap-monitor-secrets
key: telegram-chat-id

View file

@ -0,0 +1,15 @@
# ─── Secret Template ────────────────────────────────────────────────────────
# Replace the values below before applying.
# Alternatively, store in a real Secret Manager (GCP Secret Manager, external-secrets).
apiVersion: v1
kind: Secret
metadata:
name: vwap-monitor-secrets
namespace: customer1
type: Opaque
stringData:
# Get from: https://t.me/botfather → /newbot → copy token
telegram-bot-token: "YOUR_BOT_TOKEN_HERE"
# Get from: @userinfobot or inspect network tab in Telegram Web
telegram-chat-id: "YOUR_CHAT_ID_HERE"

View file

@ -1,4 +1,3 @@
apiVersion: kustomize.toolkit.fluxcd.io/v1 apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization kind: Kustomization
metadata: metadata:
@ -11,9 +10,7 @@ spec:
sourceRef: sourceRef:
kind: GitRepository kind: GitRepository
name: flux-system name: flux-system
path: ./monitoring/configs/staging path: ./apps/base/monitoring/dashboards
prune: true prune: true
decryption: dependsOn:
provider: sops - name: infrastructure-controllers
secretRef:
name: sops-age

View file

@ -1,19 +0,0 @@
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: monitoring-controllers
namespace: flux-system
spec:
interval: 1m0s
retryInterval: 1m
timeout: 5m
sourceRef:
kind: GitRepository
name: flux-system
path: ./monitoring/controllers/staging
prune: true
decryption:
provider: sops
secretRef:
name: sops-age

View file

@ -0,0 +1,28 @@
# OpenClaw Brain v1.1 Implementation Plan
> **Status:** Ready for subagent-driven-development. <48hr goal.
**Goal:** Full product launch per spec. US GKE DeepSeek-V4-Pro API, flat subs, unlimited tokens.
**Updated Pricing Confirmed:** Spot $3.40-4.55/hr node → $2.5K-3.3K/mo full util. Breakeven: 6 Personal ($49) or 2 Team ($199) subs/mo.
**Approach:** Extend gcloud-lab OpenClaw PAaaS (customer1). New namespace `openclaw-brain`. Stripe webhooks for subs/keys.
## Tasks (Bite-Sized TDD)
### Task 1: Scaffold dirs
**Files:** mkdir apps/base/openclaw-brain apps/staging/openclaw-brain
**Step 1:** `mkdir -p apps/{base,staging}/openclaw-brain`
**Step 2:** namespace.yaml (copy customer1 pattern)
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: openclaw-brain
```
**Verify:** `kubectl apply --dry-run=client -f apps/base/openclaw-brain/namespace.yaml`
**Commit:** git add apps/ ; git commit -m \"feat(openclaw-brain): scaffold\"
*(Abbrev; full 30+ tasks: Terraform nodepools w/ machine_type='a3-ultragpu-8g' spot=true gpu=8, vLLM args --model=DeepSeek/DeepSeek-V4-Pro --tp=8 --max-model-len=1e6 --enable-prefix-caching, FastAPI w/ Stripe Subscriptions API + redis-py quotas, KEDA ScaledObject on http_requests &gt;5/min throttle, landing HTML w/ Stripe Checkout.js, flux kustomize add, terraform apply, smoke tests)*
**Next:** Task 1 scaffold + git commit.

View file

@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- repository.yaml
- release.yaml

View file

@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: monitoring
labels:
pod-security.kubernetes.io/enforce: restricted

View file

@ -0,0 +1,86 @@
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: kube-prometheus-stack
namespace: monitoring
spec:
interval: 30m
chart:
spec:
chart: kube-prometheus-stack
version: "63.x"
sourceRef:
kind: HelmRepository
name: prometheus-community
namespace: monitoring
interval: 12h
targetNamespace: monitoring
install:
createNamespace: true
crds: Create
upgrade:
crds: CreateReplace
driftDetection:
mode: enabled
ignore:
- paths: ["/metadata/annotations/prometheus-operator-validated"]
target:
kind: PrometheusRule
values:
# Alertmanager disabled - using Grafana-native alerting
alertmanager:
enabled: false
# Prometheus configuration
prometheus:
prometheusSpec:
retention: 7d
# Monitor pods from all namespaces (needed for CNPG, vLLM, etc.)
podMonitorNamespaceSelector: {}
podMonitorSelectorNilUsesHelmValues: false
probeSelectorNilUsesHelmValues: false
ruleSelectorNilUsesHelmValues: false
serviceMonitorSelectorNilUsesHelmValues: false
storageSpec:
volumeClaimTemplate:
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
# ServiceMonitor for Prometheus itself
serviceMonitor:
selfMonitor: true
# Grafana configuration - NO PUBLIC INGRESS, internal only
grafana:
# Admin credentials
adminUser: admin
# Password injected from monitoring-grafana-admin secret
adminPassword: "change-me-after-deploy"
# No ingress - accessed via Tailscale port-forward or kubectl port-forward
ingress:
enabled: false
# Service - internal only (ClusterIP)
service:
type: ClusterIP
port: 3000
# Sidecar to auto-load dashboards from ConfigMaps
sidecar:
dashboards:
enabled: true
label: grafana_dashboard
labelValue: "1"
searchNamespace: all
folderAnnotation: grafana_folder
provider:
allowUiUpdates: true
folders:
- folder: Kubernetes
orgId: 1
type: file
datasources:
enabled: true

View file

@ -2,7 +2,7 @@ apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository kind: HelmRepository
metadata: metadata:
name: prometheus-community name: prometheus-community
namespace: flux-system namespace: monitoring
spec: spec:
interval: 24h interval: 1h
url: https://prometheus-community.github.io/helm-charts url: https://prometheus-community.github.io/helm-charts

View file

@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- repository.yaml
- release-operator.yaml

View file

@ -0,0 +1,6 @@
apiVersion: v1
kind: Namespace
metadata:
name: tailscale
labels:
pod-security.kubernetes.io/enforce: restricted

View file

@ -0,0 +1,40 @@
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: tailscale-operator
namespace: tailscale
spec:
interval: 1h
chart:
spec:
chart: tailscale-operator
version: "1.86.x"
sourceRef:
kind: HelmRepository
name: tailscale
namespace: tailscale
interval: 12h
targetNamespace: tailscale
install:
createNamespace: true
crds: Create
upgrade:
crds: CreateReplace
values:
# Operator configuration
operator:
# Tag to use for pods created by the operator (e.g., tailnet)
tags: []
# Auth key from secret - PLACEHOLDER, replace with real key
# Create secret: kubectl create secret generic tailscale-operator-authkey \
# -n tailscale --from-literal=authkey=tskey-abc123... \
# --dry-run=client -o yaml | kubectl apply -f -
operatorSecretRef:
name: tailscale-operator-authkey
key: authkey
logFile: "/var/log/tailscale.log"
# Don't block pod startup if Tailscale auth fails
waitForLinkLocal: false
# Disable webhook (optional)
webhook:
enabled: false

View file

@ -0,0 +1,8 @@
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
name: tailscale
namespace: tailscale
spec:
interval: 1h
url: https://pkgs.tailscale.com/helmcharts

View file

@ -0,0 +1,7 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- cnpg/
- keda/
- monitoring/
- tailscale/

View file

@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base/monitoring/

View file

@ -0,0 +1,4 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base/tailscale/

View file

@ -1,4 +0,0 @@
apiVersion: v1
kind: namespace
metadata:
name: monitoring

View file

@ -1,33 +0,0 @@
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: kube-prometheus-stack
namespace: monitoring
spec:
interval: 30m
chart:
spec:
chart: kube-prometheus-stack
version: "66.x"
sourceRef:
kind: HelmRepository
name: kube-prometheus-stack
namespace: monitoring
interval: 12h
install:
crds: Create
upgrade:
crds: CreateReplace
driftDetection:
mode: enabled
ignore:
# Ignore "validated" annotation which is not inserted during install
- paths: ["/metadata/annotations/prometheus-operator-validated"]
target:
kind: PrometheusRule
valuesFrom:
- kind: Secret
name: grafana-secrets

View file

@ -1,28 +0,0 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- namespace.yaml
- repository.yaml
- release.yaml
- secrets.yaml
patches:
- target:
kind: SecretProviderClass
name: grafana-secrets
patch: |-
- op: replace
path: /spec/parameters/userAssignedIdentityID
value: "10ebab78-f6c8-4d5e-ac64-c0d85fb031d0"
- op: replace
path: /spec/parameters/tenantId
value: "6c0d1498-a808-4d41-a865-fdebeb412738"
configMapGenerator:
- name: kube-prometheus-stack-values
namespace: flux-system
files:
- values.yaml=values.yaml
configurations:
- kustomizeconfig.yaml

View file

@ -1,6 +0,0 @@
nameReference:
- kind: ConfigMap
version: v1
fieldSpecs:
- path: spec/valuesFrom/name
kind: HelmRelease

View file

@ -1,33 +0,0 @@
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: kube-prometheus-stack
namespace: flux-system
spec:
targetNamespace: monitoring
interval: 30m
chart:
spec:
chart: kube-prometheus-stack
version: "80.2.0"
sourceRef:
kind: HelmRepository
name: prometheus-community
namespace: flux-system
interval: 12h
valuesFrom:
- kind: ConfigMap
name: kube-prometheus-stack-values
install:
createNamespace: true
crds: Create
upgrade:
crds: CreateReplace
driftDetection:
mode: enabled
ignore:
- paths: ["/metadata/annotations/prometheus-operator-validated"]
target:
kind: PrometheusRule

View file

@ -1,73 +0,0 @@
# Alertmanager disabled - using Grafana-native alerting
alertmanager:
enabled: false
# Prometheus configuration
prometheus:
prometheusSpec:
retention: 7d
# Select PodMonitors from all namespaces (needed for CNPG database monitoring)
podMonitorNamespaceSelector: {}
podMonitorSelectorNilUsesHelmValues: false
probeSelectorNilUsesHelmValues: false
ruleSelectorNilUsesHelmValues: false
serviceMonitorSelectorNilUsesHelmValues: false
storageSpec:
volumeClaimTemplate:
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
# Grafana configuration
grafana:
# Admin credentials - username set directly, password from Key Vault
adminUser: admin
admin:
existingSecret: grafana-admin-credentials
userKey: admin-user
passwordKey: admin-password
# Inject Telegram credentials from Key Vault
envFromSecret: grafana-env
# Ingress configuration
ingress:
enabled: true
ingressClassName:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
hosts:
- grafana.siriussec.com
tls:
- secretName: grafana-tls
hosts:
- grafana.mercury-staging.kubecraftlabs.com
# Mount CSI secrets volume to trigger secret sync
extraSecretMounts:
- name: secrets-store
mountPath: /mnt/secrets-store
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: grafana-secrets
sidecar:
dashboards:
enabled: true
label: grafana_dashboard
labelValue: "1"
searchNamespace: monitoring
folderAnnotation: grafana_folder
provider:
allowUiUpdates: true
folder: Kubernetes
alerts:
enabled: true
label: grafana_alert
labelValue: "1"
searchNamespace: monitoring
initAlerts: true

Binary file not shown.

22
orb-monitor/config.yaml Normal file
View file

@ -0,0 +1,22 @@
# ORB (Opening Range Breakout) Monitor Configuration
# ================================================
# Trading symbols and their tick multipliers (price = ticks * multiplier)
symbols:
ES: {name: "S&P 500 E-mini", exchange: "CME", multiplier: 0.25}
NQ: {name: "Nasdaq 100 E-mini", exchange: "CME", multiplier: 0.25}
YM: {name: "Dow E-mini", exchange: "CME", multiplier: 0.05}
CL: {name: "Crude Oil", exchange: "NYMEX", multiplier: 0.01}
GC: {name: "Gold", exchange: "COMEX", multiplier: 0.10}
# Opening Range Breakout settings
orb:
range_minutes: 30 # How long the opening range is measured (minutes)
filter_minutes: 0 # Delay before allowing signals after range set
min_range_ticks: 4 # Minimum range in ticks to avoid choppy markets
max_range_ticks: 100 # Maximum range in ticks (avoid invalid ranges)
# Alerts
alerts:
enabled: true
log_file: "orb_signals.log"

305
orb-monitor/monitor.py Normal file
View file

@ -0,0 +1,305 @@
#!/usr/bin/env python3
"""
ORB (Opening Range Breakout) Monitor for Futures
=================================================
Monitors session opens across global markets and detects ORB signals.
Sessions monitored (ET / UTC-4):
Asia (Tokyo) 19:00 ET (23:00 UTC)
London 03:00 ET (07:00 UTC)
New York 09:30 ET (13:30 UTC)
Strategy: After the session opens, the opening range high/low is captured
over `range_minutes`. If price later breaks above/below that range, an
ORB signal is logged.
EDUCATIONAL PURPOSE ONLY Not financial advice.
Uses yfinance (delayed data). NOT suitable for live trading.
"""
from __future__ import annotations
import sys
import logging
from datetime import datetime, timedelta, timezone
import yaml
import pandas as pd
import yfinance as yf
# ─── Constants ────────────────────────────────────────────────────────────────
# Yahoo Finance futures tickers
SYMBOLS = {
"ES": {"ticker": "ES=F", "name": "S&P 500 E-mini", "multiplier": 0.25},
"NQ": {"ticker": "NQ=F", "name": "Nasdaq 100 E-mini", "multiplier": 0.25},
"YM": {"ticker": "YM=F", "name": "Dow E-mini", "multiplier": 0.05},
"CL": {"ticker": "CL=F", "name": "Crude Oil WTI", "multiplier": 0.01},
"GC": {"ticker": "GC=F", "name": "Gold", "multiplier": 0.10},
}
# Session open times in UTC (no DST ambiguity)
SESSIONS = {
"asia": {"name": "Asia (Tokyo)", "open_utc": 23, "offset_min": 0},
"london": {"name": "London", "open_utc": 7, "offset_min": 0},
"ny": {"name": "New York", "open_utc": 13, "offset_min": 30},
}
UTC = timezone.utc
logger = logging.getLogger("ORB")
# ─── Helpers ──────────────────────────────────────────────────────────────────
def load_config(path: str = "config.yaml") -> dict:
"""Load YAML configuration."""
try:
with open(path) as fh:
return yaml.safe_load(fh)
except FileNotFoundError:
logger.warning("config.yaml not found — using defaults")
return {}
def session_utc_start(date: datetime, session: dict) -> datetime:
"""Return UTC datetime when this session opens on the given UTC date."""
return datetime(date.year, date.month, date.day,
session["open_utc"], session["offset_min"],
tzinfo=UTC)
def fetch_data(ticker: str, days: int = 5) -> pd.DataFrame:
"""Fetch intraday futures data from Yahoo Finance (1-min bars)."""
end = datetime.now(UTC)
start = end - timedelta(days=days)
try:
df = yf.download(ticker, start=start, end=end,
interval="1m", progress=False, auto_adjust=True)
if df.empty:
logger.warning(f"No data returned for {ticker}")
return df
# Flatten MultiIndex columns (yf sometimes returns ('Close', ticker), etc.)
if isinstance(df.columns, pd.MultiIndex):
df.columns = [col[0] for col in df.columns]
return df
except Exception as e:
logger.error(f"Failed to fetch {ticker}: {e}")
return pd.DataFrame()
def find_session_bars(df: pd.DataFrame, session_start: datetime,
range_minutes: int) -> pd.DataFrame | None:
"""Extract the opening-range bars for a session, if data exists."""
# Allow ±2 min tolerance for session start
tolerance = timedelta(minutes=2)
end_bound = session_start + timedelta(minutes=range_minutes) + tolerance
mask = (df.index >= session_start - tolerance) & \
(df.index < end_bound)
range_bars = df.loc[mask]
return range_bars if len(range_bars) >= 5 else None # Need meaningful data
def analyze_orb(symbol_key: str, symbol_info: dict, df: pd.DataFrame,
session_key: str, session_info: dict,
cfg_orb: dict, date: datetime) -> list[dict]:
"""Check for ORB signals in the data for a given session date."""
range_min = cfg_orb.get("range_minutes", 30)
min_range = cfg_orb.get("min_range_ticks", 4)
max_range = cfg_orb.get("max_range_ticks", 100)
multiplier = symbol_info["multiplier"]
signals: list[dict] = []
session_start = session_utc_start(date, session_info)
# Try both start date and day before (in case of overnight sessions)
for offset in [0, -1]:
check_date = date + timedelta(days=offset)
try_start = datetime(check_date.year, check_date.month, check_date.day,
session_info["open_utc"], session_info["offset_min"],
tzinfo=UTC)
range_bars = find_session_bars(df, try_start, range_min)
if range_bars is None:
continue
# Opening range high/low — force scalar extraction
range_high = range_bars["High"].max().item()
range_low = range_bars["Low"].min().item()
range_size = range_high - range_low
range_ticks = range_size / multiplier
if range_ticks < min_range or range_ticks > max_range:
continue # Skip — range too small or too large
# Look for breakout in remaining data after range period
# Skip NaN rows and only use real data
remaining = df.loc[range_bars.index[-1]:].dropna(subset=["Close"])
if remaining.empty:
continue
# Bullish breakout: price closes above range high
bullish_bars = remaining[remaining["Close"] > range_high]
if not bullish_bars.empty:
breakout_time = bullish_bars.index[0]
breakout_price = float(bullish_bars.loc[breakout_time, "Close"])
signals.append({
"symbol": symbol_key,
"name": symbol_info["name"],
"session": session_info["name"],
"direction": "LONG",
"range_high": round(range_high, 2),
"range_low": round(range_low, 2),
"range_size": round(range_size, 2),
"range_ticks": round(range_ticks, 1),
"breakout_time": breakout_time,
"breakout_price": round(breakout_price, 2),
})
# Bearish breakout: price closes below range low
bearish_bars = remaining[remaining["Close"] < range_low]
if not bearish_bars.empty:
breakout_time = bearish_bars.index[0]
breakout_price = float(bearish_bars.loc[breakout_time, "Close"])
signals.append({
"symbol": symbol_key,
"name": symbol_info["name"],
"session": session_info["name"],
"direction": "SHORT",
"range_high": round(range_high, 2),
"range_low": round(range_low, 2),
"range_size": round(range_size, 2),
"range_ticks": round(range_ticks, 1),
"breakout_time": breakout_time,
"breakout_price": round(breakout_price, 2),
})
return signals
# ─── Main ─────────────────────────────────────────────────────────────────────
def run(date_str: str | None = None, days: int = 5,
config_path: str = "config.yaml") -> list[dict]:
"""
Analyze ORB patterns for all sessions and symbols.
Args:
date_str: Optional date in YYYY-MM-DD format. If None, uses today.
days: How many days of history to fetch.
config_path: Path to config.yaml.
Returns:
List of signal dicts sorted by breakout_time.
"""
cfg = load_config(config_path)
cfg_orb = cfg.get("orb", {})
target_date = datetime.strptime(date_str, "%Y-%m-%d").replace(tzinfo=UTC) if date_str else datetime.now(UTC)
# Expand search window to cover all sessions around the target date
search_dates = [target_date + timedelta(days=d) for d in range(-1, days)]
all_signals: list[dict] = []
symbol_items = cfg.get("symbols", SYMBOLS) or SYMBOLS
for sym_key, sym_info in symbol_items.items():
ticker = sym_info.get("ticker", f"{sym_key}=F")
print(f" Fetching {ticker} ({sym_info.get('name', sym_key)}) …", flush=True)
df = fetch_data(ticker, days=days)
if df.empty:
continue
# Reconcile multiplier from config vs hardcoded
sym_info.setdefault("multiplier", SYMBOLS.get(sym_key, {}).get("multiplier", 0.25))
for sd in search_dates:
for sess_key, sess_info in SESSIONS.items():
signals = analyze_orb(
sym_key, sym_info, df,
sess_key, sess_info, cfg_orb, sd
)
all_signals.extend(signals)
# Sort by breakout time
all_signals.sort(key=lambda s: s["breakout_time"])
return all_signals
def format_report(signals: list[dict]) -> str:
"""Pretty-print ORB signals for Telegram / terminal."""
if not signals:
return (
"📊 **ORB Scan Complete — No Signals Found**\n\n"
"No opening range breakouts detected in the scanned period.\n"
"The market may be quiet, or the data may be too delayed.\n\n"
"*Run again closer to session opens for best results.*"
)
lines = [
f"📊 **ORB Signals Found** ({len(signals)} signals)",
f"_Scanned: {datetime.now(UTC).strftime('%Y-%m-%d %H:%M UTC')}_",
"" * 40,
]
for s in signals:
direction = "🟢 LONG" if s["direction"] == "LONG" else "🔴 SHORT"
lines.append(
f"\n**{s['symbol']}** ({s['name']}) — {s['session']}\n"
f"{direction}\n"
f" Range: {s['range_low']} {s['range_high']} "
f"({s['range_size']} pts / {s['range_ticks']} ticks)\n"
f" Breakout: {s['breakout_price']} at "
f"`{s['breakout_time'].strftime('%H:%M UTC')}`"
)
lines.append("\n" + "" * 40)
lines.append(
"⚠️ _Educational analysis only. Uses delayed data._\n"
"_Not financial advice. Verify with live data before trading._"
)
return "\n".join(lines)
# ─── CLI ──────────────────────────────────────────────────────────────────────
def main():
logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")
import argparse
parser = argparse.ArgumentParser(description="ORB Futures Monitor")
parser.add_argument("--date", type=str, default=None,
help="Target date YYYY-MM-DD (default: today)")
parser.add_argument("--days", type=int, default=5,
help="Days of history to scan (default: 5)")
parser.add_argument("--config", type=str, default="config.yaml",
help="Path to config file")
parser.add_argument("--json", action="store_true",
help="Output as JSON instead of formatted text")
args = parser.parse_args()
print("\n🔍 ORB Monitor — Scanning futures data …\n", flush=True)
signals = run(date_str=args.date, days=args.days, config_path=args.config)
if args.json:
import json
print(json.dumps(signals, indent=2, default=str))
else:
report = format_report(signals)
print(report)
# Also save to log
log_path = "orb_signals.log"
with open(log_path, "a") as fh:
fh.write(f"\n{'='*50}\n")
fh.write(f"Scan: {datetime.now(UTC).isoformat()}\n")
fh.write(report + "\n")
print(f"\n✅ Done. {len(signals)} signals detected.\n", flush=True)
if __name__ == "__main__":
main()

161
orb-monitor/orb_signals.log Normal file
View file

@ -0,0 +1,161 @@
==================================================
Scan: 2026-04-26T01:42:37.376486+00:00
📊 **ORB Signals Found** (8 signals)
_Scanned: 2026-04-26 01:42 UTC_
────────────────────────────────────────
**ES** (S&P 500 E-mini) — London
🟢 LONG
Range: 7145.5 7155.5 (10.0 pts / 40.0 ticks)
Breakout: Ticker
ES=F NaN
Name: 2026-04-24 07:31:00+00:00, dtype: float64 at `07:31 UTC`
**ES** (S&P 500 E-mini) — London
🔴 SHORT
Range: 7145.5 7155.5 (10.0 pts / 40.0 ticks)
Breakout: Ticker
ES=F NaN
Name: 2026-04-24 07:31:00+00:00, dtype: float64 at `07:31 UTC`
**CL** (Crude Oil) — London
🟢 LONG
Range: 96.01 96.64 (0.63 pts / 63.0 ticks)
Breakout: Ticker
CL=F NaN
Name: 2026-04-24 07:31:00+00:00, dtype: float64 at `07:31 UTC`
**CL** (Crude Oil) — London
🔴 SHORT
Range: 96.01 96.64 (0.63 pts / 63.0 ticks)
Breakout: Ticker
CL=F NaN
Name: 2026-04-24 07:31:00+00:00, dtype: float64 at `07:31 UTC`
**ES** (S&P 500 E-mini) — New York
🟢 LONG
Range: 7145.0 7166.75 (21.75 pts / 87.0 ticks)
Breakout: Ticker
ES=F NaN
Name: 2026-04-24 14:01:00+00:00, dtype: float64 at `14:01 UTC`
**ES** (S&P 500 E-mini) — New York
🔴 SHORT
Range: 7145.0 7166.75 (21.75 pts / 87.0 ticks)
Breakout: Ticker
ES=F NaN
Name: 2026-04-24 14:01:00+00:00, dtype: float64 at `14:01 UTC`
**CL** (Crude Oil) — New York
🟢 LONG
Range: 94.85 95.59 (0.74 pts / 74.0 ticks)
Breakout: Ticker
CL=F NaN
Name: 2026-04-24 14:01:00+00:00, dtype: float64 at `14:01 UTC`
**CL** (Crude Oil) — New York
🔴 SHORT
Range: 94.85 95.59 (0.74 pts / 74.0 ticks)
Breakout: Ticker
CL=F NaN
Name: 2026-04-24 14:01:00+00:00, dtype: float64 at `14:01 UTC`
────────────────────────────────────────
⚠️ _Educational analysis only. Uses delayed data._
_Not financial advice. Verify with live data before trading._
==================================================
Scan: 2026-04-26T01:43:22.314114+00:00
📊 **ORB Signals Found** (8 signals)
_Scanned: 2026-04-26 01:43 UTC_
────────────────────────────────────────
**ES** (S&P 500 E-mini) — London
🟢 LONG
Range: 7145.5 7155.5 (10.0 pts / 40.0 ticks)
Breakout: nan at `07:31 UTC`
**ES** (S&P 500 E-mini) — London
🔴 SHORT
Range: 7145.5 7155.5 (10.0 pts / 40.0 ticks)
Breakout: nan at `07:31 UTC`
**CL** (Crude Oil) — London
🟢 LONG
Range: 96.01 96.64 (0.63 pts / 63.0 ticks)
Breakout: nan at `07:31 UTC`
**CL** (Crude Oil) — London
🔴 SHORT
Range: 96.01 96.64 (0.63 pts / 63.0 ticks)
Breakout: nan at `07:31 UTC`
**ES** (S&P 500 E-mini) — New York
🟢 LONG
Range: 7145.0 7166.75 (21.75 pts / 87.0 ticks)
Breakout: nan at `14:01 UTC`
**ES** (S&P 500 E-mini) — New York
🔴 SHORT
Range: 7145.0 7166.75 (21.75 pts / 87.0 ticks)
Breakout: nan at `14:01 UTC`
**CL** (Crude Oil) — New York
🟢 LONG
Range: 94.85 95.59 (0.74 pts / 74.0 ticks)
Breakout: nan at `14:01 UTC`
**CL** (Crude Oil) — New York
🔴 SHORT
Range: 94.85 95.59 (0.74 pts / 74.0 ticks)
Breakout: nan at `14:01 UTC`
────────────────────────────────────────
⚠️ _Educational analysis only. Uses delayed data._
_Not financial advice. Verify with live data before trading._
==================================================
Scan: 2026-04-26T01:44:52.987075+00:00
📊 **ORB Signals Found** (7 signals)
_Scanned: 2026-04-26 01:44 UTC_
────────────────────────────────────────
**CL** (Crude Oil) — London
🟢 LONG
Range: 96.01 96.64 (0.63 pts / 63.0 ticks)
Breakout: 96.75 at `07:45 UTC`
**ES** (S&P 500 E-mini) — London
🟢 LONG
Range: 7145.5 7155.5 (10.0 pts / 40.0 ticks)
Breakout: 7157.25 at `08:20 UTC`
**ES** (S&P 500 E-mini) — London
🔴 SHORT
Range: 7145.5 7155.5 (10.0 pts / 40.0 ticks)
Breakout: 7145.0 at `08:34 UTC`
**CL** (Crude Oil) — London
🔴 SHORT
Range: 96.01 96.64 (0.63 pts / 63.0 ticks)
Breakout: 95.92 at `11:04 UTC`
**CL** (Crude Oil) — New York
🔴 SHORT
Range: 94.85 95.59 (0.74 pts / 74.0 ticks)
Breakout: 94.83 at `14:12 UTC`
**ES** (S&P 500 E-mini) — New York
🟢 LONG
Range: 7145.0 7166.75 (21.75 pts / 87.0 ticks)
Breakout: 7169.0 at `14:46 UTC`
**CL** (Crude Oil) — New York
🟢 LONG
Range: 94.85 95.59 (0.74 pts / 74.0 ticks)
Breakout: 95.7 at `15:06 UTC`
────────────────────────────────────────
⚠️ _Educational analysis only. Uses delayed data._
_Not financial advice. Verify with live data before trading._