📊 Dashboard
📈 System Overview
📅 Business Date
⚡ Quick Actions
🏭 Sources
📋 CCP Source Configuration
Target tables and their data source assignments per CCP. Each tab represents a clearing house with its specific data requirements.
| Target Table ↕ | Source Type ↕ | Status ↕ | Resolved From ↕ | Actions |
|---|---|---|---|---|
| Loading source data... | ||||
🔀 LCH Data Flow ▸ expand
How source reports flow through transforms into COP target tables and the optimiser.
👥 Member Report Overrides
Members with individual reports in dataimports/members/ can override the LCH pipeline.
Toggle a member on to use their individual reports instead of the centrally-transformed data.
🏭 Load CCP Pipeline
Reads source files configured in the Sources tab, transforms LCH reports into 19 COP tables ready for database loading.
LCH LCH Feed Pipeline
Transform actual LCH report files from report-samples/dataimports/ into
COP-format CSV files.
Source: dataimports/YYYYMMDD-lch/ + dataimports/static-lch/ + dataimports/members/ ·
Output: 19 COP CSV files → /app/data/
⏰ Scheduler
▸ expand
Quick presets or custom cron schedules for automatic data generation.
📦 Load Snapshot
▸ expand
🗄️ Stale Verified Data
Load cop.sql — a verified snapshot of real production data captured on
2025-05-13. This replaces all data in the cop schema
with a known-good baseline. Useful for regression testing and UI validation.
Source: cop.sql (mounted from project root) ·
Size: ~150 MB ·
Schema: cop (PostgreSQL 17.5 dump)
🏭 Full Production Dump
Load the full production database dump captured on 2025-02-19.
The dump is stored as split zip files in db_dump/. When you click the
button, the system will:
- Reassemble all split parts into a single zip
- Unzip to a temporary SQL file
- Restore into PostgreSQL (
DROP SCHEMA cop CASCADE+psql) - Delete the temporary SQL file (split zips are kept)
db_dump/cop_prod_dump.sql.zip.split_* (28 parts) ·
Compressed: ~1.2 GB ·
Uncompressed: ~3.1 GB ·
Restore time: ~10–15 min⚠ This will replace ALL data in the cop schema.
🔧 Generate Synthetic Data
▸ expand
📥 Load to DB Stopped
Load CSV files from the data directory into PostgreSQL (truncate + insert for all tables).
⏰ Consumer Scheduler
Automatically poll for new data (checks _control.json status). When new data is detected, it is consumed automatically.
📋 Active Scheduler Jobs
📋 Activity Log
| Time | Service | User | Action | Status | Detail |
|---|---|---|---|---|---|
| Loading… | |||||
📊 Active Data Profile
🌐 Infrastructure
… — generated/transformed CSVs written here…/dataimports → LCH source reports… — stale database dump… — production database splits
📊 Generate Page
Choose which sections are visible on the Generate page.
📂 LCH Report Paths
Paths to LCH central clearing data used by the feed pipeline. Per-table source configuration is managed in Sources.
report-samples/dataimports/ (Docker) or via PVC (Kubernetes).
👥 Member Report Sources
Configure per-member report directories for the LCH feed pipeline. When enabled, member-specific reports
replace/supplement LCH central data for that member's rows during the transform.
Members with directories in dataimports/members/ are auto-detected. Add any member below — directory can be created later.
📤 Report File Transfer
Upload or download report archives to sync between local and remote environments.
📦 Data Source Subscriptions
Enable or disable data source adapters. Enabled sources appear in the Sources and Configure tabs. ℹ️
📐 Data Model View
Business-oriented view of all COP data requirements. Shows what data is needed, where it comes from, and what's missing. ℹ️
Loading data model…