📊 Dashboard
📈 System Overview
📅 Business Date
⚡ Quick Actions
🏭 Sources
Target tables and their data-source provenance, by clearing house. Aligned with Daily Load (CCP): Direct inputs are grouped Static → CCP Report → Members → Subscriptions. Expand a row to see each field's source.
Loading source data…
🔀 LCH Data Flow ▸ expand
How source reports flow through transforms into COP target tables and the optimiser.
👥 Member Report Overrides
Members with individual reports in dataimports/members/ can override the LCH pipeline.
Toggle a member on to use their individual reports instead of the centrally-transformed data.
🏦 Daily Load — LCH
Load actual LCH member & public reports into the COP database, in dependency order (static reference → prices → positions). Recon-health checks run inline below as part of the load.
1 Sources detected
Select a date to scan source folders…
2 Load health & reconciliation
Run a validation or load to populate recon checks…
3 Run
4 Results
Per-table row counts appear here after a load.
⏰ Auto-load scheduler
▸ expand
Automatically poll for new data (checks _control.json status). When new data is detected, it is consumed automatically into PostgreSQL.
📋 Active scheduler jobs
🏦 Load CCP Pipeline
The LCH daily load (source detection, validation, recon health, and load controls) now lives in its own screen.
📦 Load Stale Data
▸ expand
🗄️ Stale Verified Data
Load cop.sql — a verified snapshot of real production data captured on
2025-05-13. This replaces all data in the cop schema
with a known-good baseline. Useful for regression testing and UI validation.
Source: cop.sql (mounted from project root) ·
Size: ~150 MB ·
Schema: cop (PostgreSQL 17.5 dump)
🏭 Full Production Dump
Load the full production database dump captured on 2025-02-19.
The dump is stored as split zip files in db_dump/. When you click the
button, the system will:
- Reassemble all split parts into a single zip
- Unzip to a temporary SQL file
- Restore into PostgreSQL (
DROP SCHEMA cop CASCADE+psql) - Delete the temporary SQL file (split zips are kept)
db_dump/cop_prod_dump.sql.zip.split_* (28 parts) ·
Compressed: ~1.2 GB ·
Uncompressed: ~3.1 GB ·
Restore time: ~10–15 min⚠ This will replace ALL data in the cop schema.
⏰ Scheduler
▸ expand
Quick presets or custom cron schedules for automatic data generation.
🔧 Generate Synthetic Data
▸ expand
📋 Activity Log
| Time | Service | User | Action | Status | Detail |
|---|---|---|---|---|---|
| Loading… | |||||
📊 Active Data Profile
🌐 Infrastructure
… — generated/transformed CSVs written here…/dataimports → LCH source reports… — stale database dump… — production database splits
📊 Generate Page
Choose which sections are visible on the Generate page.
📂 LCH Report Paths
Paths to LCH central clearing data used by the feed pipeline. Per-table source configuration is managed in Sources.
report-samples/dataimports/ (Docker) or via PVC (Kubernetes).
📊 LCH Source Reports
All internal LCH reports consumed by the pipeline, grouped by importance tier. The pipeline adapts gracefully — more reports available → better data quality.
👥 Member Report Sources
Configure per-member report directories for the LCH feed pipeline. When enabled, member-specific reports
replace/supplement LCH central data for that member's rows during the transform.
Members with directories in dataimports/members/ are auto-detected. Add any member below — directory can be created later.
📤 Report File Transfer
Upload or download report archives to sync between local and remote environments.
📦 Data Source Subscriptions
Enable or disable data source adapters. Enabled sources appear in the Sources and Configure tabs. ℹ️