2. End-to-End Operational Lifecycle & Workflow

The signal generation framework is orchestrated through a sequence of modular shell wrappers, Python vectorization modules, analytics engines, and reporting generators:

flowchart TD
    A[cron_daily_signals.sh<br/>Entrypoint: 21:00 UTC] --> B[Stage 1: Init, Mutex Lock & Log Setup]
    B --> C[Stage 2: Universe Sync<br/>run_symbol_files_update.sh]
    C --> D[Stage 3: Strategy Loop<br/>Iterate stage/strategies.yaml]

    subgraph Strategy_Execution [Per-Strategy Execution Pipeline]
        D --> E[3.1 Vectorized Sweep Runner<br/>run_signals_sweep.sh]
        E --> F[(QuestDB: EOD Daily Bars<br/>us_equities_fmp_1d)]
        F --> G[Polars SIMD Vectorized<br/>Indicator & Score Engine]
        G --> H[(QuestDB: 15m Bars<br/>us_equities_fmp_15min)]
        H --> I[15m EMA Regime Filter<br/>50 / 200 / 800 Triad]
        I --> J[Scenario Signal Artifacts<br/>signals.parquet / signals.csv]

        J --> K[Stage 4: HTML Diagnostics<br/>technosight.signals.reporting]
        K --> L[daily_signal_report.html<br/>Master Dashboard]
        K --> M[results.html<br/>Per-Scenario Dashboards]
        K --> S[sweep_summary.csv / .parquet]

        J --> N[Stage 5: Candidate Ranking<br/>run_promising_symbols.sh]
        N --> O[3-Factor Promising Score<br/>Gain % + Signal Score + Sentiment]
        O --> P[Pre-Enrichment Boundary<br/>Top 50 Long + Top 50 Short]
        P --> Q[FMP API Enrichment<br/>Valuation, PEG, Consensus]
        Q --> R[promising_symbols.csv<br/>Final Enriched Table]
    end

    Strategy_Execution --> Z[Housekeeping & Log Rotation<br/>Retain newest 30 log files]

Stage 1: Environment, Lock Management, and Logging Setup

• Trigger & Schedule: Invoked on weekday evenings after market close (typically 21:00 UTC / 17:00 EST) via cron_daily_signals.sh.
• Mutex Concurrency Lock: Creates a kernel file lock (/tmp/technosight_cron_daily.lock). If another instance is active, the script terminates immediately to prevent duplicate database loads and write collisions.
• Signal Traps: Uses trap cleanup EXIT INT TERM to guarantee automatic lock removal on termination or interruption.
• Environment & Logging: Sources .env, configures Python path, resolves the virtual environment, and streams execution output to timestamped files (logs/cron_daily_<YYYYMMDD_HHMMSS>.log).

Stage 2: Symbol Universe Synchronization

• Universe Refresh: Executes run_symbol_files_update.sh (unless --skip-symbol-update is specified).
• Directory Mirroring: Synchronizes active and delisted equity universes from upstream stores into local data/symbols_fmp/ and data/symbols_delisted/.
• Sanity Checks: Verifies ticker existence, filters out bankrupt or suspended assets, and ensures minimum trading history requirements (at least 252 trading days).

Stage 3: Strategy Manifest Discovery & Vectorized Scenario Sweeps

• Manifest Ingestion: Reads stage/strategies.yaml to identify all enabled quantitative strategies (mean_reversion, statistical_arbitrage, momentum, momentum_omega, trend_following, volatility_breakout).
• 35 Scenario Configurations: For each active strategy, loads 35 scenario files (scenario_A1.yaml through scenario_G5.yaml) spanning 7 liquidity tranches (A to G) and 5 parameter tiers (1 to 5).
• Vectorized Computation: run_signals_sweep.sh executes Polars SIMD indicator calculations in parallel worker processes, querying QuestDB daily OHLCV bars (us_equities_fmp_1d) and intraday 15-minute bars (us_equities_fmp_15min).
• Regime Evaluation: Evaluates 15m EMA alignment ($50, 200, 800$), assigning regime tags (AAA, BBB, NEUTRAL, BYPASS) to every evaluated bar.
• Raw Signal Storage: Saves full-universe signal datasets to signals.parquet and signals.csv within the date partition.

Stage 4: HTML Reporting and Scenario Diagnostics

• Reporting Engine: Invokes technosight.signals.reporting (SignalAnalyser).
• Master Dashboard Compilation: Compiles daily_signal_report.html, featuring top-level KPI cards, intraday EMA regime distribution charts, the 11-column Top Ranked Promising Symbols table, and the 7-column Parameter Sweep Scenarios Comparison table.
• Scenario Diagnostic Pages: Generates individual scenario teardowns (scenario_<ID>/results.html) with equity curves and trade metrics.
• Sweep Summaries: Aggregates total signals, long/short counts, and mean scores into sweep_summary.csv and sweep_summary.parquet.

Stage 5: Candidate Ranking, Multi-Factor Scoring, FMP Enrichment & Housekeeping

• Promising Score Engine: Invokes run_promising_symbols.sh (technosight.analytics.promising_score).
• 3-Factor Score Computation: Evaluates normalized Average Gain %, Strategy Signal Score, and Side-Aware Sentiment.
• Pre-Enrichment Selection Boundary: Partitions candidates into Long and Short pools, selecting up to 50 Longs and up to 50 Shorts (maximum 100 symbols) strictly before external API calls.
• Contextual Enrichment: Queries Financial Modeling Prep (FMP) APIs to append corporate names, forward valuation multiples (P/E, Forward PEG, FCF Yield), Wall Street recommendation distributions, and consensus price targets.
• CSV Export: Emits promising_symbols.csv formatted according to strict, non-duplicated schemas.
• Housekeeping: Purges temporary scratch files and rotates log files, preserving the newest 30 logs.

On every execution date, the pipeline compiles a structured hierarchy of artifacts organized by strategy and calculation date:

selected_stocks/{strategy_name}/result/{YYYY-MM-DD}/
├── daily_signal_report.html           # Master executive signal dashboard
├── promising_symbols.csv              # Enriched top candidate list (max 100)
├── sweep_summary.csv                  # Aggregated 35-scenario summary metrics
├── sweep_summary.parquet              # Columnar 35-scenario summary
├── signals.parquet                    # Raw full-universe vectorized signals
├── signals.csv                        # Tabular full-universe signals
└── scenario_<ID>/                     # 35 scenario sub-directories (A1...G5)
    └── results.html (or result.html)  # Per-scenario diagnostic tearsheet

Complete Artifacts Specifications