Stock Crypto Sentiment Analyzer

Sentiment-regime overlay for algorithmic trading bots. This package ingests tariff-related tweets, policy news, and market data; classifies narratives; scores crash/surge probability; detects market regimes; and exposes guardrail middleware so technical-analysis (TA) strategies can derisk automatically.

Features

• Feed ingestion: Twitter/X filtered stream, NewsAPI-compatible poller, and market-data sampler auto-register via environment variables.
• Narrative classification: Keyword pre-filter + optional transformer (via @xenova/transformers) label events as threat, escalation, de-escalation, or noise.
• Narrative taxonomy & overrides: Six-category taxonomy assets (assets/taxonomy/**) with risk-profile overrides keep recall ≥90% per storyline while enabling ticker/sector-specific tuning.
• Ticker-aware sentiment: Entity extraction + rolling symbol store expose intraday crash/surge/confidence snapshots via overlay.getSymbolSentiment(symbol) and the yarn symbol:sentiment CLI.
• Catalyst ingestors: Earnings calendar, pre/post-market gapper feeds, and the macro calendar adapter normalize catalysts into sentiment events so guardrails can differentiate planned beats/misses from CPI/FOMC/jobs shocks.
• Transformer presets: Optional FinBERT/DeBERTa/FinGPT pipelines download into a managed cache (SENTIMENT_MODEL_PATH) and blend with the heuristic scorer when SENTIMENT_SCORING_ENABLE=true.
• Custom feed hooks: Template adapters (Alpaca/Polygon) plus logging guidance let you plug proprietary catalysts directly into the overlay (docs/feeds/custom-feed-hooks.md).
• Symbol metrics & alerts: /metrics exports per-symbol threat/escalation/de-escalation counters with Alertmanager/Grafana recipes (docs/alerts/symbol-metrics.md, docs/dashboards/symbol-heatmap.json).
• Sentiment scoring: Computes (crashProb, surgeProb, confidence), persists scores with TTL, and logs context for audits.
• Regime detection: Rolling volatility/liquidity metrics classify regimes (fragile, trend, range, calm).
• Guardrail engine: Middleware that modulates TA signals, blocks high-risk trades, caps leverage after de-escalations, and emits structured decision logs.
• Auditability & governance: Encrypted decision evidence vault with role-gated exports ensures every trade links back to raw sentiment events and executed orders.
• Tooling: Annotation CLI, backtest helper, feed connectivity scripts, and staging configuration guide.

Installation

yarn add @developerzava/stock-crypto-sentiment-analyzer
# or
yarn add ../path/to/stock-crypto-sentiment-analyzer/tmp/analyzer.tgz

Environment Variables

Copy .env.example and fill in credentials:

SENTIMENT_TWITTER_BEARER=...
SENTIMENT_NEWS_API_KEY=...
SENTIMENT_NEWS_MAX_AGE_DAYS=2
SENTIMENT_OFFICIAL_SOCIAL_ENABLED=true
SENTIMENT_OFFICIAL_SOCIAL_REQUIRED=true
SENTIMENT_MARKET_DATA_URL=https://...
SENTIMENT_CLASSIFIER_ENABLE=true
SENTIMENT_CLASSIFIER_MODEL=Xenova/distilbert-base-uncased-finetuned-sst-2-english
SENTIMENT_SCORING_ENABLE=true
SENTIMENT_TRANSFORMER_PRESET=finbert
SENTIMENT_TRANSFORMER_CACHE=./data/models/transformers
SENTIMENT_SCORING_MODEL=Xenova/distilbert-base-uncased-finetuned-sst-2-english
SENTIMENT_ALERT_WEBHOOK=https://discord.com/api/webhooks/.../...
SENTIMENT_ALERT_MAX_PER_MINUTE=30
SENTIMENT_ALERT_HISTORICAL_LOOKBACK_MINUTES=60
SENTIMENT_ALERT_SUPPRESS_HISTORICAL_SLA=true
SENTIMENT_ALERT_MAX_RETRIES=3
SENTIMENT_REDIS_URL=redis://127.0.0.1:6379/0
SENTIMENT_REDIS_DB=0
SENTIMENT_REDIS_PREFIX=sentiment:
SENTIMENT_MODEL_PATH=./models/latest
SENTIMENT_GUARDRAIL_CONFIG=./config/guardrails.default.json
SENTIMENT_STRATEGY_ID=spy-0-dte
SENTIMENT_EVIDENCE_ENCRYPTION_KEY=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
SENTIMENT_EVIDENCE_ROLE=compliance
SENTIMENT_EVIDENCE_PATH=./data/evidence/decisions
SENTIMENT_SYMBOL_SENTIMENT_PATH=./data/symbol-sentiment.json
SENTIMENT_EARNINGS_API_URL=https://api.example.com/earnings
SENTIMENT_EARNINGS_API_KEY=abc123
SENTIMENT_EARNINGS_POLL_INTERVAL_MS=300000
SENTIMENT_GAPPERS_API_URL=https://api.example.com/gappers
SENTIMENT_GAPPERS_API_KEY=abc123
SENTIMENT_GAPPERS_SESSION=pre
SENTIMENT_GAPPERS_POLL_INTERVAL_MS=60000
SENTIMENT_GAPPERS_MIN_GAP_PCT=3
SENTIMENT_MACRO_CALENDAR_URL=https://api.example.com/macro
SENTIMENT_MACRO_CALENDAR_API_KEY=abc123
SENTIMENT_MACRO_CALENDAR_POLL_INTERVAL_MS=300000

When running the overlay in Docker while Redis lives on the host, point SENTIMENT_REDIS_URL at redis://host.docker.internal:6379 (or any hostname reachable from both host and container) and pair it with SENTIMENT_REDIS_DB to pick a DB index that does not collide with other apps. Use SENTIMENT_REDIS_PREFIX to namespace keys (defaults to sentiment:). Leave SENTIMENT_GUARDRAIL_CONFIG unset unless you supply a real override file; the bundled default loads automatically. See docs/staging-config.md for provisioning details and docs/integration.md for full variable descriptions.

SENTIMENT_SCORING_ENABLE=true enables the transformer blend; pick a preset with SENTIMENT_TRANSFORMER_PRESET (finbert, deberta-v3-large, or fingpt). Cache downloads under SENTIMENT_MODEL_PATH by default, or override with SENTIMENT_TRANSFORMER_CACHE if the bots mount a different volume.

Quick Start

import {
  createSentimentOverlay,
  DecisionAuditTrail,
  GuardrailEngine,
  SentimentGuardrail,
} from '@developerzava/stock-crypto-sentiment-analyzer';
import { createStructuredLogger } from '@developerzava/stock-crypto-sentiment-analyzer/logging';

const logger = createStructuredLogger({ serviceName: 'spy-0-dte' });
const overlay = await createSentimentOverlay({ env: process.env, logger });
const guardrails = new GuardrailEngine({ overlay, logger, earningsWindowMinutes: 120 });
const encryptionKey = process.env.SENTIMENT_EVIDENCE_ENCRYPTION_KEY;
if (!encryptionKey) throw new Error('SENTIMENT_EVIDENCE_ENCRYPTION_KEY must be set');
const auditTrail = new DecisionAuditTrail({
  strategyId: process.env.SENTIMENT_STRATEGY_ID ?? 'spy-0-dte',
  encryptionKey,
  logger,
});

export async function executeTaSignal(signal: TradeSignal) {
  const decision = await guardrails.evaluate(signal);
  if (!decision.allowed) {
    logger.warn({
      category: LogCategory.DECISION,
      action: 'trade-blocked',
      message: 'Guardrail blocked TA signal',
      details: { reason: decision.reason },
    });
    return;
  }

  const execution = await orderRouter.submit(decision.adjustedSignal);
  await auditTrail.record({
    decision,
    requestedSignal: signal,
    executedOrders: execution.orders, // array of { orderId, symbol, side, quantity, executedAt }
    orderBatchId: execution.batchId,
  });
}

const tslaSentiment = overlay.getSymbolSentiment('TSLA');
logger.info({
  category: LogCategory.DECISION,
  action: 'symbol-sentiment-snapshot',
  message: 'Latest symbol sentiment snapshot',
  details: { symbol: 'TSLA', sentiment: tslaSentiment },
});

The overlay automatically:

1. Registers feeds (Twitter, news, market data) if credentials exist.
2. Buffers events, classifies narratives, runs sentiment scoring, and emits structured events.
3. Updates overlay context (crash/surge probability, confidence, regime, recent events).
4. GuardrailEngine pulls that context for every TA decision.

Runtime compatibility

• The package ships both ESM and CJS bundles; asset resolution (FinBERT model config, taxonomy) uses Node's module resolver via require.resolve(), ensuring reliable path resolution in all installation scenarios (npm/yarn/pnpm, monorepos, Docker, etc.).
• Logs must include action, category, and message. The provided logger wrapper will map legacy msg to action/message, but downstream bots should emit structured events directly to avoid validator rejections.

CLI & Scripts

• yarn label:events — append labeled rows to data/annotations.csv.
• yarn evaluate:keywords — enforce ≥90% recall per taxonomy category via data/taxonomy-annotations.csv.
• yarn backtest:sentiment — run a simple crash-detection backtest using data/backtest/sample-events.json.
• yarn verify:feeds — ensure required environment variables are set.
• yarn ping:twitter|news|market — smoke test external APIs.
• yarn export:evidence --from=2025-03-01 --to=2025-03-07 --role=compliance — decrypts decision logs for investigations (requires SENTIMENT_EVIDENCE_ENCRYPTION_KEY + authorized role).
• yarn symbol:sentiment --symbol=TSLA — print the latest crash/surge/confidence snapshots for a ticker from the persisted store.

Development

yarn install
yarn lint
yarn test
yarn build

Publishing (dry-run):

npm publish --dry-run

prepublishOnly runs lint→test→build automatically.

Architecture Overview

Feeds (Twitter/News/Market) -> NarrativeClassifier -> SentimentScoring -> ScoreStore
                                                \-> AlertDispatcher (threat webhooks)
Overlay Context (crashProb, surgeProb, confidence, regime, latest events)
         ^
         |
  RegimeDetector <- MarketDataPoller
         |
GuardrailEngine + SentimentGuardrail -> Adjusted TA Signals -> Order Router

Testing

• Unit tests with Vitest live under tests/unit/**. Run yarn test for the full suite.
• Mock overlay (createMockOverlay) and GuardrailEngine tests ensure risk rules work without live feeds.

Documentation

• docs/integration.md — installation, env vars, guardrail usage, logging/metrics.
• docs/staging-config.md — credential provisioning checklist.
• internal/stories/user-stories.md (repo-only) — roadmap and status of major features.
• docs/data-lineage.md / docs/security/data-access.md — evidence lineage diagram plus access-control policy for decision logs.

License

MIT © developerzava