Meyi Insight Cost Server

Tenant-aware Express plugin that provides AWS cost overview, reports, dynamic filters, budgets, and budget-alert dismissals. It supports AWS Cost Explorer and optional centralized Cost and Usage Report (CUR) queries through Athena. The customer role and SaaS Athena identity are deliberately separate security boundaries.

The package is designed to be mounted by a host backend. The host owns login, token verification, tenant/plugin enablement, PostgreSQL connection creation, and AWS onboarding. The plugin owns cost routes and its budget-related tables.

Package contract

• Package name: meyi-insight-cost-server
• Runtime: Node.js 20 or newer, ESM
• Framework: Express 4
• Database API: injected Drizzle PostgreSQL instance
• Peer dependency: pg >= 8
• Default route: /api/v1/cost
• Public factory: createInsightCost({ app, db, apiBaseUri, logger })

Features

• Tenant-isolated payer/organization and member-account cost views
• Monthly and yearly overview data
• Service, account, region, resource, and cost-allocation-tag reports
• Dynamic service, region, linked-account, and tag filter options
• Standard and comparison report data used by meyi-cost-ui
• CUR/Athena active-resource counts and detailed cost data
• Cost Explorer fallback when CUR is unavailable in auto mode
• Tenant-scoped budget rules
• User-, tenant-, month-, and status-scoped budget-alert dismissals

Budgets are application rules stored in PostgreSQL; they are not AWS Budgets resources. The consuming application evaluates them against current cost and shows the notification UI.

Architecture and request flow

flowchart LR
  Host[External Express application] --> Auth[Authentication]
  Auth --> Enabled[Tenant plugin check]
  Enabled --> Router[/api/v1/cost]
  Router --> Controller[Cost controller]
  Controller --> Customer[Customer AWS context]
  Customer -->|Customer role| CE[AWS Cost Explorer]
  Controller --> Source{Data source}
  Source -->|CUR| SaaS[SaaS Athena context]
  SaaS --> Athena[Central tenant-partitioned CUR]
  Source -->|auto fallback| CE
  Controller --> Budget[Budget services]
  Budget --> DB[(PostgreSQL)]

src/
|-- controllers/             HTTP-to-service orchestration
|-- routes/                  Express routes and error handling
|-- models/                  Normalized immutable domain models
|-- repositories/            Tenant onboarding persistence access
|-- services/                AWS identities, Cost Explorer, CUR, budgets
|-- schema/                  Budget and dismissal table installation
|-- lib/                     Cost, date, filter, and SQL helpers
`-- plugin.js                Dependency composition and lifecycle

Local development and validation

Requirements:

• Node.js 20 or newer
• npm
• PostgreSQL available to the host used for integration testing

Install dependencies:

npm install

This package is native JavaScript ESM and has no transpilation build step. Validate syntax and package contents with:

node --check index.js
node --check cur.js
node --check src/plugin.js
npm pack --dry-run

When files below src change, syntax-check every changed .js file. The npm archive must include index.js, cur.js, the complete src directory, README.md, and AGENTS.md.

For a local package installation test:

npm pack
npm install /path/to/meyi-insight-cost-server-1.0.0.tgz

Publish to npm

Publishing changes the external registry. Run these commands only after release authorization:

npm login
npm whoami
npm version patch
npm pack --dry-run
npm publish --access public

Use npm version minor or npm version major when appropriate. This server package currently has no compile step or prepublishOnly script, so syntax and integration validation must be completed before publishing.

After publishing:

npm view meyi-insight-cost-server version
npm install meyi-insight-cost-server@<published-version>

Integrate with an external Express application

Install the package and its PostgreSQL peer dependency:

npm install meyi-insight-cost-server pg

For local development, use a file dependency:

{
  "dependencies": {
    "meyi-insight-cost-server": "file:../../meyi-market-places/insight-cost-server"
  }
}

Create the plugin using the host's Express app and Drizzle database:

import express from "express";
import { drizzle } from "drizzle-orm/node-postgres";
import { Pool } from "pg";
import { createInsightCost } from "meyi-insight-cost-server";

const app = express();
const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const db = drizzle(pool);

app.use(express.json());

// These host middlewares must be registered before plugin.start().
app.use("/api/v1/cost", authenticateRequest);
app.use("/api/v1/cost", requireCostPluginForTenant);

const costPlugin = createInsightCost({
  app,
  db,
  apiBaseUri: "/api/v1",
  logger: console,
});

await costPlugin.install();
await costPlugin.start();

Lifecycle behavior:

• install() creates or verifies plugin-owned budget and dismissal tables.
• start() mounts the router once at ${apiBaseUri}/cost.
• stop() is currently a no-op but is available for host lifecycle symmetry.
• router is also returned for hosts that need custom mounting.

app is optional only when the host mounts the returned router itself. db is required.

Host authentication and tenant contract

Before a request reaches the plugin, the host should authenticate it and set:

req.user = {
  id: "authenticated-user-id",
  tenant_id: "authenticated-tenant-id",
};

The tenant resolver checks req.user.tenant_id, then req.user.tenantId, then x-tenant-id, then DEFAULT_TENANT_ID. In production, prefer an authenticated req.user tenant and do not accept an untrusted tenant header.

The Marketplace onboarding integration loads tenant access from:

• <DB_SCHEMA>.aws_connections
• <DB_SCHEMA>.aws_accounts
• <DB_SCHEMA>.cost_cur_config
• <DB_SCHEMA>.cost_cur_discovery_jobs
• <DB_SCHEMA>.cost_cur_discovery_job_logs

Only a connected row containing Cost with verified Cost access is accepted. Connection metadata supplies the payer role and external ID; the dedicated Cost CUR table supplies the tenant partition and delivery/readiness state. The customer role is used for Cost Explorer and organization/account-name access only. It is never passed to the central Athena client.

CUR discovery runs as one shared scheduler with independent tenant job rows. Each job checks only its configured S3 prefix, finds a tenant-safe Glue table, and verifies the tenant partition through Athena. A successful job becomes READY and is no longer scheduled. Job events are retained separately in cost_cur_discovery_job_logs and also include tenant and job IDs in application logs.

Data-source behavior

Cost Explorer

Cost Explorer is the default fallback and returns gross positive unblended charges using the supported AWS record types. It supplies overview, trends, accounts, services, regions, tags, filters, forecasts, and report groupings.

Cost Explorer does not provide the exact active-resource total used by this plugin. That metric requires CUR resource IDs through Athena.

Central CUR through SaaS Athena

CUR applies line_item_unblended_cost > 0 for displayed gross cost and can also calculate net cost, credits/adjustments, active resources, detailed resource rows, account/region relationships, and allocation-tag data.

The SaaS runtime identity, or the dedicated role configured by COST_CUR_ROLE_ARN, queries the SaaS-owned Athena/Glue/S3 resources. Customer onboarding creates a CUR 2.0 Data Export directly into a tenant prefix in the Meyi-owned bucket; customer AWS credentials are not reused for Athena.

In auto mode, CUR is used when its database and Athena output location are configured. A CUR failure is logged and the request falls back to Cost Explorer. In cur mode, CUR is mandatory and failures are returned to the caller.

Environment variables

All variables below belong to the backend runtime. Never place AWS or database secrets in meyi-cost-ui, a browser environment, source control, or a Docker image layer. Supply them at container/process runtime.

Database and tenant variables

| Variable | Default | Required | Purpose | | --- | --- | --- | --- | | DB_SCHEMA | meyiconnect | No | PostgreSQL schema containing onboarding and plugin-owned tables. | | DEFAULT_TENANT_ID | default | No | Final tenant fallback when authenticated request context is absent. Prefer authenticated tenant context in production. | | DATABASE_URL | None | Host-specific | Not read by the package directly. The host commonly uses it to create the injected db connection. |

Cost Explorer and assume-role variables

| Variable | Default | Required | Purpose | | --- | --- | --- | --- | | COST_EXPLORER_ROLE_ARN | None | No | Fallback AWS role to assume when onboarding metadata has no payer/cross-account role. | | COST_EXPLORER_EXTERNAL_ID | None | No | External ID sent to STS for the fallback role. Onboarding externalId takes precedence. | | COST_EXPLORER_ALLOW_ENV_CREDENTIALS | false | Local testing only | Explicitly enables the process-wide AWS access-key fallback. Both access key and secret key must exist. | | COST_EXPLORER_PREFER_ENV_CREDENTIALS | false | No | When environment credentials are allowed and present, bypasses a configured role and uses those credentials. Useful only for controlled local testing. | | COST_EXPLORER_ALLOW_RUNTIME_IDENTITY | false | No | Explicitly lets Cost Explorer use the host runtime identity. Keep disabled in multi-tenant SaaS unless that identity is intentionally the customer identity. | | COST_EXPLORER_ACCOUNT_IDS | Empty | No | Comma-separated linked-account IDs used as local test accounts when onboarding returned none. | | AWS_ACCESS_KEY_ID | None | Local fallback | AWS SDK access key. Effective for this fallback only when COST_EXPLORER_ALLOW_ENV_CREDENTIALS=true. | | AWS_SECRET_ACCESS_KEY | None | Local fallback | AWS SDK secret key paired with AWS_ACCESS_KEY_ID. | | AWS_SESSION_TOKEN | None | Temporary credentials only | Session token required when the access key was issued by STS/SSO. |

ALLOW and PREFER are intentionally different:

• ALLOW=true permits environment credentials when no usable assumed role is selected.
• PREFER=true additionally tells the plugin to bypass an available role and use environment credentials instead. It has no effect unless ALLOW=true and both access-key variables are present.

Production tenants should use per-tenant onboarding role metadata. Environment credentials are process-wide and therefore unsuitable as the normal credential source for a multi-tenant service.

When customer environment credentials are enabled, central Athena is disabled unless COST_CUR_ROLE_ARN is also configured. This prevents the AWS SDK default credential chain from accidentally using a customer's local test key against SaaS Athena resources.

CUR/Athena variables

| Variable | Default | Required | Purpose | | --- | --- | --- | --- | | COST_DATA_SOURCE | auto | No | auto uses CUR when configured and falls back to Cost Explorer; cur requires CUR; cost-explorer disables CUR. | | COST_CUR_DATABASE | Empty | CUR | Athena/Glue database containing the CUR table. | | COST_CUR_TABLE | Empty | No | Optional manual Glue table override. When empty, the tenant discovery job supplies the verified table. | | COST_CUR_OUTPUT_LOCATION | Empty | CUR | S3 URI where Athena writes query results. | | COST_CUR_REGION | AWS_REGION or us-east-1 | No | Region for the Athena client. | | COST_CUR_WORKGROUP | meyi-cost | No | Meyi-owned Athena workgroup used for every query. | | AWS_REGION | us-east-1 for CUR fallback | No | Used as the CUR region fallback; Cost Explorer itself is created in us-east-1. | | COST_CUR_TENANT_COLUMN | tenant_id | No | CUR column used to isolate the Hive-style tenant partition. | | COST_CUR_TENANT_PARTITION | Request tenant ID | No | Overrides the partition value. Avoid a global override in multi-tenant production unless every request is intentionally mapped to that partition. | | COST_CUR_MAX_ROWS | 1000 | No | Resource-report row cap, clamped between 1 and 5000. | | COST_CUR_ROLE_ARN | Empty | No | Dedicated SaaS-side role assumed only for central Athena/Glue/S3 queries. | | COST_CUR_EXTERNAL_ID | Empty | No | External ID for the dedicated SaaS CUR role. | | COST_CUR_ALLOW_TENANT_CATALOG | false | No | Compatibility switch allowing tenant metadata to override the central catalog. Keep false for centralized SaaS CUR. | | COST_CUR_INGESTION_MODE | central | No | Describes the CUR ingestion contract returned by readiness status. | | COST_CUR_STATUS_CACHE_MS | 300000 | No | Cache duration for the live tenant CUR readiness query. | | COST_CUR_DISCOVERY_ENABLED | true | No | Enables tenant-specific background CUR discovery. | | COST_CUR_DISCOVERY_INTERVAL_MS | 1200000 | No | Retry interval for non-ready tenant jobs; minimum 60000 ms. | | COST_CUR_DISCOVERY_BATCH_SIZE | 25 | No | Maximum due tenant jobs selected per scheduler tick. |

Central catalog settings come from the SaaS backend environment. Only the tenant partition and customer source metadata come from onboarding by default. Legacy per-tenant catalog overrides are accepted only when COST_CUR_ALLOW_TENANT_CATALOG=true.

Example: production role-based Cost Explorer

The preferred production setup stores the payer/cross-account role and external ID during tenant onboarding. Only shared database settings may be required:

DB_SCHEMA=meyiconnect
DEFAULT_TENANT_ID=default
COST_DATA_SOURCE=cost-explorer

The host runtime identity must be able to call sts:AssumeRole, and the target role trust policy must trust that identity.

Example: controlled local testing before onboarding

DB_SCHEMA=meyiconnect
DEFAULT_TENANT_ID=default
COST_DATA_SOURCE=cost-explorer
COST_EXPLORER_ALLOW_ENV_CREDENTIALS=true
COST_EXPLORER_PREFER_ENV_CREDENTIALS=true
AWS_ACCESS_KEY_ID=REPLACE_ME
AWS_SECRET_ACCESS_KEY=REPLACE_ME
AWS_SESSION_TOKEN=REPLACE_IF_TEMPORARY
COST_EXPLORER_ACCOUNT_IDS=111111111111,222222222222

Omit COST_EXPLORER_ACCOUNT_IDS to query all linked accounts visible to the credential and derive the accessible accounts from Cost Explorer results.

Example: centralized CUR/Athena

COST_DATA_SOURCE=auto
COST_CUR_DATABASE=meyi_central_cur
COST_CUR_TABLE=
COST_CUR_OUTPUT_LOCATION=s3://meyi-saas-athena-results/
COST_CUR_REGION=us-east-1
COST_CUR_WORKGROUP=meyi-cost
COST_CUR_TENANT_COLUMN=tenant_id
COST_CUR_MAX_ROWS=1000
COST_CUR_ROLE_ARN=arn:aws:iam::SAAS_ACCOUNT_ID:role/meyi-cur-query
COST_CUR_ALLOW_TENANT_CATALOG=false
COST_CUR_INGESTION_MODE=central
COST_CUR_DISCOVERY_ENABLED=true
COST_CUR_DISCOVERY_INTERVAL_MS=1200000

Required AWS read-only access

Cost Explorer mode needs access to the operations used by the plugin, including:

• ce:GetCostAndUsage
• ce:GetCostForecast
• ce:GetDimensionValues
• ce:GetTags

Assume-role mode also needs sts:AssumeRole on the target role, plus a matching target-role trust policy and external ID when configured.

CUR mode additionally needs the SaaS identity to query Athena, read the Glue catalog and central CUR bucket, and write to the Athena results bucket. The customer onboarding role needs only the customer-side permissions required by the chosen CUR export/DataSync design. These policies must remain separate.

A payer/management-account role can expose organization-wide linked-account costs. A member-account role normally exposes only the costs AWS makes visible to that member. Separate member-account roles are not required merely to group organization cost by linked account when the payer role already has that data.

HTTP API

Routes are mounted below ${apiBaseUri}/cost:

| Method | Path | Purpose | | --- | --- | --- | | GET | /accounts | Selected or discovered AWS accounts. | | GET | /data-status | Central CUR configuration, ingestion, readiness, row count, and freshness without exposing secrets. | | GET | /overview | Totals, trends, accounts, services, regions, and active resources when CUR is available. | | GET | /filter-options | Dynamic report filter values. | | GET | /reports | Standard/comparison source data grouped by service, account, region, resource, or tag. | | GET | /tags | Available cost-allocation tags. | | GET | /budgets | Tenant budget rules. | | POST | /budgets | Create a tenant budget rule. | | DELETE | /budgets/:id | Delete a tenant budget rule. | | GET | /budget-alert-dismissals | Current user's persisted dismissals. | | POST | /budget-alert-dismissals | Dismiss one budget/month/status alert. |

Docker packaging

When embedding the local package in a Docker build, copy all published package files, not only index.js:

index.js
cur.js
src/
package.json
README.md
AGENTS.md

Environment files should not be copied into the image. Pass required variables with the deployment platform, Docker Compose env_file, or secret manager at runtime.

Reference Meyi Connect integration

The current host adapter is:

meyi-connect/backend/src/plugins/cost/index.mjs

It supplies the shared Drizzle database, authenticated token middleware, tenant-level plugin enablement, and plugin lifecycle wiring.

Consumer validation checklist

1. Run server syntax checks and npm pack --dry-run.
2. Install or refresh the package in the host backend.
3. Run install() against a disposable/test database.
4. Verify unauthenticated and disabled-tenant requests are rejected by the host.
5. Verify two tenants cannot access each other's accounts, budgets, or alert dismissals.
6. Test Cost Explorer, CUR auto fallback, and mandatory cur failure modes.
7. Verify payer and member-account visibility matches the AWS role used.
8. Build and start the consuming host backend before creating a Docker image.

Central CUR ingestion contract

The package implements the application side of the FinOps-style design: strict credential separation, tenant-partitioned Athena queries, Cost Explorer fallback, and live readiness/freshness reporting. The AWS data-transfer plane must deliver each customer's CUR objects into the central CUR layout consumed by Glue. That is deployment infrastructure, not an HTTP request made with a customer access key.

Onboarding may persist these non-secret fields in session meta:

{
  "curSourceBucket": "customer-cur-bucket",
  "curSourcePrefix": "reports/meyi/",
  "curSourceRegion": "us-east-1",
  "curTenantPartition": "authenticated-tenant-id",
  "curIngestionMode": "central"
}

The transfer pipeline must copy or replicate only that source prefix into the SaaS central location for the matching tenant partition. GET /data-status then verifies that Athena can see rows for that partition. Until rows arrive, COST_DATA_SOURCE=auto safely uses Cost Explorer, so cost data remains available while detailed CUR-only metrics such as active resources wait for ingestion.