Compare commits

...
151 Commits
Author SHA1 Message Date
Elie HabibandGitHub b8be2f6890 fix: sentry triage + SW POST method for PostHog ingest (#246)
- Fix PostHog /ingest 404: Workbox registerRoute defaults to GET only,
  PostHog sends POST. Add POST routes for /api/ and /ingest/.
- Fix fullscreen crash: optional chaining on exitFullscreen()?.catch()
  for browsers returning undefined instead of Promise.
- Add 6 noise filters: __firefox__, ifameElement.contentDocument,
  Invalid video id, Fetch is aborted, Stylesheet append timeout,
  Cannot assign to read only property.
- Widen Program failed to link filter (remove ": null" suffix).
2026-02-23 08:26:54 +00:00
Elie HabibandGitHub b06c8a17b3 fix: correct Vietnam flag country code in language selector (#245)
vi → vn (ISO 3166-1 alpha-2 for Vietnam). Also add explicit th mapping for Thailand.
2026-02-23 08:14:09 +00:00
Elie HabibandGitHub 8504d5649a fix: layer help, SW ingest routing, toggle colors, v2.5.5 (#244)
* feat: make intelligence alert popup opt-in via dropdown toggle

Auto-popup was interrupting users every 10s refresh cycle. Badge still
counts and pulses silently. New toggle in dropdown (default OFF) lets
users explicitly opt in to auto-popup behavior.

* chore: bump version to 2.5.5

## Changelog

### Features
- Intelligence alert popup is now opt-in (default OFF) — badge counts silently, toggle in dropdown to enable auto-popup

### Bug Fixes
- Linux: disable DMA-BUF renderer on WebKitGTK to prevent blank white screen (NVIDIA/immutable distros)
- Linux: add DejaVu Sans Mono + Liberation Mono font fallbacks for monospace rendering
- Consolidate monospace font stacks into --font-mono CSS variable (fixes undefined var bug)
- Reduce dedup coordinate rounding from 0.5° to 0.1° (~10km precision)
- Vercel build: handle missing previous deploy SHA
- Panel base class: add missing showRetrying method
- Vercel ignoreCommand shortened to fit 256-char limit

### Infrastructure
- Upstash Redis shared caching for all RPC handlers + cache key contamination fix
- Format Rust code and fix Windows focus handling

### Docs
- Community guidelines: contributing, code of conduct, security policy
- Updated .env.example

* chore: track Cargo.lock for reproducible Rust builds

* fix: update layer help popup with all current map layers

Added missing layers to the ? help popup across all 3 variants:
- Full: UCDP Events, Displacement, Spaceports, Cyber Threats, Fires,
  Climate Anomalies, Critical Minerals; renamed Shipping→Ship Traffic
- Tech: Tech Events, Cyber Threats, Fires
- Finance: GCC Investments

* docs: update README with crypto prices, analytics, typography, and dedup grid fix

* fix: add /ingest to service worker NetworkOnly routes

The SW was intercepting PostHog /ingest/* requests and returning
no-response (404) because no cache match existed. Adding NetworkOnly
ensures analytics requests pass through to Vercel's rewrite proxy.

* chore: update Cargo.lock for v2.5.5

* fix: use explicit colors for findings toggle switch visibility
2026-02-23 08:01:46 +00:00
d24d61f333 Fix Linux rendering issues and improve monospace font fallbacks (#243)
* fix: resolve AppImage blank white screen and font crash on Linux (#238)

Disable WebKitGTK DMA-BUF renderer by default on Linux to prevent blank
white screens caused by GPU buffer allocation failures (common with
NVIDIA drivers and immutable distros like Bazzite). Add Linux-native
monospace font fallbacks (DejaVu Sans Mono, Liberation Mono) to all font
stacks so WebKitGTK font resolution doesn't hit out-of-bounds vector
access when macOS-only fonts (SF Mono, Monaco) are unavailable.

https://claude.ai/code/session_01TF2NPgSSjgenmLT2XuR5b9

* fix: consolidate monospace font stacks into --font-mono variable

- Define --font-mono in :root (main.css) and .settings-shell (settings-window.css)
- Align font stack: SF Mono, Monaco, Cascadia Code, Fira Code, DejaVu Sans Mono, Liberation Mono
- Replace 3 hardcoded JetBrains Mono stacks with var(--font-mono)
- Replace 4 hardcoded settings-window stacks with var(--font-mono)
- Fix pre-existing bug: var(--font-mono) used in 4 places but never defined
- Match index.html skeleton font stack to --font-mono

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-23 07:16:48 +00:00
Sense_wangandGitHub 0f4420d9cc fix: reduce dedup coordinate rounding from 0.5° to 0.1° (~10km) (#235) 2026-02-23 06:39:53 +00:00
Elie HabibandGitHub 63a4c9ab9c feat: Upstash Redis shared caching + cache key contamination fixes (#232)
* fix(sentry): add noise filters for 5 non-actionable error patterns

Filter dynamic import alt phrasing, script parse errors, maplibre
style/WebGL crashes, and CustomEvent promise rejections. Also fix
beforeSend to catch short Firefox null messages like "E is null".

* fix: cache write race, settings stale key status, yahoo gate concurrency

P1: Replace async background thread cache write with synchronous fs::write
to prevent out-of-order writes and dirty flag cleared before persistence.

P2: Add WorldMonitorTab.refresh() called after loadDesktopSecrets() so
the API key badge reflects actual keychain state.

P3: Replace timestamp-based Yahoo gate with promise queue to ensure
sequential execution under concurrent callers.

* feat: add Upstash Redis shared caching to all RPC handlers + fix cache key contamination

- Add Redis L2 cache (getCachedJson/setCachedJson) to 28 RPC handlers
  across all service domains (market, conflict, cyber, economic, etc.)
- Fix 10 P1 cache key contamination bugs where under-specified keys
  caused cross-request data pollution (e.g. filtered requests returning
  unfiltered cached data)
- Restructure list-internet-outages to cache-then-filter pattern so
  country/timeRange filters always apply after cache read
- Add write_lock mutex to PersistentCache in main.rs to prevent
  desktop cache write-race conditions
- Document FMP (Financial Modeling Prep) as Yahoo Finance fallback TODO
  in market/v1/_shared.ts

* fix: cache-key contamination and PizzINT/GDELT partial-failure regression

- tech-events: fetch with limit=0 and cache full result, apply limit
  slice after cache read to prevent low-limit requests poisoning cache
- pizzint: restore try-catch around PizzINT fetch so GDELT tension
  pairs are still returned when PizzINT API is down

* fix: remove extra closing brace in pizzint try-catch

* fix: recompute conferenceCount/mappableCount after limit slice

* fix: bypass WM API key gate for registration endpoint

/api/register-interest must reach cloud without a WorldMonitor API key,
otherwise desktop users can never register (circular dependency).
2026-02-23 10:09:12 +04:00
8f8b0c843b Format Rust code and fix Windows focus handling (#242)
* chore: apply cargo fmt formatting to main.rs

Pure formatting normalization with no logic changes. Separated from
the behavioral fix to keep git blame clean.

https://claude.ai/code/session_01RPQ1PEqxTSEG6rB5XadzEz

* fix: restrict settings-window re-focus to macOS to avoid Windows focus churn

On Windows, the Focused(true) handler on the main window calls
show()+set_focus() on the settings window, which steals focus back,
retriggering the event in a tight loop and presenting as a UI hang.

Gate the match arm with #[cfg(target_os = "macos")] (compile-time
attribute) instead of cfg!() (runtime macro) to match the convention
used by the adjacent macOS-only handlers and eliminate dead code on
non-macOS builds entirely.

https://claude.ai/code/session_01RPQ1PEqxTSEG6rB5XadzEz

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-23 05:29:49 +00:00
Sebastien MelkiandGitHub fcd590a9e6 Update .env.example (#228) 2026-02-22 14:43:32 +00:00
3e935e3541 docs: add community guidelines (contributing, code of conduct, security) (#226)
* docs: add community guidelines — contributing guide, code of conduct, and security policy

Add three community health files for the open-source project:

- CONTRIBUTING.md: comprehensive guide covering architecture overview (sebuf,
  variants, directory structure), development setup with make commands,
  AI-assisted development policy, sebuf RPC workflow, data source and RSS
  feed contribution guides, coding standards, and PR process
- CODE_OF_CONDUCT.md: Contributor Covenant v2.1 adapted for World Monitor
- SECURITY.md: responsible disclosure policy, security considerations for
  edge functions/sebuf handlers, and contributor best practices

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing blank line before list in CONTRIBUTING.md (MD032)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: expand AI section with LLM label attribution and rationale

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: remove GitHub link from AI section

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: simplify AI section back to concise version with PR labels

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 16:26:13 +02:00
Sebastien MelkiandClaude Opus 4.6 5d7e77f8b0 fix: shorten vercel.json ignoreCommand to fit 256-char limit
Invert the path logic from an allowlist of watched directories to an
exclusion list (*.md, .planning, docs, e2e, scripts, .github). This
brings the command from 318 to 242 characters while keeping the same
build-trigger behavior for all source code changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 13:25:57 +02:00
Elie HabibandGitHub b23cac04b9 Fix Vercel build failure when previous deploy SHA is missing (#225) 2026-02-22 09:52:21 +00:00
Elie HabibandGitHub df2cdafadb fix: add missing showRetrying method to Panel base class (#224) 2026-02-22 09:39:50 +00:00
Elie Habib 84f17e6c4c chore: bump version to 2.5.4
## Changelog

### Bug Fixes
- market: Fix price falsy bug (price of 0 treated as null)
- market: Per-symbol-set caching prevents stock/commodity data leakage
- market: Yahoo request gate (600ms) reduces IP-level rate limiting
- market: ETF panel 8s delayed fetch reduces Yahoo contention on startup
- ucdp: Clear circuit breaker cache on empty responses
- ucdp: Retry loop (3 attempts, 15s) for cold start resilience
- ucdp: Negative cache, version cache, stale-on-error fallback
- analytics: Proxy PostHog through own domain to bypass ad blockers
- settings: Skip API key re-verification when no keys changed
- csp: Allow PostHog scripts from us-assets.i.posthog.com
- api: Sanitize og-story level input
- api: Restore API-key gate on config import failure

### Features
- Cable health scoring via sebuf InfrastructureService
- PostHog analytics with privacy-first design

### i18n
- Cable health evidence key added to all locales
2026-02-22 09:17:18 +00:00
Elie Habib 43cd5b3826 chore: add TODO for Yahoo Finance cloud relay fallback
When user has a WorldMonitor API key, enable cloud relay through
worldmonitor.app as fallback for Yahoo 429 rate limits.
2026-02-22 09:15:33 +00:00
Elie Habib a2bcfeede4 fix(market): harden Yahoo Finance resilience and UCDP retry logic
- Fix proto.price falsy bug: price of 0 was treated as null
- Replace global lastSuccessfulResults with per-symbol-set Map to prevent
  stock data leaking into commodity fallback
- Add yahooGate (600ms) to serialize Yahoo requests and avoid IP rate limits
- Add per-symbol-set cache key in server handler to isolate stock/commodity/sector calls
- Clear UCDP circuit breaker cache on empty responses to prevent 10-min lockout
- Add UCDP retry loop (3 attempts, 15s apart) on cold start
- Delay ETF panel initial fetch by 8s to reduce Yahoo contention on startup
2026-02-22 09:15:09 +00:00
Elie Habib 584159f35c fix(analytics): proxy PostHog through own domain to bypass ad blockers
- Add /ingest/* rewrites in vercel.json → us.i.posthog.com
- Web uses /ingest proxy, desktop uses direct PostHog host
- Enable capture_pageview so every visitor registers
2026-02-21 23:53:20 +00:00
Elie Habib 440e45a032 chore: add docs/internal/ to .gitignore 2026-02-21 23:27:56 +00:00
Elie Habib 8cc82def37 fix(settings): skip API key re-verification when no keys were changed
Plaintext keys (OLLAMA_API_URL, OLLAMA_MODEL, etc.) render pre-filled
with stored values. captureUnsavedInputs() was capturing these unchanged
values into pendingSecrets, triggering unnecessary verification on save.
Now compares against stored value and skips if unchanged.
2026-02-21 23:27:49 +00:00
Elie Habib 21d7c8b7f3 fix(ucdp): add negative cache, version cache, and stale-on-error fallback
Prevent timeout spam on Railway when UCDP API is down:
- Negative cache: 5-min backoff after upstream failure
- Version discovery cache: reuse discovered API version for 1 hour
- Parallel version probing via Promise.allSettled
- Stale-on-error: serve fallback data instead of empty array
2026-02-21 23:27:38 +00:00
Elie Habib 1a9dc02ecc fix(i18n): add cable health evidence key to all locales
PR #220 added popups.cable.health.evidence only to en.json.
Add translated values to all 15 other locales.
2026-02-21 23:27:21 +00:00
e6285e3de0 Add cable health scoring via sebuf InfrastructureService (#220)
* feat: add cable health scoring via sebuf InfrastructureService

Port submarine cable health monitoring from PR #134 to the sebuf
architecture. Adds GetCableHealth RPC to InfrastructureService that
analyzes NGA maritime warnings to detect cable faults and repair
activity, computing health scores with time-decay.

- Proto: GetCableHealthRequest/Response, CableHealthRecord, evidence
- Handler: NGA warning fetch, cable matching (name + proximity), signal
  processing, health computation with redis caching
- Client: circuit breaker, proto enum → frontend string adapter, 1-min cache
- Frontend: health-based cable coloring (fault=red, degraded=orange),
  evidence display in cable popup, SVG + DeckGL support

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for cable health scoring

- Fix geographic proximity: use cosine-latitude correction instead of
  raw Euclidean distance on lat/lon degrees, return distanceKm directly
- Fix signal kind: use 'cable_advisory' (not 'operator_fault') for
  non-fault NGA warnings so advisories don't trigger fault status
- Parallelize loadCableActivity + loadCableHealth with Promise.all
- Remove console.log from client-side cable-health service
- Add in-memory fallback cache on server so transient Redis+NGA
  failures serve stale data instead of empty response

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-22 03:22:05 +04:00
LawyeredandGitHub 4365626df3 fix(api): sanitize og-story level input (#219) 2026-02-22 03:19:01 +04:00
Elie Habib e1b9e9e8a9 fix(csp): allow PostHog scripts from us-assets.i.posthog.com 2026-02-21 17:06:53 +00:00
86eff555f0 fix: restore API-key gate and block fallback on config import failure (#218)
Restore the WORLDMONITOR_API_KEY check that was removed in 740e351,
which left desktop cloud fallback ungated — causing deterministic 401s
from the edge gateway for keyless desktop installs. Also disable cloud
fallback when the runtime-config module fails to import, since the
cloudFallback() path depends on the same module and would throw.

https://claude.ai/code/session_014yJsGsxD1sWt6B6PvQXiaA

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-21 16:49:17 +00:00
Elie HabibandGitHub 740e3513ef Fix deployment build: make WORLDMONITOR_API_KEY optional (#217) 2026-02-21 13:55:21 +00:00
Elie HabibandGitHub 1922a781cd Add PostHog analytics with privacy-first design (#216) 2026-02-21 13:37:36 +00:00
Elie Habib 8699dae5e5 fix: registration via direct Convex call + compact WM tab layout
- Sidecar calls Convex HTTP API directly (Vercel Attack Challenge Mode
  blocks server-side proxy). CONVEX_URL read from env, not hardcoded.
- Rust injects CONVEX_URL into sidecar via option_env! (CI) / env var (dev)
- GitHub Actions passes CONVEX_URL secret to all 4 build steps
- Tighten WM tab CSS spacing so all content fits in one viewport
2026-02-21 11:36:03 +00:00
Elie Habib 1fc6dc2dbb fix: World Monitor tab first, registration proxy, empty key guard
- Move World Monitor tab to first position in settings.html
- Add registration proxy in sidecar to bypass Vercel bot protection
- Fix sidecar RSS/registration handlers to use response.text()
- Skip empty values in loadDesktopSecrets (NO LICENSE vs LICENSED)
- Add skip-setup text to desktop config alert panel
2026-02-21 11:09:18 +00:00
Elie Habib 68e6a367d6 feat: redesign settings World Monitor tab + sidecar RSS proxy + v2.5.3
Rebuild the World Monitor settings tab with hero banner, license key
input, waitlist registration, and BYOK footer. Only validate API key
panels that have pending changes on save. Add local RSS proxy handler
to sidecar so desktop fetches feeds directly without cloud fallback.
Bump version to 2.5.3.
2026-02-21 11:01:01 +00:00
Elie HabibandGitHub a388afe400 feat: API key gating for desktop cloud fallback + registration (#215)
* feat: API key gating for desktop cloud fallback + registration system

Gate desktop cloud fallback behind WORLDMONITOR_API_KEY — desktop users
need a valid key for cloud access, otherwise operate local-only (sidecar).
Add email registration system via Convex DB for future key distribution.

Client-side: installRuntimeFetchPatch() checks key presence before
allowing cloud fallback, with secretsReady promise + 2s timeout.
Server-side: origin-aware validation in sebuf gateway — desktop origins
require key, web origins pass through.

- Add WORLDMONITOR_API_KEY to 3-place secret system (Rust, TS, sidecar)
- New "World Monitor" settings tab with key input + registration form
- New api/_api-key.js server-side validation (origin-aware)
- New api/register-interest.js edge function with rate limiting
- Convex DB schema + mutation for email registration storage
- CORS headers updated for X-WorldMonitor-Key + Authorization
- E2E tests for key gate (blocked without key, allowed with key)
- Deployment docs (API_KEY_DEPLOYMENT.md) + updated desktop config docs

* fix: harden worldmonitor key + registration input handling

* fix: show invalid WorldMonitor API key status

* fix: simplify key validation, trim registration checks, add env example vars

- Inline getValidKeys() in _api-key.js
- Remove redundant type checks in register-interest.js
- Simplify WorldMonitorTab status to present/missing
- Add WORLDMONITOR_VALID_KEYS and CONVEX_URL to .env.example

* feat(sidecar): integrate proto gateway bundle into desktop build

The sidecar's buildRouteTable() only discovers .js files, so the proto
gateway at api/[domain]/v1/[rpc].ts was invisible — all 45 sebuf RPCs
returned 404 in the desktop app. Wire the existing build script into
Tauri's build commands and add esbuild as an explicit devDependency.
2026-02-21 10:36:23 +00:00
Elie Habib 417b7d275d perf(rss-proxy): increase CDN cache TTL to reduce origin invocations
Bump s-maxage from 300s to 600s and stale-while-revalidate from 60s to
300s. Client already caches feeds for 10 min locally, so users see no
freshness difference while Vercel edge serves cached responses longer.
2026-02-21 09:02:17 +00:00
Elie Habib 8243147886 feat(feeds): add native-language feeds for th, vi, tr, pl, ru locales and Australian coverage
Add 15 locale-specific feeds (BBC Turkce, DW Turkish, Hurriyet, TVN24,
Polsat News, Rzeczpospolita, BBC Russian, Meduza, Novaya Gazeta Europe,
Bangkok Post, Thai PBS, VnExpress, Tuoi Tre News) and 2 Australian feeds
(ABC News Australia, Guardian Australia) to improve content for users on
non-English locales.
2026-02-21 08:08:44 +00:00
Elie Habib 5786a8f279 fix: remove dead /api/opensky fallback route (closes #212)
VITE_WS_RELAY_URL is always set in production. Without it, bail early
instead of hitting a non-existent Vercel route.
2026-02-21 08:07:16 +00:00
Elie Habib 7e86e26a9e fix(crypto): serve stale prices on CoinGecko failure instead of $0
Cache last successful crypto results and filter out price=0 quotes
(proto default). On transient CoinGecko errors, users see last-known
prices instead of misleading $0 +0.00% for all coins.
2026-02-21 07:47:43 +00:00
Elie Habib 81577cbcfa fix(feeds): replace dead RSS endpoints with Google News fallbacks
Haaretz cmlink RSS now returns HTML homepage (feed removed).
Arab News returns 403 to server-side requests (bot detection).
FAO GIEWS RSS returns 404→HTML (endpoint removed).
All three switched to Google News site: search as feed source.
2026-02-21 07:42:43 +00:00
Elie Habib d3da54bcb2 feat(crypto): add XRP (Ripple) to crypto panel 2026-02-21 07:38:18 +00:00
Elie Habib 519da36ef6 fix(relay): add in-flight request dedup for OpenSky and RSS endpoints
Multiple connected clients polling simultaneously caused stampede on upstream
APIs (53 concurrent RSS MISS, 5 concurrent OpenSky requests for same bbox).
Wraps upstream fetches in a Promise stored in an in-flight Map — concurrent
requests for the same key await the single in-flight fetch and get served
from cache with X-Cache: DEDUP.
2026-02-21 07:28:39 +00:00
Elie Habib 67259da470 feat(i18n): add Thai and Vietnamese localization
Add complete Thai (th) and Vietnamese (vi) locale files with 1361
translated keys each, matching en.json structure. Register both
languages in SUPPORTED_LANGUAGES, LANGUAGES selector, and getLocale map.
2026-02-21 07:24:52 +00:00
Elie Habib e1f392b45b security: escape user-facing strings in innerHTML template literals
Add escapeHtml() to ~40 interpolation sites across MapPopup renderers
(bases, waterways, APTs, nuclear facilities, economic centers, pipelines,
datacenters, cloud regions, tech HQs), LiveWebcamsPanel feed labels, and
App critical banner headline/summary. Prevents XSS if upstream data
contains HTML-special characters.
2026-02-21 07:13:53 +00:00
Elie Habib c1b8ffb890 fix: harden OpenSky auth flow and proxy fallback 2026-02-21 07:13:53 +00:00
Elie Habib d2490bb7b7 fix: sentry triage — noise filters and formatOilValue NaN guard
- Add 5 new ignoreErrors patterns: Chinese cancel text, stack overflow,
  fetchError, window.ethereum, SyntaxError unexpected token
- Expand existing patterns: Can't find variable adds NP, indexOf adds findIndex
- Guard formatOilValue against NaN/non-finite input with dash fallback
2026-02-21 07:13:53 +00:00
LawyeredandGitHub f082e09d55 fix(sidecar): preserve request body on cloud fallback (#209)
Cache inbound request body once so local handler dispatch and cloud fallback can both access the same payload. Adds regression coverage for POST fallback after a local non-OK response.
2026-02-21 04:10:53 +04:00
LawyeredandGitHub 3bf4244cde fix(sidecar): preserve Request body semantics in ipv4 shim (#205)
* fix(sidecar): keep Request-body semantics in ipv4 fetch shim

* fix(sidecar): throw when Request body is already consumed
2026-02-21 04:09:26 +04:00
4822a839eb perf: faster panel loading — instant flat render, higher concurrency (#206)
Three targeted changes to reduce perceived panel load time:

1. Show flat items immediately, cluster in background (NewsPanel.ts)
   - Panels now render a flat list as soon as feed data arrives
   - ML clustering + velocity enrichment runs in the background
   - When clustering completes, the panel upgrades to clustered view
   - Previously panels showed nothing until clustering finished (1-3s delay)

2. Increase finance variant category concurrency from 3 to 5 (App.ts)
   - The SWR cache means warm-start fetches return cached data instantly,
     so the API pressure concern is less relevant
   - With 14 categories at concurrency 3, there were 5 sequential waves
   - At concurrency 5, this drops to 3 waves (~10-15s faster on cold start)

3. Reduce render throttle from 250ms to 100ms (App.ts)
   - Virtual scrolling already bounds DOM impact for large lists
   - Faster throttle means partial batch results appear sooner

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-21 04:01:14 +04:00
Elie Habib 9273facad1 docs: expand README with proto-first API, cable health, OG images, and production hardening
Add new sections for proto-first API contracts (17 service domains,
code generation pipeline, edge gateway), undersea cable health monitoring
(NGA warnings + AIS cable ship tracking), dynamic OG image generation,
chunk reload guard, and storage quota management. Update tech stack,
architecture principles, security model, and roadmap. Add server/ and
proto/ to Vercel ignoreCommand watched paths.
2026-02-20 23:58:53 +00:00
Elie Habib 48f8e24353 release: v2.5.2 — quota guard, map race fixes, Vercel build skip fix 2026-02-20 23:46:34 +00:00
Elie Habib d07f14fef4 fix: prevent Vercel build skip when previous SHA is empty
Guard ignoreCommand against unset VERCEL_GIT_PREVIOUS_SHA (first deploy,
force deploy) which caused git diff to fail → build canceled. Also add
data/ to watched paths.
2026-02-20 23:45:33 +00:00
c939cc6296 Proto-first API rebuild: sebuf contracts, handlers, gateway, and generated docs (#106)
* docs: initialize sebuf integration project with codebase map

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add project config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: complete project research

* docs: define v1 requirements (34 requirements, 8 categories)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: create roadmap (8 phases)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(01): capture phase context

* docs(state): record phase 1 context session

* docs(01): research phase domain - buf toolchain, sebuf codegen, proto patterns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(01-proto-foundation): create phase plan

* chore(01-01): configure buf toolchain with buf.yaml, buf.gen.yaml, buf.lock

- buf.yaml v2 with STANDARD+COMMENTS lint, FILE+PACKAGE+WIRE_JSON breaking, deps on protovalidate and sebuf
- buf.gen.yaml configures protoc-gen-ts-client, protoc-gen-ts-server, protoc-gen-openapiv3 plugins
- buf.lock generated with resolved dependency versions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(01-01): add shared core proto type definitions

- geo.proto: GeoCoordinates with lat/lng validation, BoundingBox for spatial queries
- time.proto: TimeRange with google.protobuf.Timestamp start/end
- pagination.proto: cursor-based PaginationRequest (1-100 page_size) and PaginationResponse
- i18n.proto: LocalizableString for pre-localized upstream API strings
- identifiers.proto: typed ID wrappers (HotspotID, EventID, ProviderID) for cross-domain refs
- general_error.proto: GeneralError with RateLimited, UpstreamDown, GeoBlocked, MaintenanceMode

All files pass buf lint (STANDARD+COMMENTS) and buf build with zero errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(01-01): complete buf toolchain and core proto types plan

- SUMMARY.md documents 2 tasks, 9 files created, 2 deviations auto-fixed
- STATE.md updated: plan 1/2 in phase 1, decisions recorded
- ROADMAP.md updated: phase 01 in progress (1/2 plans)
- REQUIREMENTS.md updated: PROTO-01, PROTO-02, PROTO-03 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(01-01): use int64 epoch millis instead of google.protobuf.Timestamp

User preference: all time fields use int64 (Unix epoch milliseconds)
instead of google.protobuf.Timestamp for simpler serialization and
JS interop. Applied to TimeRange and MaintenanceMode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(01-02): create test domain proto files with core type imports

- Add test_item.proto with GeoCoordinates import and int64 timestamps
- Add get_test_items.proto with TimeRange and Pagination imports
- Add service.proto with HTTP annotations for TestService
- All proto files pass buf lint and buf build

* feat(01-02): run buf generate and create Makefile for code generation pipeline

- Add Makefile with generate, lint, clean, install, check, format, breaking targets
- Update buf.gen.yaml with managed mode and paths=source_relative for correct output paths
- Generate TypeScript client (TestServiceClient class) at src/generated/client/
- Generate TypeScript server (TestServiceHandler interface) at src/generated/server/
- Generate OpenAPI 3.1.0 specs (JSON + YAML) at docs/api/
- Core type imports (GeoCoordinates, TimeRange, Pagination) flow through to generated output

* docs(01-02): complete test domain code generation pipeline plan

- Create 01-02-SUMMARY.md with pipeline validation results
- Update STATE.md: phase 1 complete, 2/2 plans done, new decisions recorded
- Update ROADMAP.md: phase 1 marked complete (2/2)
- Update REQUIREMENTS.md: mark PROTO-04 and PROTO-05 complete

* docs(phase-01): complete phase execution and verification

* test(01): complete UAT - 6 passed, 0 issues

* feat(2A): define all 17 domain proto packages with generated clients, servers, and OpenAPI specs

Remove test domain protos (Phase 1 scaffolding). Add core enhancements
(severity.proto, country.proto, expanded identifiers.proto). Define all
17 domain services: seismology, wildfire, climate, conflict, displacement,
unrest, military, aviation, maritime, cyber, market, prediction, economic,
news, research, infrastructure, intelligence. 79 proto files producing
34 TypeScript files and 34 OpenAPI specs. buf lint clean, tsc clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2B): add server runtime phase context and handoff checkpoint

Prepare Phase 2B with full context file covering deliverables,
key reference files, generated code patterns, and constraints.
Update STATE.md with resume pointer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2B): research phase domain

* docs(2B): create phase plan

* feat(02-01): add shared server infrastructure (router, CORS, error mapper)

- router.ts: Map-based route matcher from RouteDescriptor[] arrays
- cors.ts: TypeScript port of api/_cors.js with POST/OPTIONS methods
- error-mapper.ts: onError callback handling ApiError, network, and unknown errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(02-01): implement seismology handler as first end-to-end proof

- Implements SeismologyServiceHandler from generated server types
- Fetches USGS M4.5+ earthquake GeoJSON feed and transforms to proto-shaped Earthquake[]
- Maps all fields: id, place, magnitude, depthKm, location, occurredAt (String), sourceUrl

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(02-01): complete server infrastructure plan

- SUMMARY.md with task commits, decisions, and self-check
- STATE.md updated: position, decisions, session info
- REQUIREMENTS.md: SERVER-01, SERVER-02, SERVER-06 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(02-02): create Vercel catch-all gateway, tsconfig.api.json, and typecheck:api script

- api/[[...path]].ts mounts seismology routes via catch-all with CORS on every response path
- tsconfig.api.json extends base config without vite/client types for edge runtime
- package.json adds typecheck:api script

* feat(02-02): add Vite dev server plugin for sebuf API routes

- sebufApiPlugin() intercepts /api/{domain}/v1/* in dev mode
- Uses dynamic imports to lazily load handler modules inside configureServer
- Converts Connect IncomingMessage to Web Standard Request
- CORS headers applied to all plugin responses (200, 204, 403, 404)
- Falls through to existing proxy rules for non-sebuf /api/* paths

* docs(02-02): complete gateway integration plan

- SUMMARY.md documenting catch-all gateway + Vite plugin implementation
- STATE.md updated: Phase 2B complete, decisions recorded
- ROADMAP.md updated: Phase 02 marked complete (2/2 plans)
- REQUIREMENTS.md: SERVER-03, SERVER-04, SERVER-05 marked complete

* docs(02-server-runtime): create gap closure plan for SERVER-05 Tauri sidecar

* feat(02-03): add esbuild compilation step for sebuf sidecar gateway bundle

- Create scripts/build-sidecar-sebuf.mjs that bundles api/[[...path]].ts into a single ESM .js file
- Add build:sidecar-sebuf npm script and chain it into the main build command
- Install esbuild as explicit devDependency
- Gitignore the compiled api/[[...path]].js build artifact

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(02-03): verify sidecar discovery and annotate SERVER-05 gap closure

- Confirm compiled bundle handler returns status 200 for POST requests
- Add gap closure note to SERVER-05 in REQUIREMENTS.md
- Verify typecheck:api and full build pipeline pass without regressions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(02-03): complete sidecar sebuf bundle plan

- Create 02-03-SUMMARY.md documenting esbuild bundle compilation
- Update STATE.md with plan 03 position, decisions, and metrics
- Update ROADMAP.md plan progress (3/3 plans complete)
- Annotate SERVER-05 gap closure in REQUIREMENTS.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-02): complete phase execution

* docs(2C): capture seismology migration phase context

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(state): record phase 2C context session

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2C): research seismology migration phase

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2C): create seismology migration phase plan

* feat(2C-01): annotate all int64 time fields with INT64_ENCODING_NUMBER

- Vendor sebuf/http/annotations.proto locally with Int64Encoding extension (50010)
- Remove buf.build/sebmelki/sebuf BSR dep, use local vendored proto instead
- Add INT64_ENCODING_NUMBER annotation to 34 time fields across 20 proto files
- Regenerate all TypeScript client and server code (time fields now `number` not `string`)
- Fix seismology handler: occurredAt returns number directly (no String() wrapper)
- All non-time int64 fields (displacement counts, population) left as string
- buf lint, buf generate, tsc, and sidecar build all pass with zero errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2C-01): complete INT64_ENCODING_NUMBER plan

- Create 2C-01-SUMMARY.md with execution results and deviations
- Update STATE.md: plan 01 complete, int64 blocker resolved, new decisions
- Update ROADMAP.md: mark 2C-01 plan complete
- Update REQUIREMENTS.md: mark CLIENT-01 complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(lint): exclude .planning/ from markdownlint

GSD planning docs use formatting that triggers MD032 -- these are
machine-generated and not user-facing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2C-02): rewrite earthquake adapter to use SeismologyServiceClient and adapt all consumers to proto types

- Replace legacy fetch/circuit-breaker adapter with port/adapter wrapping SeismologyServiceClient
- Update 7 consuming files to import Earthquake from @/services/earthquakes (the port)
- Adapt all field accesses: lat/lon -> location?.latitude/longitude, depth -> depthKm, time -> occurredAt, url -> sourceUrl
- Remove unused filterByTime from Map.ts (only called for earthquakes, replaced with inline filter)
- Update e2e test data to proto Earthquake shape

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(2C-02): delete legacy earthquake endpoint, remove Vite proxy, clean API_URLS config

- Delete api/earthquakes.js (legacy Vercel edge function proxying USGS)
- Remove /api/earthquake Vite dev proxy (sebufApiPlugin handles seismology now)
- Remove API_URLS.earthquakes entry from base config (no longer referenced)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2C-02): complete seismology client wiring plan

- Create 2C-02-SUMMARY.md with execution results
- Update STATE.md: phase 2C complete, decisions, metrics
- Update ROADMAP.md: mark 2C-02 and phase 2C complete
- Mark requirements CLIENT-02, CLIENT-04, CLEAN-01, CLEAN-02 complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2C): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2D): create wildfire migration phase plan

* feat(2D-01): enhance FireDetection proto and implement wildfire handler

- Add region (field 8) and day_night (field 9) to FireDetection proto
- Regenerate TypeScript client and server types
- Implement WildfireServiceHandler with NASA FIRMS CSV proxy
- Fetch all 9 monitored regions in parallel via Promise.allSettled
- Graceful degradation to empty list when API key is missing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2D-01): wire wildfire routes into gateway and rebuild sidecar

- Import createWildfireServiceRoutes and wildfireHandler in catch-all
- Mount wildfire routes alongside seismology in allRoutes array
- Rebuild sidecar-sebuf bundle with wildfire endpoint included

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2D-01): complete wildfire handler plan

- Create 2D-01-SUMMARY.md with execution results
- Update STATE.md position to 2D plan 01 complete
- Update ROADMAP.md with 2D progress (1/2 plans)
- Mark DOMAIN-01 requirement complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2D-02): create wildfires service module and rewire all consumers

- Add src/services/wildfires/index.ts with fetchAllFires, computeRegionStats, flattenFires, toMapFires
- Rewire App.ts to import from @/services/wildfires with proto field mappings
- Rewire SatelliteFiresPanel.ts to import FireRegionStats from @/services/wildfires
- Update signal-aggregator.ts source comment

* chore(2D-02): delete legacy wildfire endpoint and service module

- Remove api/firms-fires.js (replaced by api/server/worldmonitor/wildfire/v1/handler.ts)
- Remove src/services/firms-satellite.ts (replaced by src/services/wildfires/index.ts)
- Zero dangling references confirmed
- Full build passes (tsc, vite, sidecar)

* docs(2D-02): complete wildfire consumer wiring plan

- Create 2D-02-SUMMARY.md with execution results
- Update STATE.md: phase 2D complete, progress ~52%
- Update ROADMAP.md: phase 2D plan progress

* docs(phase-2D): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2E): research climate migration domain

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2E): create phase plan

* feat(2E-01): implement climate handler with 15-zone monitoring and baseline comparison

- Create ClimateServiceHandler with 15 hardcoded monitored zones matching legacy
- Parallel fetch from Open-Meteo Archive API via Promise.allSettled
- 30-day baseline comparison: last 7 days vs preceding baseline
- Null filtering with paired data points, minimum 14-point threshold
- Severity classification (normal/moderate/extreme) and type (warm/cold/wet/dry/mixed)
- 1-decimal rounding for tempDelta and precipDelta
- Proto ClimateAnomaly mapping with GeoCoordinates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2E-01): wire climate routes into gateway and rebuild sidecar

- Import createClimateServiceRoutes and climateHandler in catch-all gateway
- Mount climate routes alongside seismology and wildfire
- Rebuild sidecar-sebuf bundle with climate routes included

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2E-01): complete climate handler plan

- Create 2E-01-SUMMARY.md with execution results
- Update STATE.md: position to 2E plan 01, add decisions
- Update ROADMAP.md: mark 2E-01 complete, update progress table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2E-02): rewrite climate service module and rewire all consumers

- Replace src/services/climate.ts with src/services/climate/index.ts directory module
- Port/adapter pattern: ClimateServiceClient maps proto shapes to legacy consumer shapes
- Rewire ClimateAnomalyPanel, DeckGLMap, MapContainer, country-instability, conflict-impact
- All 6 consumers import ClimateAnomaly from @/services/climate instead of @/types
- Drop dead getSeverityColor function, keep getSeverityIcon and formatDelta
- Fix minSeverity required param in listClimateAnomalies call

* chore(2E-02): delete legacy climate endpoint and remove dead types

- Delete api/climate-anomalies.js (replaced by sebuf climate handler)
- Remove ClimateAnomaly and AnomalySeverity from src/types/index.ts
- Full build passes with zero errors

* docs(2E-02): complete climate client wiring plan

- Create 2E-02-SUMMARY.md with execution results
- Update STATE.md: phase 2E complete, decisions, session continuity
- Update ROADMAP.md: phase 2E progress

* docs(phase-2E): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2F): research prediction migration domain

* docs(2F): create prediction migration phase plans

* feat(2F-01): implement prediction handler with Gamma API proxy

- PredictionServiceHandler proxying Gamma API with 8s timeout
- Maps events/markets to proto PredictionMarket with 0-1 yesPrice scale
- Graceful degradation: returns empty markets on any failure (Cloudflare expected)
- Supports category-based events endpoint and default markets endpoint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2F-01): wire prediction routes into gateway

- Import createPredictionServiceRoutes and predictionHandler
- Mount prediction routes in allRoutes alongside seismology, wildfire, climate
- Sidecar bundle rebuilt successfully (21.2 KB)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2F-01): complete prediction handler plan

- SUMMARY.md with handler implementation details and deviation log
- STATE.md updated to 2F in-progress position with decisions
- ROADMAP.md updated to 1/2 plans complete for phase 2F
- REQUIREMENTS.md marked DOMAIN-02 complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2F-02): create prediction service module and rewire all consumers

- Create src/services/prediction/index.ts preserving all business logic from polymarket.ts
- Replace strategy 4 (Vercel edge) with PredictionServiceClient in polyFetch
- Update barrel export from polymarket to prediction in services/index.ts
- Rewire 7 consumers to import PredictionMarket from @/services/prediction
- Fix 3 yesPrice bugs: CountryIntelModal (*100), App.ts search (*100), App.ts snapshot (1-y)
- Drop dead code getPolymarketStatus()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(2F-02): delete legacy endpoint and remove dead types

- Delete api/polymarket.js (replaced by sebuf handler)
- Delete src/services/polymarket.ts (replaced by src/services/prediction/index.ts)
- Remove PredictionMarket interface from src/types/index.ts (now in prediction module)
- Type check and sidecar build both pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2F-02): complete prediction consumer wiring plan

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2F): complete phase execution

* docs(phase-2F): fix roadmap plan counts and completion status

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2G): research displacement migration phase

* docs(2G): create displacement migration phase plans

* feat(2G-01): implement displacement handler with UNHCR API pagination and aggregation

- 40-entry COUNTRY_CENTROIDS map for geographic coordinates
- UNHCR Population API pagination (10,000/page, 25-page guard)
- Year fallback: current year to current-2 until data found
- Per-country origin + asylum aggregation with unified merge
- Global totals computation across all raw records
- Flow corridor building sorted by refugees, capped by flowLimit
- All int64 fields returned as String() per proto types
- Graceful empty response on any failure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2G-01): wire displacement routes into gateway and rebuild sidecar

- Import createDisplacementServiceRoutes and displacementHandler
- Mount displacement routes alongside seismology, wildfire, climate, prediction
- Sidecar bundle rebuilt with displacement included (31.0 KB)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2G-01): complete displacement handler plan

- SUMMARY.md with execution metrics and decisions
- STATE.md updated to 2G phase position
- ROADMAP.md updated with 2G-01 plan progress

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2G-02): create displacement service module and rewire all consumers

- Create src/services/displacement/index.ts as port/adapter using DisplacementServiceClient
- Map proto int64 strings to numbers and GeoCoordinates to flat lat/lon
- Preserve circuit breaker, presentation helpers (getDisplacementColor, formatPopulation, etc.)
- Rewire App.ts, DisplacementPanel, MapContainer, DeckGLMap, conflict-impact, country-instability
- Delete legacy src/services/unhcr.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(2G-02): delete legacy endpoint and remove dead displacement types

- Delete api/unhcr-population.js (replaced by displacement handler from 2G-01)
- Remove DisplacementFlow, CountryDisplacement, UnhcrSummary from src/types/index.ts
- All consumers now import from @/services/displacement
- Sidecar rebuild, tsc, and full Vite build pass clean

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2G-02): complete displacement consumer wiring plan

- SUMMARY.md with 2 task commits, decisions, deviation documentation
- STATE.md updated: phase 2G complete, 02/02 plans done
- ROADMAP.md updated with plan progress

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2G): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2H): research aviation migration phase

* docs(2H): create aviation migration phase plans

* feat(2H-01): implement aviation handler with FAA XML parsing and simulated delays

- Install fast-xml-parser for server-side XML parsing (edge-compatible)
- Create AviationServiceHandler with FAA NASSTATUS XML fetch and parse
- Enrich US airports with MONITORED_AIRPORTS metadata (lat, lon, name, icao)
- Generate simulated delays for non-US airports with rush-hour weighting
- Map short-form strings to proto enums (FlightDelayType, FlightDelaySeverity, etc.)
- Wrap flat lat/lon into GeoCoordinates for proto response
- Graceful empty alerts on any upstream failure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2H-01): wire aviation routes into gateway and rebuild sidecar

- Mount createAviationServiceRoutes in catch-all gateway alongside 5 existing domains
- Import aviationHandler for route wiring
- Rebuild sidecar-sebuf bundle with aviation routes included

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2H-01): complete aviation handler plan

- Create 2H-01-SUMMARY.md with execution results
- Update STATE.md position to 2H-01 with aviation decisions
- Update ROADMAP.md progress for phase 2H (1/2 plans)
- Mark DOMAIN-08 requirement as complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2H-02): create aviation service module and rewire all consumers

- Create src/services/aviation/index.ts as port/adapter wrapping AviationServiceClient
- Map proto enum strings to short-form (severity, delayType, region, source)
- Unwrap GeoCoordinates to flat lat/lon, convert epoch-ms updatedAt to Date
- Preserve circuit breaker with identical name string
- Rewire Map, DeckGLMap, MapContainer, MapPopup, map-harness to import from @/services/aviation
- Update barrel export: flights -> aviation
- Delete legacy src/services/flights.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(2H-02): delete legacy endpoint and remove dead aviation types

- Delete api/faa-status.js (replaced by aviation handler in 2H-01)
- Remove FlightDelaySource, FlightDelaySeverity, FlightDelayType, AirportRegion, AirportDelayAlert from src/types/index.ts
- Preserve MonitoredAirport with inlined region type union
- Full build (tsc + vite + sidecar) passes with zero errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2H-02): complete aviation consumer wiring plan

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2H): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2I): research phase domain

* docs(2I): create phase plan

* feat(2I-01): implement ResearchServiceHandler with 3 RPCs

- arXiv XML parsing with fast-xml-parser (ignoreAttributes: false for attributes)
- GitHub trending repos with primary + fallback API URLs
- Hacker News Firebase API with 2-step fetch and bounded concurrency (10)
- All RPCs return empty arrays on failure (graceful degradation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2I-01): mount research routes in gateway and rebuild sidecar

- Import createResearchServiceRoutes and researchHandler in catch-all gateway
- Add research routes to allRoutes array (after aviation)
- Sidecar bundle rebuilt (116.6 KB)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2I-01): complete research handler plan

- SUMMARY.md with self-check passed
- STATE.md updated to phase 2I, plan 01 of 02
- ROADMAP.md updated with plan 2I-01 complete
- REQUIREMENTS.md: DOMAIN-05 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2I-02): create research service module and delete legacy code

- Add src/services/research/index.ts with fetchArxivPapers, fetchTrendingRepos, fetchHackernewsItems backed by ResearchServiceClient
- Re-export proto types ArxivPaper, GithubRepo, HackernewsItem (no enum mapping needed)
- Circuit breakers wrap all 3 client calls with empty-array fallback
- Delete legacy API endpoints: api/arxiv.js, api/github-trending.js, api/hackernews.js
- Delete legacy service files: src/services/arxiv.ts, src/services/github-trending.ts, src/services/hackernews.ts
- Remove arxiv, githubTrending, hackernews entries from API_URLS and REFRESH_INTERVALS in config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2I-02): complete research consumer wiring plan

- SUMMARY.md documenting service module creation and 6 legacy file deletions
- STATE.md updated: phase 2I complete, decisions recorded
- ROADMAP.md updated: phase 2I marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2I): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2J): complete unrest migration research

* docs(2J): create unrest migration phase plans

* feat(2J-01): implement UnrestServiceHandler with ACLED + GDELT dual-fetch

- Create handler with listUnrestEvents RPC proxying ACLED API and GDELT GEO API
- ACLED fetch uses Bearer auth from ACLED_ACCESS_TOKEN env var, returns empty on missing token
- GDELT fetch returns GeoJSON protest events with no auth required
- Deduplication uses 0.5-degree grid + date key, preferring ACLED over GDELT on collision
- Severity classification and event type mapping ported from legacy protests.ts
- Sort by severity (high first) then recency (newest first)
- Graceful degradation: returns empty events on any upstream failure

* feat(2J-01): mount unrest routes in gateway and rebuild sidecar

- Import createUnrestServiceRoutes and unrestHandler in catch-all gateway
- Add unrest service routes to allRoutes array
- Sidecar bundle rebuilt to include unrest endpoint
- RPC routable at POST /api/unrest/v1/list-unrest-events

* docs(2J-01): complete unrest handler plan

- Create 2J-01-SUMMARY.md with execution results and self-check
- Update STATE.md with phase 2J position, decisions, session continuity
- Update ROADMAP.md with plan 01 completion status

* feat(2J-02): create unrest service module with proto-to-legacy type mapping

- Full adapter maps proto UnrestEvent to legacy SocialUnrestEvent shape
- 4 enum mappers: severity, eventType, sourceType, confidence
- fetchProtestEvents returns ProtestData with events, byCountry, highSeverityCount, sources
- getProtestStatus infers ACLED configuration from response event sources
- Circuit breaker wraps client call with empty fallback

* feat(2J-02): update services barrel, remove vite proxies, delete legacy files

- Services barrel: protests -> unrest re-export
- Vite proxy entries removed: /api/acled, /api/gdelt-geo
- Legacy files deleted: api/acled.js, api/gdelt-geo.js, src/services/protests.ts
- Preserved: api/acled-conflict.js (conflict domain), SocialUnrestEvent type

* docs(2J-02): complete unrest service module plan

- SUMMARY.md created with full adapter pattern documentation
- STATE.md updated: 2J-02 complete, decisions recorded
- ROADMAP.md updated: Phase 2J marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2J): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2K): complete conflict migration research

* docs(2K): create phase plan

* feat(2K-01): implement ConflictServiceHandler with 3 RPCs

- listAcledEvents proxies ACLED API for battles/explosions/violence with Bearer auth
- listUcdpEvents discovers UCDP GED API version dynamically, fetches backward with 365-day trailing window
- getHumanitarianSummary proxies HAPI API with ISO-2 to ISO-3 country mapping
- All RPCs have graceful degradation returning empty on failure

* feat(2K-01): mount conflict routes in gateway and rebuild sidecar

- Add createConflictServiceRoutes and conflictHandler imports to catch-all gateway
- Spread conflict routes into allRoutes array (3 RPC endpoints)
- Rebuild sidecar bundle with conflict endpoints included

* docs(2K-01): complete conflict handler plan

- Create 2K-01-SUMMARY.md with execution details and self-check
- Update STATE.md: position to 2K-01, add 5 decisions
- Update ROADMAP.md: mark 2K-01 complete (1/2 plans done)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2K-02): create conflict service module with 4-shape proto-to-legacy type mapping

- Port/adapter mapping AcledConflictEvent -> ConflictEvent, UcdpViolenceEvent -> UcdpGeoEvent, HumanitarianCountrySummary -> HapiConflictSummary
- UCDP classifications derived heuristically from GED events (deaths/events thresholds -> war/minor/none)
- deduplicateAgainstAcled ported exactly with haversine + date + fatality matching
- 3 circuit breakers for 3 RPCs, exports 5 functions + 2 group helpers + all legacy types

* feat(2K-02): rewire consumer imports and delete 9 legacy conflict files

- App.ts consolidated from 4 direct imports to single @/services/conflict import
- country-instability.ts consolidated from 3 type imports to single ./conflict import
- Deleted 4 API endpoints: acled-conflict.js, ucdp-events.js, ucdp.js, hapi.js
- Deleted 4 service files: conflicts.ts, ucdp.ts, ucdp-events.ts, hapi.ts
- Deleted 1 dead code file: conflict-impact.ts
- UcdpGeoEvent preserved in src/types/index.ts (scope guard for map components)

* docs(2K-02): complete conflict service module plan

- SUMMARY.md with 4-shape proto adapter, consumer consolidation, 9 legacy deletions
- STATE.md updated: Phase 2K complete (2/2 plans), progress ~100%
- ROADMAP.md updated: Phase 2K marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2K): complete conflict migration phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2L): research maritime migration phase domain

* docs(2L): create maritime migration phase plans

* feat(2L-01): implement MaritimeServiceHandler with 2 RPCs

- getVesselSnapshot proxies WS relay with wss->https URL conversion
- Maps density/disruptions to proto shape with GeoCoordinates nesting
- Disruption type/severity mapped from lowercase to proto enums
- listNavigationalWarnings proxies NGA MSI broadcast warnings API
- NGA military date parsing (081653Z MAY 2024) to epoch ms
- Both RPCs gracefully degrade to empty on upstream failure
- No caching (client-side polling manages refresh intervals)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2L-01): mount maritime routes in gateway and rebuild sidecar

- Import createMaritimeServiceRoutes and maritimeHandler
- Add maritime routes to allRoutes array in catch-all gateway
- Sidecar bundle rebuilt (148.0 KB) with maritime endpoints
- RPCs routable at /api/maritime/v1/get-vessel-snapshot and /api/maritime/v1/list-navigational-warnings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2L-01): complete maritime handler plan

- SUMMARY.md with 2 task commits documented
- STATE.md updated to 2L phase, plan 01/02 complete
- ROADMAP.md progress updated for phase 2L
- REQUIREMENTS.md: DOMAIN-06 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2L-02): create maritime service module with hybrid fetch and polling/callback preservation

- Port/adapter wrapping MaritimeServiceClient for proto RPC path
- Full polling/callback architecture preserved from legacy ais.ts
- Hybrid fetch: proto RPC for snapshot-only, raw WS relay for candidates
- Proto-to-legacy type mapping for AisDisruptionEvent and AisDensityZone
- Exports fetchAisSignals, initAisStream, disconnectAisStream, getAisStatus, isAisConfigured, registerAisCallback, unregisterAisCallback, AisPositionData

* feat(2L-02): rewire consumer imports and delete 3 legacy maritime files

- cable-activity.ts: fetch NGA warnings via MaritimeServiceClient.listNavigationalWarnings() with NgaWarning shape reconstruction from proto fields
- military-vessels.ts: imports updated from './ais' to './maritime'
- Services barrel: updated from './ais' to './maritime'
- desktop-readiness.ts: service/api references updated to maritime handler paths
- Deleted: api/ais-snapshot.js, api/nga-warnings.js, src/services/ais.ts
- AisDisruptionEvent/AisDensityZone/AisDisruptionType preserved in src/types/index.ts

* docs(2L-02): complete maritime service module plan

- SUMMARY.md with hybrid fetch pattern, polling/callback preservation, 3 legacy files deleted
- STATE.md updated: phase 2L complete, 5 decisions recorded
- ROADMAP.md updated: 2L plans marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: bind globalThis.fetch in all sebuf service clients

Generated sebuf clients store globalThis.fetch as a class property,
then call it as this.fetchFn(). This loses the window binding and
throws "Illegal invocation" in browsers. Pass { fetch: fetch.bind(globalThis) }
to all 11 client constructors.

Also includes vite.config.ts with all 10 migrated domain handlers
registered in the sebuf dev server plugin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate cyber + economic domains to sebuf (12/17)

Cyber (Phase 2M):
- Create handler aggregating 5 upstream sources (Feodo, URLhaus, C2Intel, OTX, AbuseIPDB)
  with dedup, GeoIP hydration, country centroid fallback
- Create service module with CyberServiceClient + circuit breaker
- Delete api/cyber-threats.js, api/cyber-threats.test.mjs, src/services/cyber-threats.ts

Economic (Phase 2N) — consolidates 3 legacy services:
- Create handler with 3 RPCs: getFredSeries (FRED API), listWorldBankIndicators
  (World Bank API), getEnergyPrices (EIA API)
- Create unified service module replacing fred.ts, oil-analytics.ts, worldbank.ts
- Preserve all exported functions/types for EconomicPanel and TechReadinessPanel
- Delete api/fred-data.js, api/worldbank.js, src/services/fred.ts,
  src/services/oil-analytics.ts, src/services/worldbank.ts

Both domains registered in vite.config.ts and api/[[...path]].ts.
TypeScript check and vite build pass cleanly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate infrastructure domain to sebuf (13/17)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate market domain to sebuf (14/17)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate news domain to sebuf (15/17)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate intelligence domain to sebuf (16/17)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate military domain to sebuf (17/17)

All 17 domains now have sebuf handlers registered in the gateway.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate intelligence services to sebuf client

Rewire pizzint.ts, cached-risk-scores.ts, and threat-classifier.ts
to use IntelligenceServiceClient instead of legacy /api/ fetch calls.
Handler now preserves raw threat level in subcategory field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate military theater posture to sebuf client

Rewire cached-theater-posture.ts to use MilitaryServiceClient instead
of legacy /api/theater-posture fetch. Adds theater metadata map for
proto→legacy TheaterPostureSummary adapter. UI gracefully falls back
to total counts when per-type breakdowns aren't available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: rewire country-intel to sebuf client

Replace legacy fetch('/api/country-intel') with typed
IntelligenceServiceClient.getCountryIntelBrief() RPC call in App.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate stablecoin-markets to sebuf (market domain)

Add ListStablecoinMarkets RPC to market service. Port CoinGecko
stablecoin peg-health logic from api/stablecoin-markets.js into
the market handler. Rewire StablecoinPanel to use typed sebuf client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate etf-flows to sebuf (market domain)

Add ListEtfFlows RPC to market service. Port Yahoo Finance BTC spot
ETF flow estimation logic from api/etf-flows.js into the market handler.
Rewire ETFFlowsPanel to use typed sebuf client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate worldpop-exposure to sebuf (displacement domain)

Add GetPopulationExposure RPC to displacement service. Port country
population data and radius-based exposure estimation from
api/worldpop-exposure.js into the displacement handler. Rewire
population-exposure.ts to use typed sebuf client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove superseded legacy edge functions

Delete 4 legacy api/*.js files that are now fully replaced by
sebuf handlers:
- api/stablecoin-markets.js -> market/ListStablecoinMarkets
- api/etf-flows.js -> market/ListEtfFlows
- api/worldpop-exposure.js -> displacement/GetPopulationExposure
- api/classify-batch.js -> intelligence/ClassifyEvent

Remaining legacy files are still actively used by client code
(stock-index, opensky, gdelt-doc, rss-proxy, summarize endpoints,
macro-signals, tech-events) or are shared utilities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: delete dead legacy files and unused API_URLS config

Remove coingecko.js, debug-env.js, cache-telemetry.js, _cache-telemetry.js
(all zero active consumers). Delete unused API_URLS export from base config.
Update desktop-readiness market-panel metadata to reference sebuf paths.
Remove dead CoinGecko dev proxy from vite.config.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate stock-index and opensky to sebuf

- Add GetCountryStockIndex RPC to market domain (Yahoo Finance + cache)
- Fill ListMilitaryFlights stub in military handler (OpenSky with bounding box)
- Rewire App.ts stock-index fetch to MarketServiceClient.getCountryStockIndex()
- Delete api/stock-index.js and api/opensky.js edge functions
- OpenSky client path unchanged (relay primary, vite proxy for dev)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* wip: sebuf legacy migration paused at phase 3/10

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(03): capture phase context

* docs(state): record phase 3 context session

* docs(03): research phase domain

* docs(03): create phase plan — 5 plans in 2 waves

* feat(03-01): commit wingbits migration (step 3) -- 3 RPCs added to military domain

- Add GetAircraftDetails, GetAircraftDetailsBatch, GetWingbitsStatus RPCs
- Rewire src/services/wingbits.ts to use MilitaryServiceClient
- Update desktop-readiness.ts routes to match new RPC paths
- Delete legacy api/wingbits/ edge functions (3 files)
- Regenerate military service client/server TypeScript + OpenAPI docs

* feat(03-02): add SummarizeArticle proto and implement handler

- Create summarize_article.proto with request/response messages
- Add SummarizeArticle RPC to NewsService proto
- Implement full handler with provider dispatch (ollama/groq/openrouter)
- Port cache key builder, deduplication, prompt builder, think-token stripping
- Inline Upstash Redis helpers for edge-compatible caching

* feat(03-01): migrate gdelt-doc to intelligence RPC + delete _ip-rate-limit.js

- Add SearchGdeltDocuments RPC to IntelligenceService proto
- Implement searchGdeltDocuments handler (port from api/gdelt-doc.js)
- Rewire src/services/gdelt-intel.ts to use IntelligenceServiceClient
- Delete legacy api/gdelt-doc.js edge function
- Delete dead api/_ip-rate-limit.js (zero importers)
- Regenerate intelligence service client/server TypeScript + OpenAPI docs

* feat(03-02): rewire summarization client to NewsService RPC, delete 4 legacy files

- Replace direct fetch to /api/{provider}-summarize with NewsServiceClient.summarizeArticle()
- Preserve identical fallback chain: ollama -> groq -> openrouter -> browser T5
- Delete api/groq-summarize.js, api/ollama-summarize.js, api/openrouter-summarize.js
- Delete api/_summarize-handler.js and api/_summarize-handler.test.mjs
- Update desktop-readiness.ts to reference new sebuf route

* feat(03-03): rewire MacroSignalsPanel to EconomicServiceClient + delete legacy

- Replace fetch('/api/macro-signals') with EconomicServiceClient.getMacroSignals()
- Add mapProtoToData() to convert proto optional fields to null for rendering
- Delete legacy api/macro-signals.js edge function

* feat(03-04): add ListTechEvents proto, city-coords data, and handler

- Create list_tech_events.proto with TechEvent, TechEventCoords messages
- Add ListTechEvents RPC to ResearchService proto
- Extract 360-city geocoding table to api/data/city-coords.ts
- Implement listTechEvents handler with ICS+RSS parsing, curated events, dedup, filtering
- Regenerate TypeScript client/server from proto

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(03-01): complete wingbits + GDELT doc migration plan

- Create 03-01-SUMMARY.md with execution results
- Update STATE.md with plan 01 completion, steps 3-4 done
- Update ROADMAP.md plan progress (2/5 plans complete)
- Mark DOMAIN-10 requirement complete

* docs(03-02): complete summarization migration plan

- Create 03-02-SUMMARY.md with execution results
- Update STATE.md position to step 6/10
- Update ROADMAP.md plan progress
- Mark DOMAIN-09 requirement complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(03-04): rewire TechEventsPanel and App to ResearchServiceClient, delete legacy

- Replace fetch('/api/tech-events') with ResearchServiceClient.listTechEvents() in TechEventsPanel
- Replace fetch('/api/tech-events') with ResearchServiceClient.listTechEvents() in App.loadTechEvents()
- Delete legacy api/tech-events.js (737 lines)
- TypeScript compiles cleanly with no references to legacy endpoint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(03-03): complete macro-signals migration plan

- Create 03-03-SUMMARY.md with execution results
- Mark DOMAIN-04 requirement complete in REQUIREMENTS.md

* docs(03-04): complete tech-events migration plan

- Add 03-04-SUMMARY.md with execution results
- Update STATE.md: advance to plan 5/step 8, add decisions
- Update ROADMAP.md: 4/5 plans complete for phase 03

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(03-05): add temporal baseline protos + handler with Welford's algorithm

- GetTemporalBaseline RPC: anomaly detection with z-score thresholds
- RecordBaselineSnapshot RPC: batch update via Welford's online algorithm
- Inline mgetJson helper for Redis batch reads
- Inline getCachedJson/setCachedJson Redis helpers
- Generated TypeScript client/server + OpenAPI docs

* feat(03-05): migrate temporal-baseline + tag non-JSON + final cleanup

- Rewire temporal-baseline.ts to InfrastructureServiceClient RPCs
- Delete api/temporal-baseline.js (migrated to sebuf handler)
- Delete api/_upstash-cache.js (no importers remain)
- Tag 6 non-JSON edge functions with // Non-sebuf: comment header
- Update desktop-readiness.ts: fix stale cloudflare-outages reference

* docs(03-05): complete temporal-baseline + non-JSON tagging + final cleanup plan

- SUMMARY.md with Welford algorithm migration details
- STATE.md updated: Phase 3 complete (100%)
- ROADMAP.md updated: 5/5 plans complete

* chore(03): delete orphaned ollama-summarize test after RPC migration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-3): complete phase execution

* docs(v1): create milestone audit report

Audits all 13 phases of the v1 sebuf integration milestone.
12/13 phases verified (2L maritime missing VERIFICATION.md).
25/34 requirements satisfied, 6 superseded, 2 partial, 1 unsatisfied (CLEAN-03).
All 17 domains wired end-to-end. Integration check passes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(v1): update audit — mark CLEAN-03/04 + MIGRATE-* as superseded

CLEAN-03 superseded by port/adapter architecture (internal types
intentionally decoupled from proto wire types). MIGRATE-01-05
superseded by direct cutover approach. DOMAIN-03 checkbox updated.
Milestone status: tech_debt (no unsatisfied requirements).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(roadmap): add gap closure phase 4 — v1 milestone cleanup

Closes all audit gaps: CLIENT-03 circuit breaker coverage, DOMAIN-03/06
verification gaps, documentation staleness, orphaned code cleanup.
Fixes traceability table phase assignments to match actual roadmap phases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(03): commit generated NewsService OpenAPI specs + checkpoint update

SummarizeArticle RPC was added during Phase 3 plan 02 but generated
OpenAPI specs were not staged with that commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(04): research phase domain

* docs(04): create phase plan

* docs(04-01): fix ROADMAP.md Phase 3 staleness and delete .continue-here.md

- Change Phase 3 heading from IN PROGRESS to COMPLETE
- Check off plans 03-03, 03-04, 03-05 (all were already complete)
- Delete stale .continue-here.md (showed task 3/10 in_progress but all 10 done)

* feat(04-02): add circuit breakers to seismology, wildfire, climate, maritime

- Seismology: wrap listEarthquakes in breaker.execute with empty-array fallback
- Wildfire: replace manual try/catch with breaker.execute for listFireDetections
- Climate: replace manual try/catch with breaker.execute for listClimateAnomalies
- Maritime: wrap proto getVesselSnapshot RPC in snapshotBreaker.execute, preserve raw relay fallback

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(04-02): add circuit breakers to news summarization and GDELT intelligence

- Summarization: wrap newsClient.summarizeArticle in summaryBreaker.execute for both tryApiProvider and translateText
- GDELT: wrap client.searchGdeltDocuments in gdeltBreaker.execute, replace manual try/catch
- Fix: include all required fields (tokens, reason, error, errorType, query) in fallback objects
- CLIENT-03 fully satisfied: all 17 domains have circuit breaker coverage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(04-01): create 2L-VERIFICATION.md, fix desktop-readiness.ts, complete service barrel

- Create retroactive 2L-VERIFICATION.md with 12/12 must-haves verified
- Fix map-layers-core and market-panel stale file refs in desktop-readiness.ts
- Fix opensky-relay-cloud stale refs (api/opensky.js deleted)
- Add missing barrel re-exports: conflict, displacement, research, wildfires, climate
- Skip military/intelligence/news barrels (would cause duplicate exports)
- TypeScript compiles cleanly with zero errors

* docs(04-02): complete circuit breaker coverage plan

- SUMMARY.md: 6 domains covered, CLIENT-03 satisfied, 1 deviation (fallback type fix)
- STATE.md: Phase 4 plan 02 complete, position and decisions updated
- ROADMAP.md: Phase 04 marked complete (2/2 plans)
- REQUIREMENTS.md: CLIENT-03 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(04-01): complete documentation fixes plan

- Create 04-01-SUMMARY.md with execution results
- Update STATE.md with Plan 01 completion and decisions
- Update ROADMAP.md: Plan 04-01 checked, progress 1/2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-04): complete phase verification and fix tracking gaps

ROADMAP.md Phase 4 status updated to Complete, 04-02 checkbox checked,
progress table finalized. REQUIREMENTS.md coverage summary updated
(27 complete, 0 partial/pending). STATE.md reflects verified phase.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(military): migrate USNI fleet tracker to sebuf RPC

Port all USNI Fleet Tracker parsing logic from api/usni-fleet.js into
MilitaryService.GetUSNIFleetReport RPC with proto definitions, inline
Upstash caching (6h fresh / 7d stale), and client adapter mapping.
Deletes legacy edge function and _upstash-cache.js dependency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: add proto validation annotations and split all 17 handler files into per-RPC modules

Phase A: Added buf.validate constraints to ~25 proto files (~130 field annotations
including required IDs, score ranges, coordinate bounds, page size limits).

Phase B: Split all 17 domain handler.ts files into per-RPC modules with thin
re-export handler.ts files. Extracted shared Redis cache helpers to
api/server/_shared/redis.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add ADDING_ENDPOINTS guide and update Contributing section

All JSON endpoints must use sebuf — document the complete workflow for
adding RPCs to existing services and creating new services, including
proto conventions, validation annotations, and generated OpenAPI docs.

Update DOCUMENTATION.md Contributing section to reference the new guide
and remove the deprecated "Adding a New API Proxy" pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add blank lines before lists to pass markdown lint (MD032)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: disambiguate duplicate city keys in city-coords

Vercel's TypeScript check treats duplicate object keys as errors (TS1117).
Rename 'san jose' (Costa Rica) -> 'san jose cr' and
'cambridge' (UK) -> 'cambridge uk' to avoid collision.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve Vercel deployment errors — relocate hex-db + fix TS strict-null

- Move military-hex-db.js next to handler (fixes Edge Function unsupported module)
- Fix strict-null TS errors across 12 handler files (displacement, economic,
  infrastructure, intelligence, market, military, research, wildfire)
- Add process declare to wildfire handler, prefix unused vars, cast types

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: convert military-hex-db to .ts for Edge Function compatibility

Vercel Edge bundler can't resolve .js data modules from .ts handlers.
Also remove unused _region variable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: inline military hex db as packed string to avoid Edge Function module error

Vercel Edge bundler can't resolve separate data modules. Inline 20K hex
IDs as a single concatenated string, split into Set at runtime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove broken export { process } from 4 _shared files

declare const is stripped in JS output, making export { process }
reference nothing. No consumers import it — each handler file has
its own declare.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move server/ out of api/ to fix Vercel catch-all routing

Vercel's file-based routing was treating api/server/**/*.ts as individual
API routes, overriding the api/[[...path]].ts catch-all for multi-segment
paths like /api/infrastructure/v1/list-service-statuses (3 segments).
Moving to server/ at repo root removes the ambiguity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: rename catch-all to [...path] — [[...path]] is Next.js-only syntax

Vercel's native edge function routing only supports [...path] for
multi-segment catch-all matching. The [[...path]] double-bracket syntax
is a Next.js feature and was only matching single-segment paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add dynamic segment route for multi-segment API paths

Vercel's native file-based routing for non-Next.js projects doesn't
support [...path] catch-all matching multiple segments. Use explicit
api/[domain]/v1/[rpc].ts which matches /api/{domain}/v1/{rpc} via
standard single-segment dynamic routing that Vercel fully supports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove conflicting [...path] catch-all — replaced by [domain]/v1/[rpc]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: widen CORS pattern to match all Vercel preview URL formats

Preview URLs use elie-ab2dce63 not elie-habib-projects as the team slug.
Broaden pattern to elie-[a-z0-9]+ to cover both.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update sidecar build script for new api/[domain]/v1/[rpc] path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: trigger Vercel rebuild for all variants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 review — critical bugs, hardening, and cleanup

Fixes from @koala73's code review:

Critical:
- C-1: Add max-size eviction (2048 cap) to GeoIP in-memory cache
- C-2: Move type/source/severity filters BEFORE .slice(pageSize) in cyber handler
- C-3: Atomic SET with EX in Redis helper (single Upstash REST call)
- C-4: Add AbortSignal.timeout(30s) to LLM fetch in summarize-article

High:
- H-1: Add top-level try/catch in gateway with CORS-aware 500 response
- H-3: Sanitize error messages — generic text for 5xx, passthrough for 4xx only
- H-4: Add timeout (10s) + Redis cache (5min) to seismology handler
- H-5: Add null guards (optional chaining) in seismology USGS feature mapping
- H-6: Race OpenSky + Wingbits with Promise.allSettled instead of sequential fallback
- H-8: Add Redis cache (5min TTL) to infrastructure service-status handler

Medium:
- M-12/M-13: Fix HAPI summary field mappings (iso3 from countryCode, internallyDisplaced)

Infrastructure:
- R-1: Remove .planning/ from git tracking, add to .gitignore
- Port UCDP parallel page fetching from main branch (#198)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 review issues — hash, CORS, router, cache, LLM, GeoIP

Fixes/improvements from PR #106 code review tracking issues:

- #180: Replace 32-bit hash (Java hashCode/DJB2) with unified FNV-1a 52-bit
  hash in server/_shared/hash.ts, greatly reducing collision probability
- #182: Cache router construction in Vite dev plugin — build once, invalidate
  on HMR changes to server/ files
- #194: Add input length limits for LLM prompt injection (headlines 500 chars,
  title 500 chars, geoContext 2000 chars, max 10 headlines)
- #195/#196: GeoIP AbortController — cancel orphaned background workers on
  timeout instead of letting them fire after response is sent
- #198: Port UCDP partial-result caching from main — 10min TTL for partial
  results vs 6hr for complete, with in-memory fallback cache

Proto codegen regenerated for displacement + conflict int64_encoding changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore fast-xml-parser dependency needed by sebuf handlers

Main branch removed fast-xml-parser in v2.5.1 (legacy edge functions
no longer needed it), but sebuf handlers in aviation/_shared.ts and
research/list-arxiv-papers.ts still import it for XML API parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: fix stale paths, version badge, and local-backend-audit for sebuf

- ADDING_ENDPOINTS.md: fix handler paths from api/server/ to server/,
  fix import depth (4 levels not 5), fix gateway extension (.js not .ts)
- DOCUMENTATION.md: update version badge 2.1.4 -> 2.5.1, fix broken
  ROADMAP.md links to .planning/ROADMAP.md, fix handler path reference
- COMMUNITY-PROMOTION-GUIDE.md: add missing v2.5.1 to version table
- local-backend-audit.md: rewrite for sebuf architecture — replace all
  stale api/*.js references with sebuf domain handler paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: use make commands, add generation-before-push warning, bump sebuf to v0.7.0

- ADDING_ENDPOINTS.md: replace raw `cd proto && buf ...` with `make check`,
  `make generate`, `make install`; add warning that `make generate` must run
  before pushing proto changes (links to #200)
- Makefile: bump sebuf plugin versions from v0.6.0 to v0.7.0
- PR description also updated to use make commands

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: make install installs everything, document setup

- Makefile: `make install` now installs buf, sebuf plugins, npm deps,
  and proto deps in one command; pin buf and sebuf versions as variables
- ADDING_ENDPOINTS.md: updated prerequisites to show `make install`
- DOCUMENTATION.md: updated Installation section with `make install`
  and generation reminder

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 re-review issues — timeouts, iso3, pagination, CORS, dedup

- NEW-1: HAPI handler now returns ISO-3 code (via ISO2_TO_ISO3 lookup) instead of ISO-2
- NEW-3: Aviation FAA fetch now has AbortSignal.timeout(15s)
- NEW-4: Climate Open-Meteo fetch now has AbortSignal.timeout(20s)
- NEW-5: Wildfire FIRMS fetch now has AbortSignal.timeout(15s)
- NEW-6: Seismology now respects pagination.pageSize (default 500)
- NEW-9: Gateway wraps getCorsHeaders() in try/catch with safe fallback
- NEW-10: Tech events dedup key now includes start year to avoid dropping yearly variants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 round 2 — seismology crash, tsconfig, type contracts

- Fix _req undefined crash in seismology handler (renamed to req)
- Cache full earthquake set, slice on read (avoids cache pollution)
- Add server/ to tsconfig.api.json includes (catches type errors at build)
- Remove String() wrappers on numeric proto fields in displacement/HAPI
- Fix hashString re-export not available locally in news/_shared.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: loadMarkets heatmap regression, stale test, Makefile playwright

- Add finnhub_skipped + skip_reason to ListMarketQuotesResponse proto
- Wire skipped signal through handler → adapter → App.loadMarkets
- Fix circuit breaker cache conflating different symbol queries (cacheTtlMs: 0)
- Use dynamic fetch wrapper so e2e test mocks intercept correctly
- Update e2e test mocks from old endpoints to sebuf proto endpoints
- Delete stale summarization-chain.test.mjs (imports deleted files)
- Add install-playwright target to Makefile

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing proto fields to emptyStockFallback (build fix)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(quick-1): fix country fallback, ISO-2 contract, and proto field semantics

- BLOCKING-1: Return undefined when country has no ISO3 mapping instead of wrong country data
- BLOCKING-2: country_code field now returns ISO-2 per proto contract
- MEDIUM-1: Rename proto fields from humanitarian to conflict-event semantics (populationAffected -> conflictEventsTotal, etc.)
- Update client service adapter to use new field names
- Regenerate TypeScript types from updated proto

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(quick-1): add in-memory cache + in-flight dedup to AIS vessel snapshot

- HIGH-1: 10-second TTL cache matching client poll interval
- Concurrent requests share single upstream fetch (in-flight dedup)
- Follows same pattern as get-macro-signals.ts cache
- No change to RPC response shape

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(quick-1): stub RPCs throw UNIMPLEMENTED, remove hardcoded politics, add tests

- HIGH-2: listNewsItems, summarizeHeadlines, listMilitaryVessels now throw UNIMPLEMENTED
- LOW-1: Replace hardcoded "Donald Trump" with date-based dynamic LLM context
- LOW-1 extended: Also fix same issue in intelligence/get-country-intel-brief.ts (Rule 2)
- MEDIUM-2: Add tests/server-handlers.test.mjs with 20 tests covering all review items

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(quick-2): remove 3 dead stub RPCs (ListNewsItems, SummarizeHeadlines, ListMilitaryVessels)

- Delete proto definitions, handler stubs, and generated code for dead RPCs
- Clean _shared.ts: remove tryGroq, tryOpenRouter, buildPrompt, dead constants
- Remove 3 UNIMPLEMENTED stub tests from server-handlers.test.mjs
- Regenerate proto codegen (buf generate) and OpenAPI docs
- SummarizeArticle and all other RPCs remain intact

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 review findings (stale snapshot, iso naming, scoring, test import)

- Serve stale AIS snapshot on relay failure instead of returning undefined
- Rename HapiConflictSummary.iso3 → iso2 to match actual ISO-2 content
- Fix HAPI fallback scoring: use weight 3 for combined political violence
  (civilian targeting is folded in, was being underweighted at 0)
- Extract deduplicateHeadlines to shared .mjs so tests import production code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add coverage for PR106 iso2 and fallback regressions

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-02-21 03:39:56 +04:00
Elie Habib f8d0c81f60 fix: sentry triage — quota guard, video ID validation, map race conditions
- Stop localStorage/IndexedDB writes when QuotaExceededError detected
- Validate YouTube video ID format before passing to IFrame Player
- try/catch around deck.gl setProps (null.getProjection during teardown)
- try/catch around setFilter in mousemove/mouseout (style not loaded)
- Add 7 ignoreErrors patterns for iOS/Android/SW noise
2026-02-20 20:12:49 +00:00
Elie Habib 7440281d71 release: v2.5.1 — batch FRED, parallel UCDP, partial cache TTL, bot middleware 2026-02-20 15:07:11 +04:00
Elie Habib 92892c5b7c fix: derive cache headers from partial flag on cache hits & widen Load failed filter 2026-02-20 15:04:49 +04:00
Elie Habib 7681202c6e fix: use per-entry TTL for UCDP fallbackCache to expire partial results in 10m 2026-02-20 15:00:44 +04:00
Elie Habib b714e7b13e fix: harden API routes, batch FRED requests, and sanitize tooltip HTML
- fred-data: batch mode (comma-separated series_id) reduces 7 edge
  function invocations to 1; cap at 15 series; propagate upstream
  502s instead of masking as empty 200; add X-Data-Status header
- ucdp-events: parallelize page fetches; track failed pages and use
  short cache TTL for partial results instead of caching at full 6h
- ucdp: add OPTIONS/method guard matching ucdp-events pattern
- middleware: exact-match social bot paths instead of startsWith
- vercel.json: use VERCEL_GIT_PREVIOUS_SHA for multi-commit diffs;
  add middleware.ts, settings.html, vercel.json to watch list
- Panel.ts: use safeHtml() allowlist sanitizer for tooltip content
- dom-utils: add safeHtml() with tag/attribute allowlist and
  javascript: URI blocking
2026-02-20 14:51:54 +04:00
Elie Habib 3ffe76b208 perf: add bot protection middleware and robots.txt to reduce API abuse
Block crawlers/scrapers from /api/* routes via Edge Middleware (403 for
bot user-agents and missing/short UAs). Social preview bots (Twitter,
Facebook, LinkedIn, Slack, Discord) are allowed on /api/story and
/api/og-story for OG previews. robots.txt reinforces the same policy.
2026-02-20 13:38:43 +04:00
Elie Habib 84336a9f94 refactor: extract duplicate layerToSource mapping to LAYER_TO_SOURCE config constant
DRY extraction of identical map defined in both syncDataFreshnessWithLayers()
and setupMapLayerHandlers(). Single source of truth in config/panels.ts.
Supersedes PR #145 which was based on stale main (missing gdelt_doc).
2026-02-20 13:37:49 +04:00
Elie Habib 919e7c996e perf: reduce Vercel costs — extend API cache TTLs and skip non-code builds
- vercel.json: add ignoreCommand to skip builds when only docs/tests/CI change
- service-status: extend edge cache 60s→300s (46 service checks, slow-moving)
- cyber-threats: extend edge cache 5min→1hr (threat intel updates hourly)
- theater-posture: extend edge cache 60s→300s, stale fallback 30s→120s
- markets/crypto polling: 2min→4min (reduce edge requests by ~50%)
2026-02-20 13:25:35 +04:00
Elie Habib fd51b57911 fix: shared fetch abort isolation and USNI vessel augmentation
- cached-risk-scores/cached-theater-posture: shared fetch must not be
  canceled by one caller's AbortSignal; use withCallerAbort wrapper so
  individual callers can abort without killing the shared promise
- StrategicPosturePanel: remove isMilitaryVesselTrackingConfigured guard
  that blocked USNI fleet data when AIS key was absent
- Panel: use rawHtml for info tooltip to render HTML content correctly
2026-02-20 13:17:11 +04:00
Elie Habib 91c9982a41 fix: stabilize E2E tests and fix InvestmentsPanel event delegation
- InvestmentsPanel: replace per-element listeners (destroyed by debounced
  innerHTML) with event delegation on stable this.content
- IntelligenceGapBadge: null guard on context.actionableInsight for
  environments where i18n is not initialized
- keyword-spike test: restructure headlines so spike term appears
  mid-sentence (isLikelyProperNoun skips index-0), add initI18n
- investments-panel test: add pollUntil helper, re-query DOM elements
  after debounced render cycles
- map-harness tests: increase WebGL harness init and poll timeouts,
  relax golden screenshot pixel diff threshold
2026-02-20 13:16:56 +04:00
Elie Habib ac7f3fac5e fix: add Sentry noise filter for Android WebView bridge errors
Filter window.android.* TypeError from Huawei/Android WebView
shell injections — not our code.
2026-02-20 12:58:51 +04:00
Elie Habib 61c1dad6c9 fix: suppress maplibre-internal TypeErrors in Sentry beforeSend
Add broader filter: any TypeError where all in-app frames are in the
maplibre map chunk is suppressed. Catches hash-parsing crashes like
'o.includes' that are entirely inside maplibre-gl internals.
2026-02-20 11:57:40 +04:00
Elie Habib b49329cdef refactor: replace innerHTML with programmatic DOM via h() hyperscript
Add dom-utils.ts with h(), replaceChildren(), fragment(), rawHtml()
helpers. Convert 8 components from innerHTML string concatenation to
type-safe DOM construction, eliminating XSS surface and HTML parsing.

Components: Panel, CIIPanel, GdeltIntelPanel, MonitorPanel,
PizzIntIndicator, ServiceStatusPanel, StatusPanel, TechEventsPanel.
Event listeners now inline via onClick props (removes post-render
querySelector+addEventListener pattern). Fix ServiceStatusPanel
missing super.destroy() call.
2026-02-20 10:01:01 +04:00
Elie Habibandkoala73 5cfd827173 feat: abort in-flight fetch requests on panel destroy and page hide
- Add AbortController to Panel base class, abort on destroy()
- Wire signal into 7 panel fetch calls with AbortError guard
- Add signal param to fetchCachedRiskScores/fetchCachedTheaterPosture
- Add abort lifecycle to CountryBriefPage (show/hide)
- Fix StrategicRiskPanel missing super.destroy() call

Co-authored-by: koala73 <koala73@users.noreply.github.com>
2026-02-20 09:46:09 +04:00
Elie Habib ab69f09c60 fix: update Sentry noise filters for Safari fullscreen and deck.gl null id
- Add /webkitEnterFullscreen/ to ignoreErrors for Safari internal fullscreen crash
- Add "id" to beforeSend property alternation for deck.gl _drawLayers null access
2026-02-20 09:39:39 +04:00
7aa60f954c perf: move inline panel styles to panels.css, loaded once via main.css (PERF-012) (#154)
Inline <style> blocks were re-injected on every render in 6 components,
causing CSSOM recalc and GC pressure from repeated string allocation.
Extracted all panel styles into src/styles/panels.css, imported once via
main.css. Affected components: SatelliteFiresPanel, PopulationExposurePanel,
ClimateAnomalyPanel, DisplacementPanel, UcdpEventsPanel, DownloadBanner.

https://claude.ai/code/session_01Erkji3Bg2NwawE25NFC6vh

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-20 09:39:26 +04:00
Elie Habib 7a57d39408 perf: split i18next, sentry, and panels into separate chunks
Add manual chunk rules for i18next → i18n, @sentry/* → sentry, and
*Panel.ts → panels. Improves caching (stable vendor chunks) and
reduces main bundle size via parallel loading.

Closes #148
2026-02-20 09:25:38 +04:00
Elie HabibandLawyered 6e9615f428 fix(sidecar): deduplicate Vary header tokens with appendVary helper
Replace naive string concatenation for Vary header with appendVary()
that parses existing tokens and deduplicates case-insensitively.
Prevents duplicate Vary tokens when both Origin and Accept-Encoding
are added.

Closes #170

Co-authored-by: Lawyered <4802498+lawyered0@users.noreply.github.com>
2026-02-20 09:21:36 +04:00
Elie HabibandGitHub 4e10bbc283 IndexedDB-backed persistent cache + stale-while-revalidate API response caching (#171)
### Motivation
- Improve repeat-visit performance by persisting API responses and
data-source payloads client-side so the dashboard can show cached data
immediately while refreshing in the background (PERF-021).
- Provide a robust multi-tier storage strategy: prefer desktop/Tauri
storage, then IndexedDB in browser runtime, and fall back to
`localStorage` if needed.

### Description
- Upgrade `src/services/persistent-cache.ts` to use an IndexedDB store
(`worldmonitor_persistent_cache`) with
`getFromIndexedDb`/`setInIndexedDb` while keeping Tauri RPC and
`localStorage` fallbacks.
- Add stale-while-revalidate API response caching to
`src/utils/proxy.ts` via `fetchWithProxy`, which returns cached `/api/*`
responses immediately and refreshes them in the background (persisted
under `api-response:<url>`).
- Route key data-source fetchers through the new proxy cache by
switching `fetch` -> `fetchWithProxy` in `src/services/arxiv.ts`,
`src/services/hackernews.ts`, `src/services/github-trending.ts`, and
`src/services/earthquakes.ts` so they inherit persistent response
hydration.
- Preserve existing behaviors and fallbacks (desktop storage first,
IndexedDB where available, then `localStorage`) and avoid breaking
existing circuit-breaker logic in services.

### Testing
- Ran `npm run typecheck` (`tsc --noEmit`) and it succeeded.
- Ran `npm run test:data` and the suite executed; most tests passed but
one existing `deploy/cache` guardrail subtest failed (pre-existing
assertion about a precache glob pattern), which is unrelated to the
caching changes.

------
[Codex
Task](https://chatgpt.com/codex/tasks/task_e_6997871eb7648333a131f041010684de)
2026-02-20 09:12:18 +04:00
Elie Habib 13172cec4c fix: use graph edges for infra cascade capacity instead of hardcoded 0.1
Closes #139 — pass DependencyGraph as parameter (no redundant rebuild),
check direct edges first, then walk BFS dependency chain for indirect
impacts (chokepoint → port → country). Returns 0 for truly unaffected.
2026-02-20 09:11:13 +04:00
Elie Habib c62fad7fce Add IndexedDB-backed persistent API caching 2026-02-20 09:08:50 +04:00
Elie Habib 7ab3a8e76d fix: widen SW globIgnore to exclude all ML JS chunks
Closes #169 — the `ml-*` pattern missed chunks without a hyphen,
causing ~60 MB of ML code to be precached by the service worker.
2026-02-20 09:05:45 +04:00
Elie Habib f521377453 feat: integrate USNI fleet tracker and add parser tests 2026-02-20 09:05:23 +04:00
Elie HabibandGitHub dd34c1e4ff Add skeleton loading screen to prevent FOUC (#146)
## Summary

Adds a pre-rendered skeleton loading screen that displays instantly
before JavaScript boots, eliminating flash of unstyled content (FOUC).
The skeleton mimics the final layout with animated shimmer effects and
automatically replaces when the app initializes.

## Type of change

- [x] New feature
- [ ] Bug fix
- [ ] Refactor / code cleanup
- [ ] Other

## Affected areas

- [x] Other: Loading UX / Initial page render

## Details

This change improves the perceived performance and user experience by:

1. **Inlining critical skeleton CSS** in the `<head>` to ensure it
renders before any external stylesheets load
2. **Pre-rendering skeleton HTML** in the `#app` div that displays
instantly on page load
3. **Supporting both themes** with dark mode as default and light mode
variants via `[data-theme="light"]` selectors
4. **Adding shimmer animation** to skeleton lines for visual feedback
that content is loading
5. **Marking skeleton as hidden from accessibility** with
`aria-hidden="true"` so screen readers skip it

The skeleton layout includes:
- Header with logo, filters, and controls
- Map section with toolbar
- Grid of 6 data panels with headers and content lines

When the Vue app initializes and calls `renderLayout()`, the skeleton is
automatically replaced with the actual content.

## Checklist

- [x] No API keys or secrets committed
- [x] Tested theme switching (dark/light mode skeleton variants)
- [x] Skeleton is properly hidden from screen readers

https://claude.ai/code/session_01Fxk8GMRn2cEUq3ThC2a8e5
2026-02-20 08:49:05 +04:00
Elie HabibandGitHub 49f9fc860f Optimize GPU memory usage with will-change CSS property (#155)
## Summary

Optimize GPU memory usage by strategically applying and removing the
`will-change` CSS property on animated elements. This prevents
unnecessary GPU memory allocation for elements that don't require
continuous optimization, while maintaining smooth animations through the
transition/animation lifecycle.

## Type of change

- [x] Refactor / code cleanup
- [x] Performance optimization

## Affected areas

- [x] Map / Globe
- [x] Other: UI animations and modals

## Changes

### CSS Updates (`src/styles/main.css`)
- Added `will-change: transform, opacity;` to `.panel-header` (hover
state)
- Added `will-change: transform, opacity;` to `.signal-modal` (entrance
animation)
- Added `will-change: transform;` to `.map-popup` (slide-in transition)
- Added `will-change: transform, opacity;` to `.mobile-warning-modal`
(entrance animation)
- Added `will-change: transform;` to `.tech-event-marker` (transform
animations)

### JavaScript Updates
- **MapPopup.ts**: Added listener to remove `will-change` after slide-in
transition completes
- **MobileWarningModal.ts**: Added listener to remove `will-change`
after entrance animation completes
- **SignalModal.ts**: Added listener to remove `will-change` after
entrance animation completes

## Rationale

The `will-change` property hints to the browser to prepare for
animations, but maintaining it indefinitely wastes GPU memory. By
removing it after animations complete (via `transitionend` or
`animationend` events), we free up resources while preserving animation
performance.

## Checklist

- [x] TypeScript compiles without errors
- [x] No API keys or secrets committed

https://claude.ai/code/session_01DDhT6Ex596eX1CtSb6mHdH
2026-02-20 08:48:38 +04:00
Elie HabibandGitHub b7ca969228 PERF-010: Debounce Panel.setContent to reduce re-renders (#163)
### Motivation
- Panels can receive multiple content updates in quick succession,
causing many DOM writes and janky re-renders during data bursts.
- Pairing a debounce with the existing DOM diffing/RAF throttling
reduces unnecessary work and smooths the UI.

### Description
- Added a 150ms debounce to `Panel.setContent()` and new private fields
(`contentDebounceMs`, `pendingContentHtml`, `contentDebounceTimer`) to
buffer rapid updates and apply only the latest payload.
- `setContent()` now skips no-op updates when the pending or current
HTML already matches and resets the timer on rapid calls.
- Introduced `setContentImmediate()` and switched `showLoading()`,
`showError()`, and `showConfigError()` to use it so immediate UI states
are not overwritten by stale debounced writes.
- Extended `destroy()` to clear any pending debounce timers and pending
content to avoid delayed updates after teardown.

### Testing
- Ran `npm run typecheck` (i.e. `tsc --noEmit`) and it completed
successfully.

------
[Codex
Task](https://chatgpt.com/codex/tasks/task_e_699782a113a88333ab664a828cb00b02)
2026-02-20 08:47:22 +04:00
Elie HabibandGitHub 383da59ffc Add CSS containment to panel elements for performance (#152)
## Summary

Add CSS `contain: content` property to panel elements to enable browser
optimizations and improve rendering performance by establishing a new
stacking context and limiting layout recalculations.

## Type of change

- [x] Refactor / code cleanup

## Affected areas

- [x] Other: CSS performance optimization

## Details

This change adds the `contain: content` CSS property to panel elements
(`.panel` class). This CSS containment property:

- Enables the browser to optimize rendering by limiting the scope of
layout, style, and paint calculations
- Establishes a new stacking context, which can improve performance when
panels are frequently repositioned or resized
- Has no visual impact on the UI

This is a low-risk performance optimization that leverages modern CSS
capabilities to improve responsiveness, especially when dealing with
multiple panels on the dashboard.

## Checklist

- [x] No API keys or secrets committed
- [x] TypeScript compiles without errors

## Testing

No testing needed - this is a CSS-only performance optimization with no
behavioral changes.

https://claude.ai/code/session_01E9FpgiebjEuUPhNt8mwX9U
2026-02-20 08:46:53 +04:00
Elie HabibandGitHub ffb5ae0413 Fix deep-link polling from retrying indefinitely (#160)
### Motivation
- Prevent unbounded polling when deep-link handling waits for data,
since the previous recursive `setTimeout` loops could run forever if
sources never become available.
- Provide a clear user-facing error when deep-link data never appears so
the UI stops noisy background activity and informs the user.

### Description
- Added bounded retry configuration constants `MAX_DEEP_LINK_RETRIES`,
`DEEP_LINK_RETRY_INTERVAL_MS`, and `DEEP_LINK_INITIAL_DELAY_MS` inside
`handleDeepLinks()`.
- Implemented attempt counters for both the story deep-link
(`checkAndOpen`) and country-brief deep-link (`checkAndOpenBrief`) flows
and stop retrying once the max attempts are reached.
- Show a user-facing error via `showToast('Data not available')` when
the retry limit is exceeded and `return` early on success or timeout.
- Replaced hardcoded delay values with the new constants for easier
tuning and consistency.

### Testing
- Ran `npm run typecheck` and the TypeScript typecheck completed
successfully.

------
[Codex
Task](https://chatgpt.com/codex/tasks/task_e_6997819f2ab48333b9972b4e28212e9a)
2026-02-20 08:44:24 +04:00
Elie HabibandGitHub e549b6e5a6 Handle per-cluster render failures in NewsPanel (#159)
### Motivation
- Rendering could abort when a clustering result contained malformed
data (e.g. `undefined` headline), leaving the News panel blank;
individual cluster renders were not isolated.
- Add per-cluster error isolation so one bad cluster does not break the
entire list.

### Description
- Added a safe wrapper `renderClusterHtmlSafely` that calls
`renderClusterHtml` inside a `try/catch`, logs the error and cluster
payload, and returns a simple fallback card when rendering fails.
- Replaced direct calls to `renderClusterHtml` in both the virtualized
callback and the small-list `.map(...)` path with
`renderClusterHtmlSafely` so both rendering paths are protected.
- Fallback card includes a `data-cluster-id` attribute (or
`unknown-cluster`) and the visible text `Failed to display this
cluster.` to indicate a problem without breaking the remaining UI.
- Changes made in `src/components/NewsPanel.ts`.

### Testing
- Ran type checking with `npm run typecheck`, which completed
successfully.
- Launched the dev server with `npm run dev` for visual verification and
it started (Vite ready), and captured a visual snapshot using
Playwright; a screenshot artifact was produced.
- No automated unit tests were added or modified for this change; type
checking and a dev-time visual smoke test passed.

------
[Codex
Task](https://chatgpt.com/codex/tasks/task_e_6997816d9080833383026a60d2e9da52)
2026-02-20 08:43:31 +04:00
Elie HabibandGitHub db4b401b8f Add Brotli-first compression for local API sidecar and Nginx proxy (#168)
### Motivation
- Reduce sidecar API egress by preferring Brotli (br) for clients that
support it while keeping gzip as a fallback for broad compatibility.
- Ensure the sidecar negotiates compression consistently (set
`Content-Encoding`, append `Vary: Accept-Encoding`) and avoids sending
stale `Content-Length` when responses are compressed.
- Provide an Nginx proxy baseline that preserves client
`Accept-Encoding` and prefers Brotli at the proxy layer.

### Description
- Updated `src-tauri/sidecar/local-api-server.mjs` to negotiate response
compression via a new `maybeCompressResponseBody` routine that prefers
Brotli (`br`) and falls back to gzip for payloads > 1KB, preserves
existing `Content-Encoding` from handlers, and appends `Vary:
Accept-Encoding` when compressing.
- Added a small helper `canCompress` and promisified `brotliCompress` so
Brotli compression runs asynchronously while gzip remains synchronous
for fallback.
- Ensure `Content-Length` is removed when `Content-Encoding` is applied
so downstream callers/proxies don't see stale lengths.
- Added tests to `src-tauri/sidecar/local-api-server.test.mjs` that
verify Brotli is preferred when `Accept-Encoding` includes `br` and gzip
is used when `br` is not present, including decompression checks with
`brotliDecompressSync`/`gunzipSync`.
- Added `deploy/nginx/brotli-api-proxy.conf` as an example Nginx
configuration enabling Brotli with gzip fallback and forwarding
`Accept-Encoding` while disabling upstream decompression (`gunzip off`).
- Updated `README.md` to reflect the sidecar now advertising `br/gzip`
behavior and adjusted the expected payload reduction notes.

### Testing
- Ran the sidecar unit tests with `node --test
src-tauri/sidecar/local-api-server.test.mjs`, which executed 19 tests
and all passed (`# pass 19`, `# fail 0`).
- No other automated test suites were modified; the new tests are
self-contained under the sidecar test file and validated the compression
behavior.

------
[Codex
Task](https://chatgpt.com/codex/tasks/task_e_6997871302188333bb1ecb20714d2ad3)
2026-02-20 08:42:26 +04:00
Elie Habib 2b7b35efd8 Add Brotli-first API compression for sidecar and nginx 2026-02-20 08:41:22 +04:00
Elie HabibandGitHub 4b0e50e0fe Optimize i18n bundle with lazy-loaded locale files (#167)
## Summary

Refactors the i18n service to lazy-load locale files instead of bundling
all translations upfront. English is eagerly imported as the fallback
language, while all other locales are dynamically imported only when
needed. Updates Vite configuration to handle locale chunk naming and
service worker caching.

## Type of change

- [x] Refactor / code cleanup
- [x] CI / Build / Infrastructure

## Affected areas

- [x] Config / Settings
- [x] Other: Internationalization (i18n) and build optimization

## Details

### Changes to `src/services/i18n.ts`
- Eagerly imports English translation (`en.json`) as the fallback
language
- Replaces the static `LOCALE_LOADERS` object with `import.meta.glob()`
for dynamic locale loading
- Updates `ensureLanguageLoaded()` to handle both the eagerly-loaded
English and lazy-loaded locales
- Adds fallback logic to use English if a requested locale file fails to
load
- Simplifies `initI18n()` to use the pre-imported English translation

### Changes to `vite.config.ts`
- Adds `**/locale-*.js` to service worker `globIgnores` to prevent
precaching of lazy-loaded locale chunks
- Adds a new service worker cache strategy for locale files
(`CacheFirst` with 30-day expiration)
- Implements custom chunk naming in the build config to prefix
lazy-loaded locale files with `locale-` (excluding `en.json` which stays
in the main bundle)

## Benefits

- **Reduced initial bundle size**: Only English is bundled by default;
other locales load on-demand
- **Improved code splitting**: Each locale becomes a separate chunk that
can be cached independently
- **Better service worker handling**: Locale chunks are excluded from
precache and use appropriate caching strategies

## Checklist

- [x] TypeScript compiles without errors
- [x] No API keys or secrets committed

https://claude.ai/code/session_01Hd1kUsbZaAw5vzhBWS9Mkx
2026-02-20 08:34:45 +04:00
Claude cbd7c29f03 Merge main into tree-shake-locale-files, resolve workbox conflict
Resolve conflict in vite.config.ts workbox config by combining both
branches: keep main's navigateFallback: null fix (prevents stale HTML
precache) while preserving the PR's locale-*.js glob ignore and
runtime cache strategy for lazy-loaded locale chunks.

https://claude.ai/code/session_01Hd1kUsbZaAw5vzhBWS9Mkx
2026-02-20 04:32:14 +00:00
Elie HabibandGitHub 86b83bae78 Add Brotli pre-compression for Vite build assets (#162)
### Motivation
- Reduce client transfer sizes for static assets by emitting
pre-compressed Brotli artifacts at build time so edge/origin can serve
`.br` directly.
- Avoid dependency on third-party Vite compression packages when npm
registry/policy blocks installation by implementing a small,
self-contained build plugin.

### Description
- Added an inlined Vite build plugin `brotliPrecompressPlugin()` to
`vite.config.ts` that uses Node's `zlib.brotliCompress` to write `.br`
files for emitted assets with extensions
`(.js,.mjs,.css,.html,.svg,.json,.txt,.xml,.wasm)` larger than `1024`
bytes.
- Registered the plugin in the Vite `plugins` array so Brotli artifacts
are generated automatically during `vite build`.
- Documented build-time Brotli pre-compression and recommended Hetzner
Nginx settings (`gzip_static on` and `brotli_static on`) and Cloudflare
serving behavior in `README.md` under the Bandwidth Optimization
section.

### Testing
- Attempted to install `vite-plugin-compression2` and
`vite-plugin-compression`, but both `npm install -D` runs failed with a
`403 Forbidden` from the registry in this environment. (expected
fallback)
- Ran `npm run build` successfully and observed build completion without
errors.
- Verified generated Brotli files exist under `dist/` (examples:
`dist/index.html.br`, `dist/settings.html.br`, and multiple
`dist/assets/*.br`).

------
[Codex
Task](https://chatgpt.com/codex/tasks/task_e_6997827871e483339853aaae2971d2a3)
2026-02-20 08:27:09 +04:00
Elie HabibandGitHub 814c01fea2 Missing GDELT Doc in Data Freshness Tracker (#165)
layerToSource maps layers to freshness source IDs, but several
API-backed data sources (GDELT Doc intelligence feed, FRED, EIA oil,
USASpending, PizzINT, Polymarket, Predictions) are not tracked in the
freshness system.
The Status Panel cannot report staleness for these feeds.
2026-02-20 08:26:23 +04:00
Elie Habib 7972248e1c Track missing backend feeds in data freshness 2026-02-20 08:25:28 +04:00
Elie HabibandGitHub 380f1b7235 Implement YouTube live stream detection via HTML parsing (#144)
## Summary

Implements proper YouTube live stream detection by fetching the
channel's live page and parsing the HTML response for video ID and live
status, replacing the previous placeholder implementation.

## Type of change

- [x] New feature
- [ ] Bug fix
- [ ] New data source / feed
- [ ] New map layer
- [ ] Refactor / code cleanup
- [ ] Documentation
- [ ] CI / Build / Infrastructure

## Affected areas

- [x] API endpoints (`/api/*`)
- [ ] Other

## Changes

The YouTube Live plugin now:
- Constructs the proper YouTube channel live URL (handling both
`@handle` and plain channel names)
- Fetches the live page with appropriate User-Agent headers
- Parses the HTML response to extract `videoId` and `isLive` status
using regex patterns
- Returns structured JSON with video ID and live status, or null values
if no live stream is detected
- Maintains 5-minute cache control headers for performance

## Checklist

- [x] TypeScript compiles without errors
- [ ] No API keys or secrets committed
- [ ] Tested on [worldmonitor.app](https://worldmonitor.app) variant
- [ ] Tested on [tech.worldmonitor.app](https://tech.worldmonitor.app)
variant (if applicable)

## Test Plan

The implementation uses standard HTTP fetching and regex parsing. Verify
that:
- The endpoint correctly identifies live streams on active YouTube
channels
- The endpoint returns null values for channels without active streams
- Response caching works as expected (5-minute TTL)
2026-02-20 08:22:33 +04:00
Elie HabibandGitHub 316df2997d Add i18n support for UI components and new features (#151)
## Summary

This PR adds comprehensive internationalization (i18n) support across
multiple UI components and introduces new localization strings for
recently added features. The changes convert hardcoded English strings
to use the i18n translation system (`t()` function), enabling
multi-language support for the application.

## Type of change

- [ ] Bug fix
- [x] New feature
- [ ] New data source / feed
- [ ] New map layer
- [x] Refactor / code cleanup
- [ ] Documentation
- [ ] CI / Build / Infrastructure

## Affected areas

- [ ] Map / Globe
- [x] News panels / RSS feeds
- [x] AI Insights / World Brief
- [ ] Market Radar / Crypto
- [ ] Desktop app (Tauri)
- [ ] API endpoints (`/api/*`)
- [x] Config / Settings
- [x] Other: Component localization, regulation dashboard, strategic
risk panel

## Changes

### Locale Files (ar.json, ja.json, tr.json, zh.json, sv.json, de.json,
es.json, fr.json, it.json, nl.json, pl.json, pt.json, ru.json, en.json)

- Added `OLLAMA_API_URL` and `OLLAMA_MODEL` configuration descriptions
for local LLM support
- Added new `levels` object with risk level translations (Critical,
High, Elevated, Moderate, Normal, Low)
- Added new `trends` object with trend direction translations (Rising,
Falling, Stable)
- Added new `fallback` object with instability index and geopolitical
event templates
- Added AI Regulation Dashboard strings: `dashboard`, `actionsCount`,
`deadlinesCount`, `activeCount`, `proposedCount`, `moreProvisions`,
`source`, and `stances` (strict, moderate, permissive, undefined)
- Added insights progress tracking strings: `step`, `waitingForData`,
`rankingStories`, `analyzingSentiment`, `generatingBrief`
- Added tech events panel strings: `retry`, `upcoming`, `conferences`,
`earnings`, `all`, `conferencesCount`, `onMap`, `techmemeEvents`,
`today`, `soon`
- Added tech readiness panel strings: `fetchingData`,
`internetUsersIndicator`, `mobileSubscriptionsIndicator`,
`broadbandAccess`, `rdExpenditure`, `analyzingCountries`, `source`,
`updated`
- Added strategic risk panel strings for data source management and risk
assessment states

### Component Files

- **InvestmentsPanel.ts**: Converted `SECTOR_LABELS` constant to
`getSectorLabel()` function using i18n translations
- **RegulationPanel.ts**: Updated hardcoded "AI Regulation Dashboard"
and tab labels to use i18n
- **VerificationChecklist.ts**: Added i18n import and converted
hardcoded verification check labels to use translations
- **StrategicRiskPanel.ts**: Updated risk assessment UI strings to use
i18n
- **ServiceStatusPanel.ts**: Converted `CATEGORY_LABELS` constant to
`getCategoryLabel()` function using i18n
- **TechEventsPanel.ts**: Updated error handling and retry button text
to use i18n
- **TechReadinessPanel.ts**: Updated data fetching UI strings to use
i18n
- **InsightsPanel.ts**: Updated progress tracking strings to use i18n
- **LiveNewsPanel.ts**: Updated offline status messages to use i18n
- **Panel.ts**: Updated aria-label to use i18n
- **LanguageSelector.ts**: Added `t` function import for consistency

## Checklist

- [x] Tested on [worldmonitor.app](https://worldmonitor.app) variant
- [x] Tested on [tech.worldmonitor.app](https://tech.worldmonitor.app)
variant (if applicable)
- [x] No API keys or secrets committed
- [x] TypeScript compiles without errors

## Notes

All hardcoded English strings in components have been replaced with i18n
translation calls, enabling the application to support multiple
languages consistently. The locale files have been updated with
translations for all new UI elements and features introduced in recent
updates.
2026-02-20 08:21:26 +04:00
Claude fb1efd2ebc fix: scope videoId and isLive to same videoDetails block
The previous regex matched the first "videoId" anywhere in the YouTube
HTML and independently checked for "isLive": true anywhere else. On
channel pages with multiple video objects this could combine fields
from different objects, returning a non-live or unrelated video ID.

Now both fields are extracted from within the same "videoDetails" block
(the primary player's data), ensuring the videoId and live status
always correspond to each other. Fixed in both the Vite dev plugin and
the production edge function.

https://claude.ai/code/session_01684qa7XvS7sf9CShqU8zNg
2026-02-20 04:19:37 +00:00
Elie HabibandGitHub ef13263c7b Fix memory leak: properly clear clock interval on cleanup (#143)
## Summary

Fixes a memory leak where the clock update interval was not being
cleared when the app is destroyed. The interval ID is now stored and
properly cleaned up in the `destroy()` method, matching the pattern
already used for the update check interval.

## Type of change

- [x] Bug fix
- [ ] New feature
- [ ] New data source / feed
- [ ] New map layer
- [ ] Refactor / code cleanup
- [ ] Documentation
- [ ] CI / Build / Infrastructure

## Affected areas

- [ ] Map / Globe
- [ ] News panels / RSS feeds
- [ ] AI Insights / World Brief
- [ ] Market Radar / Crypto
- [ ] Desktop app (Tauri)
- [ ] API endpoints (`/api/*`)
- [ ] Config / Settings
- [x] Other: Memory management / cleanup

## Details

The `setupClock()` method was creating a recurring interval every second
to update the UTC time display, but the interval ID was not being
stored. This meant the interval would continue running even after the
app was destroyed, causing a memory leak.

Changes:
- Added `clockIntervalId` property to store the interval reference
- Updated `setupClock()` to assign the interval ID instead of discarding
it
- Added cleanup logic in `destroy()` to clear the interval and reset the
reference

## Checklist

- [x] TypeScript compiles without errors
- [x] Follows existing cleanup patterns in the codebase
2026-02-20 08:15:41 +04:00
Elie HabibandGitHub 2df58717e4 WebGL2/DeckGL fails (Linux Mint example) (#142)
### Motivation
- Some Linux/WebKitGTK environments expose only partial WebGL (WebGL1)
and/or cause DeckGL/maplibre initialization to throw, leaving the
desktop app with a black or white blank map surface instead of the
expected UI.

### Description
- Tighten desktop map capability detection to require `webgl2` in
`hasWebGLSupport()` so DeckGL is only selected when WebGL2 is available.
- Wrap DeckGL construction in a `try/catch` in `MapContainer.init()` and
automatically fall back to the existing SVG `MapComponent` if
initialization throws, while logging a warning for diagnostics.
- Changes are contained in `src/components/MapContainer.ts` and preserve
existing SVG fallback behavior for mobile and failure cases.

### Testing
- Ran type checking with `npm run typecheck` (`tsc --noEmit`) which
completed successfully.

------
[Codex
Task](https://chatgpt.com/codex/tasks/task_e_69978099ec6c833396ec89ce651b8772)
2026-02-20 08:09:09 +04:00
Elie Habib 282eba9838 fix: harden map fallback and add regression coverage 2026-02-20 08:07:52 +04:00
Elie Habib cf46ff8940 Debounce panel content updates to reduce rerenders 2026-02-20 08:04:13 +04:00
Elie Habib 0735ce5c78 Add build-time Brotli precompression for static assets 2026-02-20 08:04:02 +04:00
Elie Habib a1973d4681 Fix deep-link polling retries and timeout error 2026-02-20 08:02:58 +04:00
Elie Habib aae90df9c9 Handle per-cluster render failures in NewsPanel 2026-02-20 08:02:49 +04:00
Elie HabibandGitHub a78410b8d2 fix(runtime+pwa): block local-secret cloud fallback and stale HTML precache (#140)
## Summary
- prevent cloud fallback for local-only sidecar endpoints
(`/api/local-*`) in desktop runtime fetch patch
- ensure local secret/config routes never send payloads to remote hosts
on local errors
- add runtime E2E regression proving no remote fallback for:
  - `/api/local-env-update`
  - `/api/local-validate-secret`
- apply deploy/cache guardrail fixes so SPA HTML is network-driven (not
precached)
  - remove `index.html` from Workbox precache glob
  - explicitly disable `navigateFallback` (`navigateFallback: null`)
  - add deploy guardrail test coverage for `navigateFallback: null`

## Why
- Desktop secret-management routes can carry API keys and credentials;
they must remain local-only even when local calls fail.
- Precaching HTML can serve stale shell documents after deploy and point
users at deleted chunk hashes.

## Changes
- `src/services/runtime.ts`
  - classify `/api/local-*` as local-only
  - block cloud fallback for local-only routes on both:
    - non-OK local responses
    - local fetch/network errors
- `e2e/runtime-fetch.spec.ts`
- add test: `runtime fetch patch never sends local-only endpoints to
cloud`
- `vite.config.ts`
  - remove `index.html` from Workbox precache `globPatterns`
- set `navigateFallback: null` to prevent default SW navigation fallback
route generation
- `tests/deploy-config.test.mjs`
  - assert precache glob remains HTML-free
  - assert `navigateFallback: null` and no `navigateFallbackDenylist`

## Validation
- `npm run typecheck` 
- `npm run test:e2e:runtime`  (6/6)
- `npm run test:sidecar` 
- `npm run test:data` 
- `npm run build` 
2026-02-20 08:00:18 +04:00
Elie Habib 44d2254b5b docs(runtime): document local-only API boundary 2026-02-20 07:52:23 +04:00
Elie Habib e10ed6c981 Add Linux-safe fallback when DeckGL WebGL2 is unavailable 2026-02-20 07:38:57 +04:00
Elie Habib 07e14deace fix: update Sentry noise filters for CSP regex and truncated loads
Fix ignoreErrors regex for unsafe-eval CSP violations — word order
didn't match actual error messages. Add filter for "Unexpected end of
input" from truncated WebView script loads.
2026-02-20 07:37:20 +04:00
Lawyered f2a1a2ccb5 fix(pwa): disable default navigateFallback in generated SW 2026-02-19 20:14:11 -05:00
Lawyered 7b7fe3cbdd fix(pwa): prevent stale HTML precache regression 2026-02-19 20:14:09 -05:00
Lawyered 2e267cf307 fix(runtime): block cloud fallback for local-only api routes 2026-02-19 19:13:14 -05:00
Claude 8697c543ea chore: update package-lock.json
https://claude.ai/code/session_01DDhT6Ex596eX1CtSb6mHdH
2026-02-19 22:04:39 +00:00
Claude 81ccace81b perf: add will-change hints to animated elements for GPU compositing
Add will-change: transform, opacity to dragged panels, signal modal,
and mobile warning modal. Add will-change: transform to map markers
and map popup sheet. Remove will-change via animationend/transitionend
listeners after one-shot animations complete to free GPU memory.

https://claude.ai/code/session_01DDhT6Ex596eX1CtSb6mHdH
2026-02-19 22:03:59 +00:00
Claude c170ada79c perf: add CSS contain: content to .panel for layout isolation
Add contain: content to the .panel class so the browser knows that
layout changes inside a panel don't affect siblings. The .panel-content
class already has contain: layout style for scroll performance.

Expected gain: faster layout recalculations during panel updates.

https://claude.ai/code/session_01E9FpgiebjEuUPhNt8mwX9U
2026-02-19 21:56:53 +00:00
Claude 01a65fb201 fix: internationalize verification checklist labels
Convert VERIFICATION_TEMPLATE module-level constant to
getVerificationTemplate() function to defer t() calls, and add
8 new i18n keys for checklist item labels.

https://claude.ai/code/session_018UKmgomYsVsEmJfX7Ava3v
2026-02-19 21:51:33 +00:00
Claude fe2181b92b chore: sync package-lock.json version to 2.5.0
https://claude.ai/code/session_01TfRgC5GWsv51swxRSGxxeJ
2026-02-19 21:48:23 +00:00
Claude 2e85a64712 fix: replace hardcoded English strings with i18n t() calls
Audit and fix localization coverage gaps across 12 components that
were using hardcoded English strings instead of the i18next t() system.

Components fixed:
- IntelligenceGapBadge.ts: context menu label
- InvestmentsPanel.ts: filters, table headers, sector labels, statuses
- Panel.ts: aria-label, resize tooltip, settings button
- LanguageSelector.ts: aria-label
- ServiceStatusPanel.ts: loading state, category filters, status labels
- TechEventsPanel.ts: tab labels, stats, badges
- StrategicRiskPanel.ts: risk metrics, section titles, time formatting
- RegulationPanel.ts: dashboard title, tabs, section headers, stances
- TechReadinessPanel.ts: fetching state indicators, source attribution
- InsightsPanel.ts: progress steps, empty states
- VerificationChecklist.ts: all UI labels and verdict text
- LiveNewsPanel.ts: offline/error messages

Added ~150 new translation keys to en.json and propagated them
as English placeholders to all 13 other locale files.

https://claude.ai/code/session_018UKmgomYsVsEmJfX7Ava3v
2026-02-19 21:48:19 +00:00
Claude 4121113547 perf: tree-shake unused locale files from initial bundle
Replace the static LOCALE_LOADERS map (14 explicit dynamic imports) with
import.meta.glob for lazy loading. English is now statically imported as
the always-needed fallback; all other 13 locales are loaded on demand.

- Statically import en.json so it's bundled eagerly (no extra fetch)
- Use import.meta.glob with negative pattern to lazy-load non-English
  locales only when the user actually switches language
- Add manualChunks rule to prefix lazy locale chunks with `locale-`
- Exclude locale-*.js from service worker precache via globIgnores
- Add CacheFirst runtime caching for locale files when loaded on demand

SW precache reduced from 43 entries (5587 KiB) to 29 entries (4840 KiB),
saving ~747 KiB from the initial download.

https://claude.ai/code/session_01TfRgC5GWsv51swxRSGxxeJ
2026-02-19 21:47:27 +00:00
Claude 2473c4bfd4 feat: pre-render critical CSS skeleton in index.html for instant perceived load
Inline a minimal HTML skeleton (header bar, map placeholder, panels grid)
with critical CSS directly in index.html so the page shows a structured
layout immediately instead of a blank screen while JavaScript boots.

- Dark theme skeleton with hardcoded colors matching CSS variables
- Light theme support via [data-theme="light"] selectors
- Shimmer animation on placeholder lines for visual feedback
- 6 skeleton panels in a responsive grid matching the real layout
- Map section with radial gradient matching the map background
- Skeleton is automatically replaced when App.renderLayout() sets innerHTML
- Marked aria-hidden="true" for screen reader accessibility

Expected gain: perceived load time drops to <0.5s.

https://claude.ai/code/session_01Fxk8GMRn2cEUq3ThC2a8e5
2026-02-19 21:40:39 +00:00
Claude 6c02bd2eb6 chore: sync package-lock.json with package.json version
npm install regenerated the lockfile to reflect the current
version (2.5.0) and license field from package.json.

https://claude.ai/code/session_01684qa7XvS7sf9CShqU8zNg
2026-02-19 21:34:39 +00:00
Claude 816015357a Fix setInterval clock leak in startHeaderClock()
Store the interval ID returned by setInterval in a new clockIntervalId
field and clear it in App.destroy(). Previously, the interval was never
stored or cleared, causing DOM writes to double on each Vite HMR reload.

https://claude.ai/code/session_0111CXxXM5qKR83UAdUTQDyL
2026-02-19 21:34:18 +00:00
Claude 9fd1bf293d fix: implement live-stream detection in youtubeLivePlugin dev middleware
The Vite dev plugin was hardcoding `{ videoId: null }` with a TODO,
causing LiveNewsPanel to never resolve actual live streams during local
development. Replace the stub with the same fetch-and-scrape approach
used by the production edge function (api/youtube/live.js): fetch the
channel's /live page and extract videoId + isLive from the HTML.

https://claude.ai/code/session_01684qa7XvS7sf9CShqU8zNg
2026-02-19 21:34:02 +00:00
Elie Habib bf2c0b1598 fix: replace Polymarket prod proxy with local Vite middleware plugin
Removes circular dev→prod dependency. The new polymarketPlugin()
mirrors the edge function logic locally: validates params, fetches
from gamma-api.polymarket.com, and gracefully returns [] when
Cloudflare JA3 blocks server-side TLS (expected behavior).
2026-02-20 01:33:24 +04:00
Elie Habib a39aed1d64 fix: add Desktop app (Linux), LLMs settings, and live webcams to bug report template 2026-02-20 01:30:08 +04:00
Elie Habib 58389ba440 fix: sync Cargo.toml version to 2.5.0 (was missed in release commit) 2026-02-20 01:21:18 +04:00
Elie Habib fce6c52970 release: v2.5.0 — Ollama/LM Studio local LLM support, settings tabs, keychain vault
- Ollama/LM Studio integration with auto model discovery and 4-tier fallback chain
- Settings window split into LLMs, API Keys, and Debug tabs
- Consolidated keychain vault (1 OS prompt instead of 20+)
- README expanded with privacy architecture, summarization chain docs
- CHANGELOG updated with full v2.5.0 release notes
- 5 new defense/intel RSS feeds, Koeberg nuclear plant added
2026-02-20 01:14:16 +04:00
Elie Habib e040994e9f feat: add Koeberg nuclear power plant (South Africa) to facilities map 2026-02-20 01:02:16 +04:00
Elie Habib 7dc53c0f4c feat: add 5 defense/intel RSS feeds (Military Times, Task & Purpose, USNI News, Oryx OSINT, UK MOD) 2026-02-20 01:00:55 +04:00
Elie Habib bd3b71f4d8 fix(map): clear PathLayer cache on toggle-off to prevent stale WebGL buffers
Toggling cables/pipelines off then on caused deck.gl assertion failure
because the cached PathLayer had its WebGL resources destroyed on removal.
2026-02-20 00:56:35 +04:00
Elie HabibandGitHub 20480feae4 Add Ollama support to summarization chain with shared handler factory (#124)
## Summary

Extracts shared summarization logic (CORS, validation, caching, prompt
building) into a reusable `_summarize-handler.js` factory, then uses it
to add Ollama as the first provider in the fallback chain. This reduces
code duplication across Groq, OpenRouter, and the new Ollama endpoint
while maintaining identical behavior.

**Fallback chain is now:** Ollama → Groq → OpenRouter → Browser T5

## Type of change

- [x] New feature
- [x] Refactor / code cleanup
- [x] API endpoints (`/api/*`)

## Affected areas

- [x] AI Insights / World Brief
- [x] Desktop app (Tauri)
- [x] API endpoints (`/api/*`)
- [x] Config / Settings

## Changes

### New Files
- **`api/_summarize-handler.js`** – Shared handler factory with:
- `createSummarizeHandler(providerConfig)` – Creates edge handlers for
any LLM provider
- `getCacheKey()` – Stable cache key generation (extracted from
per-provider code)
- `deduplicateHeadlines()` – Headline deduplication logic (extracted
from per-provider code)
- Unified prompt building for brief/analysis/translate modes with
tech/full variants
  - CORS, validation, caching, and error handling pipeline

- **`api/ollama-summarize.js`** – New Ollama endpoint (34 lines):
- Calls local/remote Ollama instance via OpenAI-compatible
`/v1/chat/completions`
  - Reads `OLLAMA_API_URL` and `OLLAMA_MODEL` from environment
  - Shares Redis cache with Groq/OpenRouter (same cache key strategy)
  - Returns fallback signal when unconfigured or API fails

- **`api/ollama-summarize.test.mjs`** – Unit tests for Ollama endpoint:
  - Fallback signal when `OLLAMA_API_URL` not configured
  - Success response with provider="ollama"
  - Error handling (API errors, empty responses)
  - Model selection via `OLLAMA_MODEL` env

- **`api/_summarize-handler.test.mjs`** – Unit tests for shared factory:
  - Cache key stability and variation by mode/variant/lang/geoContext
  - Headline deduplication logic
  - Handler creation with missing credentials
  - API provider calls and response shaping

- **`tests/summarization-chain.test.mjs`** – Integration tests for
fallback chain:
  - Ollama success short-circuits (no downstream calls)
  - Ollama failure → Groq success
  - Both fail → fallback signals propagate

### Modified Files

**`api/groq-summarize.js`** & **`api/openrouter-summarize.js`**
- Replaced ~290 lines of duplicated logic with single call to
`createSummarizeHandler()`
- Now thin wrappers: 26 lines (Groq) and 28 lines (OpenRouter)
- Identical behavior, zero functional changes

**`src/services/summarization.ts`**
- Updated fallback chain: `Ollama → Groq → OpenRouter → Browser T5`
- Refactored `tryGroq()` and `tryOpenRouter()` into unified
`tryApiProvider()` function
- Added `API_PROVIDERS` config array for provider ordering
- Updated `SummarizationProvider` type to include `'ollama'`

**`src-tauri/sidecar/local-api-server.mjs`**
- Added `OLLAMA_API_URL` and `OLLAMA_MODEL` to `ALLOWED_ENV_KEYS`
- Added validation for `OLLAMA_API_URL` in
`validateSecretAgainstProvider()`:
  - Probes `/v1/models` (OpenAI-compatible endpoint)
  - Falls back to `/api/tags` (native Ollama endpoint)
- Returns "Ollama endpoint verified" or "Ollama endpoint verified
(native API
2026-02-20 00:52:53 +04:00
Elie Habib 8d20069830 fix: resolve markdown lint errors in docs 2026-02-20 00:47:30 +04:00
Elie Habib 6a10ca3cc0 fix(sentry): null guard getProjection crash and add 6 noise filters
DeckGLMap: guard updateLayers/debounce/raf against null maplibreMap,
null out references in destroy() to prevent post-destroy setProps crash.

main.ts: filter contentWindow.postMessage (Facebook WebView),
vertex shader compile (GPU driver), objectStoreNames (iOS background),
Unexpected identifier https (Safari 16), _0x obfuscated vars (extensions),
WKWebView deallocated (Tauri lifecycle).
2026-02-20 00:45:42 +04:00
Elie Habib bab0974407 docs: add Ollama/local LLM coverage to community promotion guide
Add local LLM support mentions across feature descriptions, talking
points, screenshot suggestions, and changelog. New dedicated section
for Ollama/LM Studio as feature #11.
2026-02-20 00:12:15 +04:00
Elie Habib 6c3d2770f7 feat: split settings into LLMs and API Keys tabs, fix keychain vault and Ollama UX
- Split settings window into 3 tabs: LLMs (Ollama/Groq/OpenRouter),
  API Keys (data feeds), and Debug & Logs
- Add featureFilter option to RuntimeConfigPanel for rendering subsets
- Consolidate keychain to single JSON vault entry (1 macOS prompt vs 20)
- Add Ollama model discovery with /api/tags + /v1/models fallback
- Strip <think> reasoning tokens from Ollama responses
- Suppress thinking with think:false in Ollama request body
- Parallel secret verification with 15s global timeout
- Fix manual model input overlapping dropdown (CSS grid-area + hidden-input class)
- Add loading spinners to settings tab panels
- Suppress notification popups when settings window is open
- Filter embed models from Ollama dropdown
- Fix settings window black screen flash with inline dark background
2026-02-20 00:02:48 +04:00
Elie Habib eedf43e94a fix: show URL and model inputs as plaintext instead of masked password dots 2026-02-19 20:29:44 +04:00
Elie Habib 1d1b1b209f fix: harden OpenAI-compatible endpoint flow for Ollama/LM Studio 2026-02-19 20:18:13 +04:00
Elie Habib bb14f0e9a8 fix: resolve TS build errors and add missing Ollama keys to Rust keyring
- Use for...of entries() instead of index-based loops in summarization.ts
  to satisfy strict noUncheckedIndexedAccess (7 TS18048/TS2345 errors)
- Replace fragile API_PROVIDERS[1] with .find(p => p.name === groq)
- Add OLLAMA_API_URL and OLLAMA_MODEL to SUPPORTED_SECRET_KEYS in main.rs
  so keychain secrets are injected into sidecar on desktop startup
2026-02-19 19:33:40 +04:00
Claude 5cdc41712c refactor: unify summarization providers behind common interfaces
Server-side: extract shared CORS, validation, caching, prompt building,
and response shaping into api/_summarize-handler.js factory. Each
endpoint (Groq, OpenRouter, Ollama) becomes a thin wrapper calling
createSummarizeHandler() with a provider config: credentials, API URL,
model, headers, and provider label.

Client-side: replace three near-identical tryOllama/tryGroq/tryOpenRouter
functions with a single tryApiProvider() driven by an API_PROVIDERS
config array. Add runApiChain() helper that loops the chain with
progress callbacks. Simplify translateText() from three copy-pasted
blocks to a single loop over the same provider array.

- groq-summarize.js: 297 → 30 lines
- openrouter-summarize.js: 295 → 33 lines
- ollama-summarize.js: 289 → 34 lines
- summarization.ts: 336 → 239 lines
- New _summarize-handler.js: 315 lines (shared)
- Net: -566 lines of duplication removed

Adding a new LLM provider now requires only a provider config object
in the endpoint file + one entry in the API_PROVIDERS array.

Tests: 13 new tests for the shared factory (cache key, dedup, handler
creation, fallback, error casing, HTTP methods). All 42 existing tests
pass unchanged.

https://claude.ai/code/session_01AGg9fG6LZ8Y6XhvLszdfeY
2026-02-19 15:11:25 +00:00
Claude ba329e2a2a test: add Ollama provider tests across endpoint, sidecar, and chain layers
Three test files covering Ollama integration:

api/ollama-summarize.test.mjs (9 tests):
- Fallback signal when unconfigured, on API error, on empty response
- Success path with correct provider label and response shape
- Model selection via OLLAMA_MODEL env / default fallback
- Network error handling (ECONNREFUSED)
- Translate mode prompt verification

tests/summarization-chain.test.mjs (7 tests):
- Ollama success short-circuits chain (Groq never called)
- Ollama fail → Groq success fallback
- Full fallback when both unconfigured
- Provider label correctness for Ollama and Groq
- Uniform response shape across providers
- Identical fallback signal shapes

src-tauri/sidecar/local-api-server.test.mjs (8 new tests):
- OLLAMA_API_URL and OLLAMA_MODEL accepted via env-update allowlist
- Unknown keys rejected (403)
- Validation via /v1/models probe (reachable mock)
- Validation via /api/tags native fallback
- OLLAMA_MODEL pass-through validation
- Non-http protocol rejection (422)
- Auth-required behavior preserved with token

https://claude.ai/code/session_01AGg9fG6LZ8Y6XhvLszdfeY
2026-02-19 14:41:32 +00:00
Claude 3f5fa51f40 feat: add Ollama (OpenAI-compatible) local LLM summarization support
Add Ollama as the primary summarization provider for desktop builds,
sitting before Groq/OpenRouter in the fallback chain. This enables
fully local, unlimited LLM inference via Ollama's OpenAI-compatible
endpoint (/v1/chat/completions).

Changes across six layers:
- runtime-config: OLLAMA_API_URL + OLLAMA_MODEL secret keys, aiOllama
  feature toggle (default on), URL validation
- sidecar: allowlist + endpoint probe validation (tries /v1/models
  then /api/tags)
- api/ollama-summarize.js: new handler mirroring Groq/OpenRouter with
  shared Redis cache keys
- summarization.ts: tryOllama() + updated chain order in normal, beta,
  and translation paths (Ollama → Groq → OpenRouter → Browser T5)
- RuntimeConfigPanel: signup URLs + i18n help text for new keys
- desktop-readiness: aiOllama in key-backed features + readiness check

https://claude.ai/code/session_01AGg9fG6LZ8Y6XhvLszdfeY
2026-02-19 14:22:53 +00:00
Elie Habib f92de9f9da feat(i18n): localize remaining hardcoded English in 6 panel components
- CIIPanel: use existing t('components.cii.noSignals') for empty state
- CountryBriefPage: localize level badge and trend indicator labels
- ETFFlowsPanel: localize summary labels, table headers, net direction
- LiveWebcamsPanel: localize region filter tab labels
- MacroSignalsPanel: localize verdict, signal names, and bullish count
- StrategicRiskPanel: localize score levels, trend label, and trend values
2026-02-19 15:44:39 +04:00
Elie Habib a1214c9b1e feat(i18n): add theater name translations and webcam region localizations
- Add i18n keys for all 9 strategic posture theater names (Iran Theater,
  Baltic Theater, Taiwan Strait, etc.) across all 14 languages
- Wire StrategicPosturePanel to use t() for theater display names with
  fallback to English if key is missing
- Include webcam region button translations (ALL, MIDEAST, EUROPE, etc.)
  that were missing from deployed locale files

Fixes #121
2026-02-19 15:32:03 +04:00
Elie Habib 471d76223f Fix ultrawide map height mismatch and re-enable map resize 2026-02-19 15:13:24 +04:00
Elie Habib c494f5b995 fix(sentry): filter Safari "Importing a module script failed" noise
Move pattern from beforeSend extension-only check to ignoreErrors array.
All 16 events were iOS Safari with no stack trace — stale cached assets
after deploys, not actionable bugs.
2026-02-19 14:26:15 +04:00
Elie Habib 987a7b5e08 Fix wide-screen panel layout gaps and drag reorder behavior 2026-02-19 14:09:58 +04:00
Elie Habib 98797c9b02 fix: restore update link fallback and PWA nav precache 2026-02-19 13:30:06 +04:00
Elie Habib bc96da9106 feat(i18n): add Turkish language support (14th language)
Add Turkish (tr) as the 14th supported language with full translation
of all 1,134 locale keys, locale loader, tr-TR formatting, and flag
mapping. Update documentation to reflect 14 languages.
2026-02-19 13:23:08 +04:00
Elie Habib 9cff125078 feat: add llms.txt and llms-full.txt for LLM discoverability
Follow llmstxt.org standard so LLMs can understand the project.
Concise version (~5KB) and extended version (~21KB) with full
data layers, sources, and architecture details.
2026-02-19 12:37:50 +04:00
Elie Habib aa823bbb60 fix: exclude HTML from Workbox precache glob 2026-02-19 12:28:41 +04:00
Elie Habib 62a69acc11 fix(sentry): triage 3 issues — 2 noise filters, 1 beforeSend fix
- WORLDMONITOR-2A: filter AbortError from fetch abort on navigation
- WORLDMONITOR-29: broaden maplibre _layers null crash pattern in beforeSend
- WORLDMONITOR-Q: filter stale dynamic import module errors (post-deploy 404s)
2026-02-19 12:08:21 +04:00
Elie HabibandGitHub 4be7065f07 chore: fix spacing issues and formatting (#118) 2026-02-19 10:36:37 +04:00
sethandGitHub d7a6e95611 chore: fix spacing issues and formatting
- fixed incorrect spacing on diagram boxes and arrows
- formatted the tables
everything stays same, no visual changes
2026-02-18 20:54:17 -08:00
Elie Habib a851d5e8a1 release: v2.4.1 — README overhaul, sentry triage, ultra-wide layout
- Comprehensive README update: live webcams, ultra-wide layout, Linux
  AppImage, theme system, auto-updater, error tracking, responsive
  layout, virtual scrolling, 13 languages, and 8 new roadmap items
- Sentry triage: WORLDMONITOR-28 noise filter broadened for smart quotes
- Ultra-wide layout: CSS float L-shape for 2000px+ screens (#114)
- Version bump: 2.4.0 → 2.4.1
2026-02-19 08:31:35 +04:00
Elie Habib 172e7018a3 fix(sentry): broaden ignoreErrors regex for smart-quote apostrophe variants
WORLDMONITOR-28: Twitter in-app browser (iPad/iOS) injects CONFIG variable.
Existing filter used literal apostrophe which may miss Safari's U+2019.
Changed Can't → Can.t to match any apostrophe character.

Closes WORLDMONITOR-28
2026-02-19 08:23:09 +04:00
Elie Habib 18f4bb98a4 feat: ultra-wide layout — panels wrap around map on 2000px+ screens
Closes #114. On ultra-wide monitors, the map floats left (60% width,
65vh) and panels flow in an L-shape: to the right and below the map.
Uses display:contents on panels-grid so individual panels become flow
children, wrapping naturally around the CSS float. No JS changes.
2026-02-19 08:17:17 +04:00
Elie Habib 22869e8062 feat: harden desktop updater flow 2026-02-19 08:16:34 +04:00
Elie Habib ad32dd899d fix(tauri): suppress native WKWebView context menu for custom right-click menus
macOS WKWebView shows native Lookup/Translate/Copy menu on right-click,
overriding the custom "Hide Intelligence Findings" context menu.
Prevents default contextmenu on non-input elements in Tauri only.
2026-02-19 07:55:33 +04:00
Elie Habib f6e7bbbfbc chore: enforce desktop version sync 2026-02-19 07:53:15 +04:00
Elie HabibandGitHub 85e2384f0f fix: background transparency to correct colored background (#116)
## Summary

Fixes a css transparency issue when selecting languages.

Hi! First time contributing here. Found this app and thought it was
really cool. I realized it was a github repo and thought I'd try and
commit some fixes that I find. If y'all need me to update this PR in any
way let me know.

## Type of change

- [x] Bug fix
- [ ] New feature
- [ ] New data source / feed
- [ ] New map layer
- [ ] Refactor / code cleanup
- [ ] Documentation
- [ ] CI / Build / Infrastructure

## Affected areas

- [ ] Map / Globe
- [ ] News panels / RSS feeds
- [ ] AI Insights / World Brief
- [ ] Market Radar / Crypto
- [ ] Desktop app (Tauri)
- [ ] API endpoints (`/api/*`)
- [ ] Config / Settings
- [x] Other: <!-- Language dropdown-->

## Checklist

- [x] Tested on [worldmonitor.app](https://worldmonitor.app) variant
- [ ] Tested on [tech.worldmonitor.app](https://tech.worldmonitor.app)
variant (if applicable)
- [ ] New RSS feed domains added to `api/rss-proxy.js` allowlist (if
adding feeds)
- [x] No API keys or secrets committed
- [x] TypeScript compiles without errors (`npm run typecheck`)

## Screenshots
Before:

<img width="1470" height="803" alt="Screenshot 2026-02-18 at 7 00 25 PM"
src="https://github.com/user-attachments/assets/f0e2b8ce-58dc-42f0-a73f-76b7a539fecf"
/>

After:

<img width="1470" height="800" alt="Screenshot 2026-02-18 at 6 59 54 PM"
src="https://github.com/user-attachments/assets/a528fc36-45db-40d7-8463-49fb25d07438"
/>
2026-02-19 07:36:49 +04:00
Elie Habib be485ad1c2 chore: switch license to AGPL-3.0, externalize Sentry DSN
- Replace MIT with AGPL-3.0-only to enforce attribution on derivatives
- Move hardcoded Sentry DSN to VITE_SENTRY_DSN env var
- Add null-coalesce guards for i18n legend keys and SVG viewBox
- Extend Sentry ignoreErrors with 7 additional noise patterns
2026-02-19 07:24:47 +04:00
cwnicoletti 434cd120fe fix(css): replace undefined --bg-panel 2026-02-18 18:55:48 -08:00
505 changed files with 59749 additions and 16815 deletions
+21
View File
@@ -114,7 +114,28 @@ VITE_WS_RELAY_URL=
# Site variant: "full" (worldmonitor.app) or "tech" (tech.worldmonitor.app)
VITE_VARIANT=full
# Client-side Sentry DSN (optional). Leave empty to disable error reporting.
VITE_SENTRY_DSN=
# PostHog product analytics (optional). Leave empty to disable analytics.
VITE_POSTHOG_KEY=
VITE_POSTHOG_HOST=
# Map interaction mode:
# - "flat" keeps pitch/rotation disabled (2D interaction)
# - "3d" enables pitch/rotation interactions (default)
VITE_MAP_INTERACTION_MODE=3d
# ------ Desktop Cloud Fallback (Vercel) ------
# Comma-separated list of valid API keys for desktop cloud fallback.
# Generate with: openssl rand -hex 24 | sed 's/^/wm_/'
WORLDMONITOR_VALID_KEYS=
# ------ Registration DB (Convex) ------
# Convex deployment URL for email registration storage.
# Set up at: https://dashboard.convex.dev/
CONVEX_URL=
+3
View File
@@ -18,6 +18,7 @@ body:
- finance.worldmonitor.app (Finance)
- Desktop app (Windows)
- Desktop app (macOS)
- Desktop app (Linux)
validations:
required: true
@@ -37,6 +38,8 @@ body:
- Live video streams
- Desktop app (Tauri)
- Settings / API keys
- Settings / LLMs (Ollama, Groq, OpenRouter)
- Live webcams
- Other
validations:
required: true
+7
View File
@@ -94,6 +94,9 @@ jobs:
- name: Install frontend dependencies
run: npm ci
- name: Check version consistency
run: npm run version:check
- name: Bundle Node.js runtime
shell: bash
env:
@@ -189,6 +192,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VITE_VARIANT: full
VITE_DESKTOP_RUNTIME: '1'
CONVEX_URL: ${{ secrets.CONVEX_URL }}
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
APPLE_SIGNING_IDENTITY: ${{ env.APPLE_SIGNING_IDENTITY }}
@@ -212,6 +216,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VITE_VARIANT: full
VITE_DESKTOP_RUNTIME: '1'
CONVEX_URL: ${{ secrets.CONVEX_URL }}
with:
tagName: v__VERSION__
releaseName: 'World Monitor v__VERSION__'
@@ -229,6 +234,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VITE_VARIANT: tech
VITE_DESKTOP_RUNTIME: '1'
CONVEX_URL: ${{ secrets.CONVEX_URL }}
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
APPLE_SIGNING_IDENTITY: ${{ env.APPLE_SIGNING_IDENTITY }}
@@ -253,6 +259,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VITE_VARIANT: tech
VITE_DESKTOP_RUNTIME: '1'
CONVEX_URL: ${{ secrets.CONVEX_URL }}
with:
tagName: v__VERSION__-tech
releaseName: 'Tech Monitor v__VERSION__'
+9 -1
View File
@@ -1,6 +1,5 @@
node_modules/
.idea/
.planning/
dist/
.DS_Store
*.log
@@ -8,6 +7,8 @@ dist/
.env.local
.playwright-mcp/
.vercel
api/\[domain\]/v1/\[rpc\].js
api/\[\[...path\]\].js
.claude/
.cursor/
CLAUDE.md
@@ -18,6 +19,13 @@ CLAUDE.md
.windsurf/
skills/
ideas/
docs/internal/
test-results/
src-tauri/sidecar/node/*
!src-tauri/sidecar/node/.gitkeep
# AI planning session state (not source code)
.planning/
# Compiled sebuf gateway bundle (built by scripts/build-sidecar-sebuf.mjs)
api/[[][[].*.js
+1 -1
View File
@@ -6,5 +6,5 @@
"MD022": true,
"MD032": true
},
"ignores": ["node_modules/**", "dist/**", "src-tauri/target/**"]
"ignores": ["node_modules/**", "dist/**", "src-tauri/target/**", ".planning/**"]
}
+86
View File
@@ -2,6 +2,92 @@
All notable changes to World Monitor are documented here.
## [2.5.2] - 2026-02-21
### Fixed
- **QuotaExceededError handling** — detect storage quota exhaustion and stop further writes to localStorage/IndexedDB instead of silently failing; shared `markStorageQuotaExceeded()` flag across persistent-cache and utility storage
- **deck.gl null.getProjection crash** — wrap `setProps()` calls in try/catch to survive map mid-teardown races in debounced/RAF callbacks
- **MapLibre "Style is not done loading"** — guard `setFilter()` in mousemove/mouseout handlers during theme switches
- **YouTube invalid video ID** — validate video ID format (`/^[\w-]{10,12}$/`) before passing to IFrame Player constructor
- **Vercel build skip on empty SHA** — guard `ignoreCommand` against unset `VERCEL_GIT_PREVIOUS_SHA` (first deploy, force deploy) which caused `git diff` to fail and cancel builds
- **Sentry noise filters** — added 7 patterns: iOS readonly property, SW FetchEvent, toLowerCase/trim/indexOf injections, QuotaExceededError
---
## [2.5.1] - 2026-02-20
### Performance
- **Batch FRED API requests** — frontend now sends a single request with comma-separated series IDs instead of 7 parallel edge function invocations, eliminating Vercel 25s timeouts
- **Parallel UCDP page fetches** — replaced sequential loop with Promise.all for up to 12 pages, cutting fetch time from ~96s worst-case to ~8s
- **Bot protection middleware** — blocks known social-media crawlers from hitting API routes, reducing unnecessary edge function invocations
- **Extended API cache TTLs** — country-intel 12h→24h, GDELT 2h→4h, nuclear 12h→24h; Vercel ignoreCommand skips non-code deploys
### Fixed
- **Partial UCDP cache poisoning** — failed page fetches no longer silently produce incomplete results cached for 6h; partial results get 10-min TTL in both Redis and memory, with `partial: true` flag propagated to CDN cache headers
- **FRED upstream error masking** — single-series failures now return 502 instead of empty 200; batch mode surfaces per-series errors and returns 502 when all fail
- **Sentry `Load failed` filter** — widened regex from `^TypeError: Load failed$` to `^TypeError: Load failed( \(.*\))?$` to catch host-suffixed variants (e.g., gamma-api.polymarket.com)
- **Tooltip XSS hardening** — replaced `rawHtml()` with `safeHtml()` allowlist sanitizer for panel info tooltips
- **UCDP country endpoint** — added missing HTTP method guards (OPTIONS/GET)
- **Middleware exact path matching** — social preview bot allowlist uses `Set.has()` instead of `startsWith()` prefix matching
### Changed
- FRED batch API supports up to 15 comma-separated series IDs with deduplication
- Missing FRED API key returns 200 with `X-Data-Status: skipped-no-api-key` header instead of silent empty response
- LAYER_TO_SOURCE config extracted from duplicate inline mappings into shared constant
---
## [2.5.0] - 2026-02-20
### Highlights
**Local LLM Support (Ollama / LM Studio)** — Run AI summarization entirely on your own hardware with zero cloud dependency. The desktop app auto-discovers models from any OpenAI-compatible local inference server (Ollama, LM Studio, llama.cpp, vLLM) and populates a selection dropdown. A 4-tier fallback chain ensures summaries always generate: Local LLM → Groq → OpenRouter → browser-side T5. Combined with the Tauri desktop app, this enables fully air-gapped intelligence analysis where no data leaves your machine.
### Added
- **Ollama / LM Studio integration** — local AI summarization via OpenAI-compatible `/v1/chat/completions` endpoint with automatic model discovery, embedding model filtering, and fallback to manual text input
- **4-tier summarization fallback chain** — Ollama (local) → Groq (cloud) → OpenRouter (cloud) → Transformers.js T5 (browser), each with 5-second timeout before silently advancing to the next
- **Shared summarization handler factory** — all three API tiers use identical logic for headline deduplication (Jaccard >0.6), variant-aware prompting, language-aware output, and Redis caching (`summary:v3:{mode}:{variant}:{lang}:{hash}`)
- **Settings window with 3 tabs** — dedicated **LLMs** tab (Ollama endpoint/model, Groq, OpenRouter), **API Keys** tab (12+ data source credentials), and **Debug & Logs** tab (traffic log, verbose mode, log file access). Each tab runs an independent verification pipeline
- **Consolidated keychain vault** — all desktop secrets stored as a single JSON blob in one OS keychain entry (`secrets-vault`), reducing macOS Keychain authorization prompts from 20+ to exactly 1 on app startup. One-time auto-migration from individual entries with cleanup
- **Cross-window secret synchronization** — saving credentials in the Settings window immediately syncs to the main dashboard via `localStorage` broadcast, with no app restart needed
- **API key verification pipeline** — each credential is validated against its provider's actual API endpoint. Network errors (timeouts, DNS failures) soft-pass to prevent transient failures from blocking key storage; only explicit 401/403 marks a key invalid
- **Plaintext URL inputs** — endpoint URLs (Ollama API, relay URLs, model names) display as readable text instead of masked password dots in Settings
- **5 new defense/intel RSS feeds** — Military Times, Task & Purpose, USNI News, Oryx OSINT, UK Ministry of Defence
- **Koeberg nuclear power plant** — added to the nuclear facilities map layer (the only commercial reactor in Africa, Cape Town, South Africa)
- **Privacy & Offline Architecture** documentation — README now details the three privacy levels: full cloud, desktop with cloud APIs, and air-gapped local with Ollama
- **AI Summarization Chain** documentation — README includes provider fallback flow diagram and detailed explanation of headline deduplication, variant-aware prompting, and cross-user cache deduplication
### Changed
- AI fallback chain now starts with Ollama (local) before cloud providers
- Feature toggles increased from 14 to 15 (added AI/Ollama)
- Desktop architecture uses consolidated vault instead of per-key keychain entries
- README expanded with ~85 lines of new content covering local LLM support, privacy architecture, summarization chain internals, and desktop readiness framework
### Fixed
- URL and model fields in Settings display as plaintext instead of masked password dots
- OpenAI-compatible endpoint flow hardened for Ollama/LM Studio response format differences (thinking tokens, missing `choices` array edge cases)
- Sentry null guard for `getProjection()` crash with 6 additional noise filters
- PathLayer cache cleared on layer toggle-off to prevent stale WebGL buffer rendering
---
## [2.4.1] - 2026-02-19
### Fixed
- **Map PathLayer cache**: Clear PathLayer on toggle-off to prevent stale WebGL buffers
- **Sentry noise**: Null guard for `getProjection()` crash and 6 additional noise filters
- **Markdown docs**: Resolve lint errors in documentation files
---
## [2.4.0] - 2026-02-19
### Added
+119
View File
@@ -0,0 +1,119 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in the
World Monitor community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, caste, color, religion, or sexual
identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our
community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall
community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or advances of
any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address,
without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Scope
This Code of Conduct applies within all community spaces (GitHub issues, pull
requests, discussions, and any associated communication channels) and also
applies when an individual is officially representing the community in public
spaces.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the project maintainer at **[GitHub Issues](https://github.com/koala73/worldmonitor/issues)** or by contacting the
repository owner directly through GitHub.
All complaints will be reviewed and investigated promptly and fairly. The project
team is obligated to maintain confidentiality with regard to the reporter of an
incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of
actions.
**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or permanent
ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the
community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.1, available at
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
Community Impact Guidelines were inspired by
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
For answers to common questions about this code of conduct, see the FAQ at
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
[https://www.contributor-covenant.org/translations][translations].
[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations
+301
View File
@@ -0,0 +1,301 @@
# Contributing to World Monitor
Thank you for your interest in contributing to World Monitor! This project thrives on community contributions — whether it's code, data sources, documentation, or bug reports.
## Table of Contents
- [Architecture Overview](#architecture-overview)
- [Getting Started](#getting-started)
- [Development Setup](#development-setup)
- [How to Contribute](#how-to-contribute)
- [Pull Request Process](#pull-request-process)
- [AI-Assisted Development](#ai-assisted-development)
- [Coding Standards](#coding-standards)
- [Working with Sebuf (RPC Framework)](#working-with-sebuf-rpc-framework)
- [Adding Data Sources](#adding-data-sources)
- [Adding RSS Feeds](#adding-rss-feeds)
- [Reporting Bugs](#reporting-bugs)
- [Feature Requests](#feature-requests)
- [Code of Conduct](#code-of-conduct)
## Architecture Overview
World Monitor is a real-time OSINT dashboard built with **Vanilla TypeScript** (no UI framework), **MapLibre GL + deck.gl** for map rendering, and a custom Proto-first RPC framework called **Sebuf** for all API communication.
### Key Technologies
| Technology | Purpose |
|---|---|
| **TypeScript** | All code — frontend, edge functions, and handlers |
| **Vite** | Build tool and dev server |
| **Sebuf** | Proto-first HTTP RPC framework for typed API contracts |
| **Protobuf / Buf** | Service and message definitions across 17 domains |
| **MapLibre GL** | Base map rendering (tiles, globe mode, camera) |
| **deck.gl** | WebGL overlay layers (scatterplot, geojson, arcs, heatmaps) |
| **d3** | Charts, sparklines, and data visualization |
| **Vercel Edge Functions** | Serverless API gateway |
| **Tauri v2** | Desktop app (Windows, macOS, Linux) |
| **Convex** | Minimal backend (beta interest registration only) |
| **Playwright** | End-to-end and visual regression testing |
### Variant System
The codebase produces three app variants from the same source, each targeting a different audience:
| Variant | Command | Focus |
|---|---|---|
| `full` | `npm run dev` | Geopolitics, military, conflicts, infrastructure |
| `tech` | `npm run dev:tech` | Startups, AI/ML, cloud, cybersecurity |
| `finance` | `npm run dev:finance` | Markets, trading, central banks, commodities |
Variants share all code but differ in default panels, map layers, and RSS feeds. Variant configs live in `src/config/variants/`.
### Directory Structure
| Directory | Purpose |
|---|---|
| `src/components/` | UI components — Panel subclasses, map, modals (~50 panels) |
| `src/services/` | Data fetching modules — sebuf client wrappers, AI, signal analysis |
| `src/config/` | Static data and variant configs (feeds, geo, military, pipelines, ports) |
| `src/generated/` | Auto-generated sebuf client + server stubs (**do not edit by hand**) |
| `src/types/` | TypeScript type definitions |
| `src/locales/` | i18n JSON files (14 languages) |
| `src/workers/` | Web Workers for analysis |
| `server/` | Sebuf handler implementations for all 17 domain services |
| `api/` | Vercel Edge Functions (sebuf gateway + legacy endpoints) |
| `proto/` | Protobuf service and message definitions |
| `data/` | Static JSON datasets |
| `docs/` | Documentation + generated OpenAPI specs |
| `src-tauri/` | Tauri v2 Rust app + Node.js sidecar for desktop builds |
| `e2e/` | Playwright end-to-end tests |
| `scripts/` | Build and packaging scripts |
## Getting Started
1. **Fork** the repository on GitHub
2. **Clone** your fork locally:
```bash
git clone https://github.com/<your-username>/worldmonitor.git
cd worldmonitor
```
3. **Create a branch** for your work:
```bash
git checkout -b feature/your-feature-name
```
## Development Setup
```bash
# Install everything (buf CLI, sebuf plugins, npm deps, Playwright browsers)
make install
# Start the development server (full variant, default)
npm run dev
# Start other variants
npm run dev:tech
npm run dev:finance
# Run type checking
npm run typecheck
# Run tests
npm run test:data # Data integrity tests
npm run test:e2e # Playwright end-to-end tests
# Production build (per variant)
npm run build # full
npm run build:tech
npm run build:finance
```
The dev server runs at `http://localhost:3000`. Run `make help` to see all available make targets.
### Environment Variables (Optional)
For full functionality, copy `.env.example` to `.env.local` and fill in the API keys you need. The app runs without any API keys — external data sources will simply be unavailable.
See [API Dependencies](docs/DOCUMENTATION.md#api-dependencies) for the full list.
## How to Contribute
### Types of Contributions We Welcome
- **Bug fixes** — found something broken? Fix it!
- **New data layers** — add new geospatial data sources to the map
- **RSS feeds** — expand our 100+ feed collection with quality sources
- **UI/UX improvements** — make the dashboard more intuitive
- **Performance optimizations** — faster loading, better caching
- **Documentation** — improve docs, add examples, fix typos
- **Accessibility** — make the dashboard usable by everyone
- **Internationalization** — help make World Monitor available in more languages
- **Tests** — add unit or integration tests
### What We're Especially Looking For
- New data layers (see [Adding Data Sources](#adding-data-sources))
- Feed quality improvements and new RSS sources
- Mobile responsiveness improvements
- Performance optimizations for the map rendering pipeline
- Better anomaly detection algorithms
## Pull Request Process
1. **Update documentation** if your change affects the public API or user-facing behavior
2. **Run type checking** before submitting: `npm run typecheck`
3. **Test your changes** locally with at least the `full` variant, and any other variant your change affects
4. **Keep PRs focused** — one feature or fix per pull request
5. **Write a clear description** explaining what your PR does and why
6. **Link related issues** if applicable
### PR Title Convention
Use a descriptive title that summarizes the change:
- `feat: add earthquake magnitude filtering to map layer`
- `fix: resolve RSS feed timeout for Al Jazeera`
- `docs: update API dependencies section`
- `perf: optimize marker clustering at low zoom levels`
- `refactor: extract threat classifier into separate module`
### Review Process
- All PRs require review from a maintainer before merging
- Maintainers may request changes — this is normal and collaborative
- Once approved, a maintainer will merge your PR
## AI-Assisted Development
We fully embrace AI-assisted development. Many of our own PRs are labeled with the LLM that helped produce them (e.g., `claude`, `codex`, `cursor`), and contributors are welcome to use any AI tools they find helpful.
That said, **all code is held to the same quality bar regardless of how it was written**. AI-generated code will be reviewed with the same scrutiny as human-written code. Contributors are responsible for understanding and being able to explain every line they submit. Blindly pasting LLM output without review is discouraged — treat AI as a collaborator, not a replacement for your own judgement.
## Coding Standards
### TypeScript
- Use TypeScript for all new code
- Avoid `any` types — use proper typing or `unknown` with type guards
- Export interfaces/types for public APIs
- Use meaningful variable and function names
### Code Style
- Follow the existing code style in the repository
- Use `const` by default, `let` when reassignment is needed
- Prefer functional patterns (map, filter, reduce) over imperative loops
- Keep functions focused — one responsibility per function
- Add JSDoc comments for exported functions and complex logic
### File Organization
- Static layer/geo data and variant configs go in `src/config/`
- Sebuf handler implementations go in `server/worldmonitor/{domain}/v1/`
- Edge function gateway and legacy endpoints go in `api/`
- UI components (panels, map, modals) go in `src/components/`
- Service modules (data fetching, client wrappers) go in `src/services/`
- Proto definitions go in `proto/worldmonitor/{domain}/v1/`
## Working with Sebuf (RPC Framework)
Sebuf is the project's custom Proto-first HTTP RPC framework — a lightweight alternative to gRPC-Web. All API communication between client and server uses Sebuf.
### How It Works
1. **Proto definitions** in `proto/worldmonitor/{domain}/v1/` define services and messages
2. **Code generation** (`make generate`) produces:
- TypeScript clients in `src/generated/client/` (e.g., `MarketServiceClient`)
- Server route factories in `src/generated/server/` (e.g., `createMarketServiceRoutes`)
3. **Handlers** in `server/worldmonitor/{domain}/v1/handler.ts` implement the service interface
4. **Gateway** in `api/[domain]/v1/[rpc].ts` registers all handlers and routes requests
5. **Clients** in `src/services/{domain}/index.ts` wrap the generated client for app use
### Adding a New RPC Method
1. Add the method to the `.proto` service definition
2. Run `make generate` to regenerate client/server stubs
3. Implement the handler method in the domain's `handler.ts`
4. The client stub is auto-generated — use it from `src/services/{domain}/`
Use `make lint` to lint proto files and `make breaking` to check for breaking changes against main.
### Proto Conventions
- **Time fields**: Use `int64` (Unix epoch milliseconds), not `google.protobuf.Timestamp`
- **int64 encoding**: Apply `[(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER]` on time fields so TypeScript receives `number` instead of `string`
- **HTTP annotations**: Every RPC method needs `option (sebuf.http.config) = { path: "...", method: POST }`
### Proto Codegen Requirements
Run `make install` to install everything automatically, or install individually:
```bash
make install-buf # Install buf CLI (requires Go)
make install-plugins # Install sebuf protoc-gen plugins (requires Go)
```
## Adding Data Sources
To add a new data layer to the map:
1. **Define the data source** — identify the API or dataset you want to integrate
2. **Add the proto service** (if the data needs a backend proxy) — define messages and RPC methods in `proto/worldmonitor/{domain}/v1/`
3. **Generate stubs** — run `make generate`
4. **Implement the handler** in `server/worldmonitor/{domain}/v1/`
5. **Register the handler** in `api/[domain]/v1/[rpc].ts` and `vite.config.ts` (for local dev)
6. **Create the service module** in `src/services/{domain}/` wrapping the generated client
7. **Add the layer config** and implement the map renderer following existing layer patterns
8. **Add to layer toggles** — make it toggleable in the UI
9. **Document the source** — add it to `docs/DOCUMENTATION.md`
For endpoints that deal with non-JSON payloads (XML feeds, binary data, HTML embeds), you can add a standalone Edge Function in `api/` instead of Sebuf. For anything returning JSON, prefer Sebuf — the typed contracts are always worth it.
### Data Source Requirements
- Must be freely accessible (no paid-only APIs for core functionality)
- Must have a permissive license or be public government data
- Should update at least daily for real-time relevance
- Must include geographic coordinates or be geo-locatable
## Adding RSS Feeds
To add new RSS feeds:
1. Verify the feed is reliable and actively maintained
2. Assign a **source tier** (1-4) based on editorial reliability
3. Flag any **state affiliation** or **propaganda risk**
4. Categorize the feed (geopolitics, defense, energy, tech, etc.)
5. Test that the feed parses correctly through the RSS proxy
## Reporting Bugs
When filing a bug report, please include:
- **Description** — clear description of the issue
- **Steps to reproduce** — how to trigger the bug
- **Expected behavior** — what should happen
- **Actual behavior** — what actually happens
- **Screenshots** — if applicable
- **Browser/OS** — your environment details
- **Console errors** — any relevant browser console output
Use the [Bug Report issue template](https://github.com/koala73/worldmonitor/issues/new/choose) when available.
## Feature Requests
We welcome feature ideas! When suggesting a feature:
- **Describe the problem** it solves
- **Propose a solution** with as much detail as possible
- **Consider alternatives** you've thought about
- **Provide context** — who would benefit from this feature?
Use the [Feature Request issue template](https://github.com/koala73/worldmonitor/issues/new/choose) when available.
## Code of Conduct
This project follows the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior through GitHub issues or by contacting the repository owner.
---
Thank you for helping make World Monitor better! 🌍
+665 -17
View File
@@ -1,21 +1,669 @@
MIT License
World Monitor — Real-time global intelligence dashboard
Copyright (C) 2024-2026 Elie Habib
Copyright (c) 2025-2026 Elie Habib
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
+72
View File
@@ -0,0 +1,72 @@
.PHONY: help lint generate breaking format check clean deps install install-buf install-plugins install-npm install-playwright
.DEFAULT_GOAL := help
# Variables
PROTO_DIR := proto
GEN_CLIENT_DIR := src/generated/client
GEN_SERVER_DIR := src/generated/server
DOCS_API_DIR := docs/api
# Go install settings
GO_PROXY := GOPROXY=direct
GO_PRIVATE := GOPRIVATE=github.com/SebastienMelki
GO_INSTALL := $(GO_PROXY) $(GO_PRIVATE) go install
# Required tool versions
BUF_VERSION := v1.64.0
SEBUF_VERSION := v0.7.0
help: ## Show this help message
@echo 'Usage: make [target]'
@echo ''
@echo 'Targets:'
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf " %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
install: install-buf install-plugins install-npm install-playwright deps ## Install everything (buf, sebuf plugins, npm deps, proto deps, browsers)
install-buf: ## Install buf CLI
@if command -v buf >/dev/null 2>&1; then \
echo "buf already installed: $$(buf --version)"; \
else \
echo "Installing buf..."; \
$(GO_INSTALL) github.com/bufbuild/buf/cmd/buf@$(BUF_VERSION); \
echo "buf installed!"; \
fi
install-plugins: ## Install sebuf protoc plugins (requires Go)
@echo "Installing sebuf protoc plugins $(SEBUF_VERSION)..."
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-ts-client@$(SEBUF_VERSION)
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-ts-server@$(SEBUF_VERSION)
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-openapiv3@$(SEBUF_VERSION)
@echo "Plugins installed!"
install-npm: ## Install npm dependencies
npm install
install-playwright: ## Install Playwright browsers for e2e tests
npx playwright install chromium
deps: ## Install/update buf proto dependencies
cd $(PROTO_DIR) && buf dep update
lint: ## Lint protobuf files
cd $(PROTO_DIR) && buf lint
generate: clean ## Generate code from proto definitions
@mkdir -p $(GEN_CLIENT_DIR) $(GEN_SERVER_DIR) $(DOCS_API_DIR)
cd $(PROTO_DIR) && buf generate
@echo "Code generation complete!"
breaking: ## Check for breaking changes against main
cd $(PROTO_DIR) && buf breaking --against '.git#branch=main,subdir=proto'
format: ## Format protobuf files
cd $(PROTO_DIR) && buf format -w
check: lint generate ## Run all checks (lint + generate)
clean: ## Clean generated files
@rm -rf $(GEN_CLIENT_DIR)
@rm -rf $(GEN_SERVER_DIR)
@rm -rf $(DOCS_API_DIR)
@echo "Clean complete!"
+618 -250
View File
File diff suppressed because it is too large Load Diff
+100
View File
@@ -0,0 +1,100 @@
# Security Policy
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| main | :white_check_mark: |
Only the latest version on the `main` branch is actively maintained and receives security updates.
## Reporting a Vulnerability
**Please do NOT report security vulnerabilities through public GitHub issues.**
If you discover a security vulnerability in World Monitor, please report it responsibly:
1. **GitHub Private Vulnerability Reporting**: Use [GitHub's private vulnerability reporting](https://github.com/koala73/worldmonitor/security/advisories/new) to submit your report directly through the repository.
2. **Direct Contact**: Alternatively, reach out to the repository owner [@koala73](https://github.com/koala73) directly through GitHub.
### What to Include
- A description of the vulnerability and its potential impact
- Steps to reproduce the issue
- Affected components (edge functions, client-side code, data layers, etc.)
- Any potential fixes or mitigations you've identified
### Response Timeline
- **Acknowledgment**: Within 48 hours of your report
- **Initial Assessment**: Within 1 week
- **Fix/Patch**: Depending on severity, critical issues will be prioritized
### What to Expect
- You will receive an acknowledgment of your report
- We will work with you to understand and validate the issue
- We will keep you informed of progress toward a fix
- Credit will be given to reporters in the fix commit (unless you prefer anonymity)
## Security Considerations
World Monitor is a client-side intelligence dashboard that aggregates publicly available data. Here are the key security areas:
### API Keys & Secrets
- All API keys are stored server-side in Vercel Edge Functions
- No API keys should ever be committed to the repository
- Environment variables (`.env.local`) are gitignored
- The RSS proxy uses domain allowlisting to prevent SSRF
### Edge Functions & Sebuf Handlers
- All 17 domain APIs are served through Sebuf (a Proto-first RPC framework) via Vercel Edge Functions
- Edge functions and handlers should validate/sanitize all input
- CORS headers are configured per-function
- Rate limiting and circuit breakers protect against abuse
### Client-Side Security
- No sensitive data is stored in localStorage or sessionStorage
- External content (RSS feeds, news) is sanitized before rendering
- Map data layers use trusted, vetted data sources
### Data Sources
- World Monitor aggregates publicly available OSINT data
- No classified or restricted data sources are used
- State-affiliated sources are flagged with propaganda risk ratings
- All data is consumed read-only — the platform does not modify upstream sources
## Scope
The following are **in scope** for security reports:
- Vulnerabilities in the World Monitor codebase
- Edge function security issues (SSRF, injection, auth bypass)
- XSS or content injection through RSS feeds or external data
- API key exposure or secret leakage
- Dependency vulnerabilities with a viable attack vector
The following are **out of scope**:
- Vulnerabilities in third-party services we consume (report to the upstream provider)
- Social engineering attacks
- Denial of service attacks
- Issues in forked copies of the repository
- Security issues in user-provided environment configurations
## Best Practices for Contributors
- Never commit API keys, tokens, or secrets
- Use environment variables for all sensitive configuration
- Sanitize external input in edge functions
- Keep dependencies updated — run `npm audit` regularly
- Follow the principle of least privilege for API access
---
Thank you for helping keep World Monitor and its users safe! 🔒
+138
View File
@@ -0,0 +1,138 @@
/**
* Vercel edge function for sebuf RPC routes.
*
* Matches /api/{domain}/v1/{rpc} via Vercel dynamic segment routing.
* CORS headers are applied to every response (200, 204, 403, 404).
*/
export const config = { runtime: 'edge' };
import { createRouter } from '../../../server/router';
import { getCorsHeaders, isDisallowedOrigin } from '../../../server/cors';
// @ts-expect-error — JS module, no declaration file
import { validateApiKey } from '../../_api-key.js';
import { mapErrorToResponse } from '../../../server/error-mapper';
import { createSeismologyServiceRoutes } from '../../../src/generated/server/worldmonitor/seismology/v1/service_server';
import { seismologyHandler } from '../../../server/worldmonitor/seismology/v1/handler';
import { createWildfireServiceRoutes } from '../../../src/generated/server/worldmonitor/wildfire/v1/service_server';
import { wildfireHandler } from '../../../server/worldmonitor/wildfire/v1/handler';
import { createClimateServiceRoutes } from '../../../src/generated/server/worldmonitor/climate/v1/service_server';
import { climateHandler } from '../../../server/worldmonitor/climate/v1/handler';
import { createPredictionServiceRoutes } from '../../../src/generated/server/worldmonitor/prediction/v1/service_server';
import { predictionHandler } from '../../../server/worldmonitor/prediction/v1/handler';
import { createDisplacementServiceRoutes } from '../../../src/generated/server/worldmonitor/displacement/v1/service_server';
import { displacementHandler } from '../../../server/worldmonitor/displacement/v1/handler';
import { createAviationServiceRoutes } from '../../../src/generated/server/worldmonitor/aviation/v1/service_server';
import { aviationHandler } from '../../../server/worldmonitor/aviation/v1/handler';
import { createResearchServiceRoutes } from '../../../src/generated/server/worldmonitor/research/v1/service_server';
import { researchHandler } from '../../../server/worldmonitor/research/v1/handler';
import { createUnrestServiceRoutes } from '../../../src/generated/server/worldmonitor/unrest/v1/service_server';
import { unrestHandler } from '../../../server/worldmonitor/unrest/v1/handler';
import { createConflictServiceRoutes } from '../../../src/generated/server/worldmonitor/conflict/v1/service_server';
import { conflictHandler } from '../../../server/worldmonitor/conflict/v1/handler';
import { createMaritimeServiceRoutes } from '../../../src/generated/server/worldmonitor/maritime/v1/service_server';
import { maritimeHandler } from '../../../server/worldmonitor/maritime/v1/handler';
import { createCyberServiceRoutes } from '../../../src/generated/server/worldmonitor/cyber/v1/service_server';
import { cyberHandler } from '../../../server/worldmonitor/cyber/v1/handler';
import { createEconomicServiceRoutes } from '../../../src/generated/server/worldmonitor/economic/v1/service_server';
import { economicHandler } from '../../../server/worldmonitor/economic/v1/handler';
import { createInfrastructureServiceRoutes } from '../../../src/generated/server/worldmonitor/infrastructure/v1/service_server';
import { infrastructureHandler } from '../../../server/worldmonitor/infrastructure/v1/handler';
import { createMarketServiceRoutes } from '../../../src/generated/server/worldmonitor/market/v1/service_server';
import { marketHandler } from '../../../server/worldmonitor/market/v1/handler';
import { createNewsServiceRoutes } from '../../../src/generated/server/worldmonitor/news/v1/service_server';
import { newsHandler } from '../../../server/worldmonitor/news/v1/handler';
import { createIntelligenceServiceRoutes } from '../../../src/generated/server/worldmonitor/intelligence/v1/service_server';
import { intelligenceHandler } from '../../../server/worldmonitor/intelligence/v1/handler';
import { createMilitaryServiceRoutes } from '../../../src/generated/server/worldmonitor/military/v1/service_server';
import { militaryHandler } from '../../../server/worldmonitor/military/v1/handler';
import type { ServerOptions } from '../../../src/generated/server/worldmonitor/seismology/v1/service_server';
const serverOptions: ServerOptions = { onError: mapErrorToResponse };
const allRoutes = [
...createSeismologyServiceRoutes(seismologyHandler, serverOptions),
...createWildfireServiceRoutes(wildfireHandler, serverOptions),
...createClimateServiceRoutes(climateHandler, serverOptions),
...createPredictionServiceRoutes(predictionHandler, serverOptions),
...createDisplacementServiceRoutes(displacementHandler, serverOptions),
...createAviationServiceRoutes(aviationHandler, serverOptions),
...createResearchServiceRoutes(researchHandler, serverOptions),
...createUnrestServiceRoutes(unrestHandler, serverOptions),
...createConflictServiceRoutes(conflictHandler, serverOptions),
...createMaritimeServiceRoutes(maritimeHandler, serverOptions),
...createCyberServiceRoutes(cyberHandler, serverOptions),
...createEconomicServiceRoutes(economicHandler, serverOptions),
...createInfrastructureServiceRoutes(infrastructureHandler, serverOptions),
...createMarketServiceRoutes(marketHandler, serverOptions),
...createNewsServiceRoutes(newsHandler, serverOptions),
...createIntelligenceServiceRoutes(intelligenceHandler, serverOptions),
...createMilitaryServiceRoutes(militaryHandler, serverOptions),
];
const router = createRouter(allRoutes);
export default async function handler(request: Request): Promise<Response> {
// Origin check first — skip CORS headers for disallowed origins (M-2 fix)
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
let corsHeaders: Record<string, string>;
try {
corsHeaders = getCorsHeaders(request);
} catch {
corsHeaders = { 'Access-Control-Allow-Origin': '*' };
}
// OPTIONS preflight
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
// API key validation (origin-aware)
const keyCheck = validateApiKey(request);
if (keyCheck.required && !keyCheck.valid) {
return new Response(JSON.stringify({ error: keyCheck.error }), {
status: 401,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
// Route matching
const matchedHandler = router.match(request);
if (!matchedHandler) {
return new Response(JSON.stringify({ error: 'Not found' }), {
status: 404,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
// Execute handler with top-level error boundary (H-1 fix)
let response: Response;
try {
response = await matchedHandler(request);
} catch (err) {
console.error('[gateway] Unhandled handler error:', err);
response = new Response(JSON.stringify({ message: 'Internal server error' }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
// Merge CORS headers into response
const mergedHeaders = new Headers(response.headers);
for (const [key, value] of Object.entries(corsHeaders)) {
mergedHeaders.set(key, value);
}
return new Response(response.body, {
status: response.status,
statusText: response.statusText,
headers: mergedHeaders,
});
}
+30
View File
@@ -0,0 +1,30 @@
const DESKTOP_ORIGIN_PATTERNS = [
/^https?:\/\/tauri\.localhost(:\d+)?$/,
/^https?:\/\/[a-z0-9-]+\.tauri\.localhost(:\d+)?$/i,
/^tauri:\/\/localhost$/,
/^asset:\/\/localhost$/,
];
function isDesktopOrigin(origin) {
return Boolean(origin) && DESKTOP_ORIGIN_PATTERNS.some(p => p.test(origin));
}
export function validateApiKey(req) {
const key = req.headers.get('X-WorldMonitor-Key');
const origin = req.headers.get('Origin') || '';
if (isDesktopOrigin(origin)) {
if (!key) return { valid: false, required: true, error: 'API key required for desktop access' };
const validKeys = (process.env.WORLDMONITOR_VALID_KEYS || '').split(',').filter(Boolean);
if (!validKeys.includes(key)) return { valid: false, required: true, error: 'Invalid API key' };
return { valid: true, required: true };
}
if (key) {
const validKeys = (process.env.WORLDMONITOR_VALID_KEYS || '').split(',').filter(Boolean);
if (!validKeys.includes(key)) return { valid: false, required: true, error: 'Invalid API key' };
return { valid: true, required: true };
}
return { valid: false, required: false };
}
-53
View File
@@ -1,53 +0,0 @@
const statsByEndpoint = new Map();
const MAX_ENDPOINTS = 128;
const LOG_EVERY = Math.max(0, Number(process.env.CACHE_TELEMETRY_LOG_EVERY || 200));
function cleanupOldEndpoints() {
if (statsByEndpoint.size <= MAX_ENDPOINTS) return;
const entries = Array.from(statsByEndpoint.entries())
.sort((a, b) => a[1].lastSeen - b[1].lastSeen);
const overflow = statsByEndpoint.size - MAX_ENDPOINTS;
for (let i = 0; i < overflow; i++) {
statsByEndpoint.delete(entries[i][0]);
}
}
export function recordCacheTelemetry(endpoint, outcome) {
if (!endpoint || !outcome) return;
const now = Date.now();
const current = statsByEndpoint.get(endpoint) || {
total: 0,
outcomes: {},
firstSeen: now,
lastSeen: now,
};
current.total += 1;
current.outcomes[outcome] = (current.outcomes[outcome] || 0) + 1;
current.lastSeen = now;
statsByEndpoint.set(endpoint, current);
cleanupOldEndpoints();
if (LOG_EVERY > 0 && current.total % LOG_EVERY === 0) {
console.log(`[CacheTelemetry] ${endpoint} total=${current.total} outcomes=${JSON.stringify(current.outcomes)}`);
}
}
export function getCacheTelemetrySnapshot() {
const endpoints = Array.from(statsByEndpoint.entries())
.sort((a, b) => b[1].lastSeen - a[1].lastSeen)
.map(([endpoint, stats]) => ({
endpoint,
total: stats.total,
outcomes: stats.outcomes,
firstSeen: new Date(stats.firstSeen).toISOString(),
lastSeen: new Date(stats.lastSeen).toISOString(),
}));
return {
generatedAt: new Date().toISOString(),
endpointCount: endpoints.length,
endpoints,
note: 'In-memory per instance telemetry (resets on cold start).',
};
}
+2 -2
View File
@@ -1,6 +1,6 @@
const ALLOWED_ORIGIN_PATTERNS = [
/^https:\/\/(.*\.)?worldmonitor\.app$/,
/^https:\/\/worldmonitor-[a-z0-9-]+-elie-habib-projects\.vercel\.app$/,
/^https:\/\/worldmonitor-[a-z0-9-]+-elie-[a-z0-9]+\.vercel\.app$/,
/^https?:\/\/localhost(:\d+)?$/,
/^https?:\/\/127\.0\.0\.1(:\d+)?$/,
/^https?:\/\/tauri\.localhost(:\d+)?$/,
@@ -19,7 +19,7 @@ export function getCorsHeaders(req, methods = 'GET, OPTIONS') {
return {
'Access-Control-Allow-Origin': allowOrigin,
'Access-Control-Allow-Methods': methods,
'Access-Control-Allow-Headers': 'Content-Type',
'Access-Control-Allow-Headers': 'Content-Type, Authorization, X-WorldMonitor-Key',
'Access-Control-Max-Age': '86400',
'Vary': 'Origin',
};
-61
View File
@@ -1,61 +0,0 @@
export function createIpRateLimiter({
limit,
windowMs,
maxEntries = 5000,
cleanupIntervalMs = 30 * 1000,
}) {
const records = new Map();
let lastCleanupAt = 0;
function cleanup(now) {
if (now - lastCleanupAt < cleanupIntervalMs && records.size <= maxEntries) {
return;
}
lastCleanupAt = now;
const cutoff = now - windowMs;
for (const [ip, record] of records) {
if (record.windowStart < cutoff) {
records.delete(ip);
}
}
if (records.size <= maxEntries) {
return;
}
const overflow = records.size - maxEntries;
const oldest = Array.from(records.entries())
.sort((a, b) => a[1].windowStart - b[1].windowStart);
for (let i = 0; i < overflow; i++) {
const entry = oldest[i];
if (!entry) break;
records.delete(entry[0]);
}
}
function check(ip) {
const now = Date.now();
cleanup(now);
const key = (ip || 'unknown').trim() || 'unknown';
const record = records.get(key);
if (!record || now - record.windowStart > windowMs) {
records.set(key, { count: 1, windowStart: now });
return true;
}
if (record.count >= limit) {
return false;
}
record.count += 1;
return true;
}
return {
check,
size: () => records.size,
};
}
-185
View File
@@ -1,185 +0,0 @@
const isSidecar = (process.env.LOCAL_API_MODE || '').includes('sidecar');
// ── In-memory cache (desktop/sidecar) ──
const mem = new Map();
let persistPath = null;
let persistTimer = null;
let persistInFlight = false;
let persistQueued = false;
let loaded = false;
const MAX_PERSIST_ENTRIES = Math.max(100, Number(process.env.LOCAL_API_CACHE_PERSIST_MAX || 5000));
async function ensureDesktopCache() {
if (loaded) return;
loaded = true;
try {
const { join } = await import('node:path');
const { readFileSync } = await import('node:fs');
const dir = process.env.LOCAL_API_RESOURCE_DIR || '.';
persistPath = join(dir, 'api-cache.json');
const data = JSON.parse(readFileSync(persistPath, 'utf8'));
const now = Date.now();
for (const [k, entry] of Object.entries(data)) {
if (entry.expiresAt > now) mem.set(k, entry);
}
console.log(`[Cache] Loaded ${mem.size} entries from disk`);
} catch {
// File doesn't exist yet
}
setInterval(() => {
const now = Date.now();
for (const [k, v] of mem) {
if (v.expiresAt <= now) mem.delete(k);
}
}, 60_000).unref?.();
}
function buildPersistSnapshot() {
const now = Date.now();
const payload = Object.create(null);
let kept = 0;
for (const [key, entry] of mem) {
if (!entry || entry.expiresAt <= now) continue;
payload[key] = entry;
kept += 1;
if (kept >= MAX_PERSIST_ENTRIES) break;
}
return payload;
}
async function persistToDisk() {
if (!persistPath) return;
if (persistInFlight) {
persistQueued = true;
return;
}
persistInFlight = true;
try {
const snapshot = buildPersistSnapshot();
const json = JSON.stringify(snapshot);
const { writeFile, rename } = await import('node:fs/promises');
const tmp = persistPath + '.tmp';
await writeFile(tmp, json, 'utf8');
await rename(tmp, persistPath);
} catch (err) {
console.warn('[Cache] Persist error:', err.message);
} finally {
persistInFlight = false;
if (persistQueued) {
persistQueued = false;
void persistToDisk();
}
}
}
function debouncedPersist() {
if (!persistPath) return;
clearTimeout(persistTimer);
persistTimer = setTimeout(() => {
void persistToDisk();
}, 2000);
if (persistTimer?.unref) persistTimer.unref();
}
// ── Redis (cloud/Vercel) ──
let RedisClass = null;
let redis = null;
let redisInitFailed = false;
export async function getRedis() {
if (isSidecar) return null;
if (redis) return redis;
if (redisInitFailed) return null;
const url = process.env.UPSTASH_REDIS_REST_URL;
const token = process.env.UPSTASH_REDIS_REST_TOKEN;
if (!url || !token) return null;
try {
if (!RedisClass) {
const mod = await import('@upstash/redis');
RedisClass = mod.Redis;
}
redis = new RedisClass({ url, token });
return redis;
} catch (err) {
redisInitFailed = true;
console.warn('[Cache] Redis init failed:', err.message);
return null;
}
}
// ── Shared API ──
export async function getCachedJson(key) {
if (isSidecar) {
await ensureDesktopCache();
const entry = mem.get(key);
if (!entry) return null;
if (entry.expiresAt <= Date.now()) {
mem.delete(key);
return null;
}
return entry.value;
}
const r = await getRedis();
if (!r) return null;
try {
return await r.get(key);
} catch (err) {
console.warn('[Cache] Read failed:', err.message);
return null;
}
}
export async function setCachedJson(key, value, ttlSeconds) {
if (isSidecar) {
await ensureDesktopCache();
mem.set(key, { value, expiresAt: Date.now() + ttlSeconds * 1000 });
debouncedPersist();
return true;
}
const r = await getRedis();
if (!r) return false;
try {
await r.set(key, value, { ex: ttlSeconds });
return true;
} catch (err) {
console.warn('[Cache] Write failed:', err.message);
return false;
}
}
export async function mget(...keys) {
if (isSidecar) {
await ensureDesktopCache();
const now = Date.now();
return keys.map(k => {
const entry = mem.get(k);
if (!entry || entry.expiresAt <= now) return null;
return entry.value;
});
}
const r = await getRedis();
if (!r) return keys.map(() => null);
try {
return await r.mget(...keys);
} catch (err) {
console.warn('[Cache] mget failed:', err.message);
return keys.map(() => null);
}
}
export function hashString(input) {
let hash = 5381;
for (let i = 0; i < input.length; i++) {
hash = ((hash << 5) + hash) + input.charCodeAt(i);
}
return (hash >>> 0).toString(36);
}
-188
View File
@@ -1,188 +0,0 @@
// ACLED Conflict Events API proxy - battles, explosions, violence against civilians
// Separate from protest proxy to avoid mixing data flows
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { createIpRateLimiter } from './_ip-rate-limit.js';
export const config = { runtime: 'edge' };
const CACHE_KEY = 'acled:conflict:v2';
const CACHE_TTL_SECONDS = 10 * 60;
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
let fallbackCache = { data: null, timestamp: 0 };
const RATE_LIMIT = 10;
const RATE_WINDOW_MS = 60 * 1000;
const rateLimiter = createIpRateLimiter({
limit: RATE_LIMIT,
windowMs: RATE_WINDOW_MS,
maxEntries: 5000,
});
function getClientIp(req) {
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
req.headers.get('x-real-ip') ||
'unknown';
}
function toErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'unknown error');
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return Response.json({ error: 'Method not allowed', data: [] }, {
status: 405,
headers: corsHeaders,
});
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed', data: [] }, {
status: 403,
headers: corsHeaders,
});
}
const ip = getClientIp(req);
if (!rateLimiter.check(ip)) {
return Response.json({ error: 'Rate limited', data: [] }, {
status: 429,
headers: {
...corsHeaders,
'Retry-After': '60',
},
});
}
const token = process.env.ACLED_ACCESS_TOKEN;
if (!token) {
return Response.json({ error: 'ACLED not configured', data: [], configured: false }, {
status: 200,
headers: corsHeaders,
});
}
const now = Date.now();
const cached = await getCachedJson(CACHE_KEY);
if (cached && typeof cached === 'object' && Array.isArray(cached.data)) {
recordCacheTelemetry('/api/acled-conflict', 'REDIS-HIT');
return Response.json(cached, {
status: 200,
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'X-Cache': 'REDIS-HIT',
},
});
}
if (fallbackCache.data && now - fallbackCache.timestamp < CACHE_TTL_MS) {
recordCacheTelemetry('/api/acled-conflict', 'MEMORY-HIT');
return Response.json(fallbackCache.data, {
status: 200,
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'X-Cache': 'MEMORY-HIT',
},
});
}
try {
const endDate = new Date().toISOString().split('T')[0];
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
const params = new URLSearchParams({
event_type: 'Battles|Explosions/Remote violence|Violence against civilians',
event_date: `${startDate}|${endDate}`,
event_date_where: 'BETWEEN',
limit: '500',
_format: 'json',
});
const response = await fetch(`https://acleddata.com/api/acled/read?${params}`, {
headers: {
Accept: 'application/json',
Authorization: `Bearer ${token}`,
},
});
if (!response.ok) {
const text = await response.text();
return Response.json({ error: `ACLED API error: ${response.status}`, details: text.substring(0, 200), data: [] }, {
status: response.status,
headers: corsHeaders,
});
}
const rawData = await response.json();
const events = Array.isArray(rawData?.data) ? rawData.data : [];
const sanitizedEvents = events.map((e) => ({
event_id_cnty: e.event_id_cnty,
event_date: e.event_date,
event_type: e.event_type,
sub_event_type: e.sub_event_type,
actor1: e.actor1,
actor2: e.actor2,
country: e.country,
admin1: e.admin1,
location: e.location,
latitude: e.latitude,
longitude: e.longitude,
fatalities: e.fatalities,
notes: typeof e.notes === 'string' ? e.notes.substring(0, 500) : undefined,
source: e.source,
tags: e.tags,
}));
const result = {
success: true,
count: sanitizedEvents.length,
data: sanitizedEvents,
cached_at: new Date().toISOString(),
};
fallbackCache = { data: result, timestamp: now };
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/acled-conflict', 'MISS');
return Response.json(result, {
status: 200,
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'X-Cache': 'MISS',
},
});
} catch (error) {
if (fallbackCache.data) {
recordCacheTelemetry('/api/acled-conflict', 'STALE');
return Response.json(fallbackCache.data, {
status: 200,
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
'X-Cache': 'STALE',
},
});
}
recordCacheTelemetry('/api/acled-conflict', 'ERROR');
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, data: [] }, {
status: 500,
headers: corsHeaders,
});
}
}
-200
View File
@@ -1,200 +0,0 @@
// ACLED API proxy - keeps token server-side only
// Token is stored in ACLED_ACCESS_TOKEN (no VITE_ prefix)
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { createIpRateLimiter } from './_ip-rate-limit.js';
export const config = { runtime: 'edge' };
const CACHE_KEY = 'acled:protests:v2';
const CACHE_TTL_SECONDS = 10 * 60;
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
// In-memory fallback cache when Redis is unavailable.
let fallbackCache = { data: null, timestamp: 0 };
const RATE_LIMIT = 10; // requests per minute
const RATE_WINDOW_MS = 60 * 1000;
const rateLimiter = createIpRateLimiter({
limit: RATE_LIMIT,
windowMs: RATE_WINDOW_MS,
maxEntries: 5000,
});
function getClientIp(req) {
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
req.headers.get('x-real-ip') ||
'unknown';
}
function toErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'unknown error');
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return Response.json({ error: 'Method not allowed', data: [] }, {
status: 405,
headers: corsHeaders,
});
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed', data: [] }, {
status: 403,
headers: corsHeaders,
});
}
const ip = getClientIp(req);
if (!rateLimiter.check(ip)) {
return Response.json({ error: 'Rate limited', data: [] }, {
status: 429,
headers: {
...corsHeaders,
'Retry-After': '60',
},
});
}
const token = process.env.ACLED_ACCESS_TOKEN;
if (!token) {
return Response.json({
error: 'ACLED not configured',
data: [],
configured: false,
}, {
status: 200,
headers: corsHeaders,
});
}
const now = Date.now();
const cached = await getCachedJson(CACHE_KEY);
if (cached && typeof cached === 'object' && Array.isArray(cached.data)) {
recordCacheTelemetry('/api/acled', 'REDIS-HIT');
return Response.json(cached, {
status: 200,
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'X-Cache': 'REDIS-HIT',
},
});
}
if (fallbackCache.data && now - fallbackCache.timestamp < CACHE_TTL_MS) {
recordCacheTelemetry('/api/acled', 'MEMORY-HIT');
return Response.json(fallbackCache.data, {
status: 200,
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'X-Cache': 'MEMORY-HIT',
},
});
}
try {
const endDate = new Date().toISOString().split('T')[0];
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
const params = new URLSearchParams({
event_type: 'Protests',
event_date: `${startDate}|${endDate}`,
event_date_where: 'BETWEEN',
limit: '500',
_format: 'json',
});
const response = await fetch(`https://acleddata.com/api/acled/read?${params}`, {
headers: {
Accept: 'application/json',
Authorization: `Bearer ${token}`,
},
});
if (!response.ok) {
const text = await response.text();
return Response.json({
error: `ACLED API error: ${response.status}`,
details: text.substring(0, 200),
data: [],
}, {
status: response.status,
headers: corsHeaders,
});
}
const rawData = await response.json();
const events = Array.isArray(rawData?.data) ? rawData.data : [];
const sanitizedEvents = events.map((e) => ({
event_id_cnty: e.event_id_cnty,
event_date: e.event_date,
event_type: e.event_type,
sub_event_type: e.sub_event_type,
actor1: e.actor1,
actor2: e.actor2,
country: e.country,
admin1: e.admin1,
location: e.location,
latitude: e.latitude,
longitude: e.longitude,
fatalities: e.fatalities,
notes: typeof e.notes === 'string' ? e.notes.substring(0, 500) : undefined,
source: e.source,
tags: e.tags,
}));
const result = {
success: true,
count: sanitizedEvents.length,
data: sanitizedEvents,
cached_at: new Date().toISOString(),
};
fallbackCache = { data: result, timestamp: now };
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/acled', 'MISS');
return Response.json(result, {
status: 200,
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'X-Cache': 'MISS',
},
});
} catch (error) {
if (fallbackCache.data) {
recordCacheTelemetry('/api/acled', 'STALE');
return Response.json(fallbackCache.data, {
status: 200,
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
'X-Cache': 'STALE',
},
});
}
recordCacheTelemetry('/api/acled', 'ERROR');
return Response.json({
error: `Fetch failed: ${toErrorMessage(error)}`,
data: [],
}, {
status: 500,
headers: corsHeaders,
});
}
}
-206
View File
@@ -1,206 +0,0 @@
export const config = { runtime: 'edge' };
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
const CACHE_TTL_SECONDS = 8;
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
const CACHE_VERSION = 'v1';
const MEMORY_CACHE_MAX_ENTRIES = 8;
const MEMORY_FALLBACK_MAX_AGE_MS = 60 * 1000;
const memoryCache = new Map();
const inFlightByKey = new Map();
function getErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'Failed to fetch AIS snapshot');
}
function getMemoryCachedSnapshot(cacheKey, allowStale = false) {
const entry = memoryCache.get(cacheKey);
if (!entry) return null;
const now = Date.now();
const age = now - entry.timestamp;
if (age > MEMORY_FALLBACK_MAX_AGE_MS) {
memoryCache.delete(cacheKey);
return null;
}
if (!allowStale && age > CACHE_TTL_MS) {
return null;
}
entry.lastSeen = now;
return entry.data;
}
function setMemoryCachedSnapshot(cacheKey, data) {
const now = Date.now();
memoryCache.set(cacheKey, {
data,
timestamp: now,
lastSeen: now,
});
if (memoryCache.size <= MEMORY_CACHE_MAX_ENTRIES) return;
const overflow = memoryCache.size - MEMORY_CACHE_MAX_ENTRIES;
const oldestEntries = Array.from(memoryCache.entries())
.sort((a, b) => a[1].lastSeen - b[1].lastSeen);
for (let i = 0; i < overflow; i++) {
const entry = oldestEntries[i];
if (!entry) break;
memoryCache.delete(entry[0]);
}
}
function getRelayBaseUrl() {
const relayUrl = process.env.WS_RELAY_URL;
if (!relayUrl) return null;
return relayUrl
.replace('wss://', 'https://')
.replace('ws://', 'http://')
.replace(/\/$/, '');
}
function isValidSnapshot(data) {
return Boolean(
data &&
typeof data === 'object' &&
data.status &&
typeof data.status === 'object' &&
Array.isArray(data.disruptions) &&
Array.isArray(data.density)
);
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
const requestUrl = new URL(req.url);
const includeCandidates = requestUrl.searchParams.get('candidates') === 'true';
const cacheKey = `ais-snapshot:${CACHE_VERSION}:${includeCandidates ? 'full' : 'lite'}`;
const redisCached = await getCachedJson(cacheKey);
if (isValidSnapshot(redisCached)) {
setMemoryCachedSnapshot(cacheKey, redisCached);
recordCacheTelemetry('/api/ais-snapshot', 'REDIS-HIT');
return new Response(JSON.stringify(redisCached), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
'X-Cache': 'REDIS-HIT',
...corsHeaders,
},
});
}
const memoryCached = getMemoryCachedSnapshot(cacheKey);
if (isValidSnapshot(memoryCached)) {
recordCacheTelemetry('/api/ais-snapshot', 'MEMORY-HIT');
return new Response(JSON.stringify(memoryCached), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
'X-Cache': 'MEMORY-HIT',
...corsHeaders,
},
});
}
const relayBaseUrl = getRelayBaseUrl();
if (!relayBaseUrl) {
recordCacheTelemetry('/api/ais-snapshot', 'NO-RELAY-CONFIG');
return new Response(JSON.stringify({ vessels: [], skipped: true, reason: 'AIS relay not configured' }), {
status: 200,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
try {
let requestPromise = inFlightByKey.get(cacheKey);
if (!requestPromise) {
requestPromise = (async () => {
const upstreamUrl = `${relayBaseUrl}/ais/snapshot?candidates=${includeCandidates ? 'true' : 'false'}`;
const response = await fetch(upstreamUrl, {
headers: { 'Accept': 'application/json' },
});
if (!response.ok) {
throw new Error(`AIS relay HTTP ${response.status}`);
}
const data = await response.json();
if (!isValidSnapshot(data)) {
throw new Error('Invalid AIS snapshot payload');
}
return data;
})();
inFlightByKey.set(cacheKey, requestPromise);
}
const data = await requestPromise;
if (!isValidSnapshot(data)) {
throw new Error('Invalid AIS snapshot payload');
}
setMemoryCachedSnapshot(cacheKey, data);
void setCachedJson(cacheKey, data, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/ais-snapshot', 'MISS');
return new Response(JSON.stringify(data), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
'X-Cache': 'MISS',
...corsHeaders,
},
});
} catch (error) {
const staleMemory = getMemoryCachedSnapshot(cacheKey, true);
if (isValidSnapshot(staleMemory)) {
recordCacheTelemetry('/api/ais-snapshot', 'MEMORY-ERROR-FALLBACK');
return new Response(JSON.stringify(staleMemory), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
'X-Cache': 'MEMORY-ERROR-FALLBACK',
...corsHeaders,
},
});
}
recordCacheTelemetry('/api/ais-snapshot', 'ERROR');
return new Response(JSON.stringify({ error: getErrorMessage(error) }), {
status: 502,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
} finally {
inFlightByKey.delete(cacheKey);
}
}
-59
View File
@@ -1,59 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
// Fetch AI/ML papers from ArXiv
// Categories: cs.AI, cs.LG (Machine Learning), cs.CL (Computation and Language)
export default async function handler(request) {
const cors = getCorsHeaders(request);
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
try {
const { searchParams } = new URL(request.url);
const category = searchParams.get('category') || 'cs.AI'; // cs.AI, cs.LG, cs.CL
const maxResults = searchParams.get('max_results') || '50';
const sortBy = searchParams.get('sortBy') || 'submittedDate'; // submittedDate, lastUpdatedDate, relevance
// ArXiv API search query
// Search for papers in specified category, sorted by date
const query = `cat:${category}`;
const apiUrl = `https://export.arxiv.org/api/query?search_query=${encodeURIComponent(query)}&start=0&max_results=${maxResults}&sortBy=${sortBy}&sortOrder=descending`;
const response = await fetch(apiUrl, {
headers: {
'User-Agent': 'WorldMonitor/1.0 (AI Research Tracker)',
},
});
if (!response.ok) {
throw new Error(`ArXiv API returned ${response.status}`);
}
const xmlData = await response.text();
// Parse XML to extract key information
// Return raw XML for client-side parsing or transform here
return new Response(xmlData, {
status: 200,
headers: {
'Content-Type': 'application/xml',
...cors,
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', // 1 hour cache
},
});
} catch (error) {
return new Response(
JSON.stringify({
error: 'Failed to fetch ArXiv data',
message: error.message
}),
{
status: 500,
headers: {
'Content-Type': 'application/json',
...cors
},
}
);
}
}
-38
View File
@@ -1,38 +0,0 @@
export const config = { runtime: 'edge' };
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCacheTelemetrySnapshot } from './_cache-telemetry.js';
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
return new Response(JSON.stringify(getCacheTelemetrySnapshot()), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
...corsHeaders,
},
});
}
-198
View File
@@ -1,198 +0,0 @@
import { getCachedJson, setCachedJson, mget, hashString } from './_upstash-cache.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
const MODEL = 'llama-3.1-8b-instant';
const CACHE_TTL_SECONDS = 86400;
const CACHE_VERSION = 'v1';
const MAX_BATCH_SIZE = 20;
const VALID_LEVELS = ['critical', 'high', 'medium', 'low', 'info'];
const VALID_CATEGORIES = [
'conflict', 'protest', 'disaster', 'diplomatic', 'economic',
'terrorism', 'cyber', 'health', 'environmental', 'military',
'crime', 'infrastructure', 'tech', 'general',
];
export default async function handler(request) {
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
if (request.method !== 'POST') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
const apiKey = process.env.GROQ_API_KEY;
if (!apiKey) {
return new Response(JSON.stringify({ results: [], fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
if (contentLength > 51200) {
return new Response(JSON.stringify({ error: 'Payload too large' }), {
status: 413,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
let body;
try {
body = await request.json();
} catch {
return new Response(JSON.stringify({ error: 'Invalid JSON body' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
const { titles, variant = 'full' } = body;
if (!Array.isArray(titles) || titles.length === 0) {
return new Response(JSON.stringify({ error: 'titles array required' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
const batch = titles.slice(0, MAX_BATCH_SIZE);
const results = new Array(batch.length).fill(null);
const uncachedIndices = [];
const cacheKeys = batch.map(
(t) => `classify:${CACHE_VERSION}:${hashString(t.toLowerCase() + ':' + variant)}`
);
const cached = await mget(...cacheKeys);
for (let i = 0; i < cached.length; i++) {
const val = cached[i];
if (val && typeof val === 'object' && val.level) {
results[i] = { level: val.level, category: val.category, cached: true };
} else {
uncachedIndices.push(i);
}
}
if (uncachedIndices.length === 0) {
return new Response(JSON.stringify({ results }), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
const uncachedTitles = uncachedIndices.map((i) => batch[i]);
const isTech = variant === 'tech';
const numberedList = uncachedTitles.map((t, i) => `${i + 1}. ${t}`).join('\n');
const systemPrompt = `You classify news headlines into threat level and category. Return ONLY a valid JSON array, no other text.
Levels: critical, high, medium, low, info
Categories: conflict, protest, disaster, diplomatic, economic, terrorism, cyber, health, environmental, military, crime, infrastructure, tech, general
${isTech ? 'Focus: technology, startups, AI, cybersecurity. Most tech news is "low" or "info" unless it involves outages, breaches, or major disruptions.' : 'Focus: geopolitical events, conflicts, disasters, diplomacy. Classify by real-world severity and impact.'}
Return a JSON array with one object per headline in order: [{"level":"...","category":"..."},...]`;
try {
const response = await fetch(GROQ_API_URL, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: MODEL,
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: numberedList },
],
temperature: 0,
max_tokens: uncachedTitles.length * 60,
}),
});
if (!response.ok) {
console.error('[ClassifyBatch] Groq error:', response.status);
return new Response(JSON.stringify({ results, fallback: true }), {
status: response.status,
headers: { 'Content-Type': 'application/json' },
});
}
const data = await response.json();
const raw = data.choices?.[0]?.message?.content?.trim();
if (!raw) {
return new Response(JSON.stringify({ results, fallback: true }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
let parsed;
try {
parsed = JSON.parse(raw);
} catch {
const match = raw.match(/\[[\s\S]*\]/);
if (match) {
try { parsed = JSON.parse(match[0]); } catch { /* fall through */ }
}
}
if (!Array.isArray(parsed)) {
return new Response(JSON.stringify({ results, fallback: true }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
const cacheWrites = [];
for (let i = 0; i < uncachedIndices.length; i++) {
const classification = parsed[i];
if (!classification) continue;
const level = VALID_LEVELS.includes(classification.level) ? classification.level : null;
const category = VALID_CATEGORIES.includes(classification.category) ? classification.category : null;
if (!level || !category) continue;
const idx = uncachedIndices[i];
results[idx] = { level, category, cached: false };
const cacheKey = `classify:${CACHE_VERSION}:${hashString(batch[idx].toLowerCase() + ':' + variant)}`;
cacheWrites.push(
setCachedJson(cacheKey, { level, category, timestamp: Date.now() }, CACHE_TTL_SECONDS)
);
}
if (cacheWrites.length > 0) {
await Promise.allSettled(cacheWrites);
}
return new Response(JSON.stringify({ results }), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
} catch (error) {
console.error('[ClassifyBatch] Error:', error.message);
return new Response(JSON.stringify({ results, fallback: true }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
-164
View File
@@ -1,164 +0,0 @@
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
const MODEL = 'llama-3.1-8b-instant';
const CACHE_TTL_SECONDS = 86400;
const CACHE_VERSION = 'v1';
const VALID_LEVELS = ['critical', 'high', 'medium', 'low', 'info'];
const VALID_CATEGORIES = [
'conflict', 'protest', 'disaster', 'diplomatic', 'economic',
'terrorism', 'cyber', 'health', 'environmental', 'military',
'crime', 'infrastructure', 'tech', 'general',
];
export default async function handler(request) {
const corsHeaders = getCorsHeaders(request, 'GET, OPTIONS');
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
if (request.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
const apiKey = process.env.GROQ_API_KEY;
if (!apiKey) {
return new Response(JSON.stringify({ fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
const url = new URL(request.url);
const title = url.searchParams.get('title');
const variant = url.searchParams.get('variant') || 'full';
if (!title) {
return new Response(JSON.stringify({ error: 'title param required' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
const cacheKey = `classify:${CACHE_VERSION}:${hashString(title.toLowerCase() + ':' + variant)}`;
try {
const cached = await getCachedJson(cacheKey);
if (cached && typeof cached === 'object' && cached.level) {
return new Response(JSON.stringify({
level: cached.level,
category: cached.category,
confidence: 0.9,
source: 'llm',
cached: true,
}), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
const isTech = variant === 'tech';
const systemPrompt = `You classify news headlines into threat level and category. Return ONLY valid JSON, no other text.
Levels: critical, high, medium, low, info
Categories: conflict, protest, disaster, diplomatic, economic, terrorism, cyber, health, environmental, military, crime, infrastructure, tech, general
${isTech ? 'Focus: technology, startups, AI, cybersecurity. Most tech news is "low" or "info" unless it involves outages, breaches, or major disruptions.' : 'Focus: geopolitical events, conflicts, disasters, diplomacy. Classify by real-world severity and impact.'}
Return: {"level":"...","category":"..."}`;
const response = await fetch(GROQ_API_URL, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: MODEL,
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: title },
],
temperature: 0,
max_tokens: 50,
}),
});
if (!response.ok) {
console.error('[Classify] Groq error:', response.status);
return new Response(JSON.stringify({ fallback: true }), {
status: response.status,
headers: { 'Content-Type': 'application/json' },
});
}
const data = await response.json();
const raw = data.choices?.[0]?.message?.content?.trim();
if (!raw) {
return new Response(JSON.stringify({ fallback: true }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
let parsed;
try {
parsed = JSON.parse(raw);
} catch {
console.warn('[Classify] Invalid JSON from LLM:', raw);
return new Response(JSON.stringify({ fallback: true }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
const level = VALID_LEVELS.includes(parsed.level) ? parsed.level : null;
const category = VALID_CATEGORIES.includes(parsed.category) ? parsed.category : null;
if (!level || !category) {
return new Response(JSON.stringify({ fallback: true }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
await setCachedJson(cacheKey, { level, category, timestamp: Date.now() }, CACHE_TTL_SECONDS);
return new Response(JSON.stringify({
level,
category,
confidence: 0.9,
source: 'llm',
cached: false,
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600',
},
});
} catch (error) {
console.error('[Classify] Error:', error.message);
return new Response(JSON.stringify({ fallback: true }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
-206
View File
@@ -1,206 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { createIpRateLimiter } from './_ip-rate-limit.js';
export const config = { runtime: 'edge' };
const CACHE_KEY = 'climate:anomalies:v1';
const CACHE_TTL_SECONDS = 6 * 60 * 60;
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
let fallbackCache = { data: null, timestamp: 0 };
const rateLimiter = createIpRateLimiter({
limit: 15,
windowMs: 60 * 1000,
maxEntries: 5000,
});
function getClientIp(req) {
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
req.headers.get('x-real-ip') ||
'unknown';
}
function toErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'unknown error');
}
function isValidResult(data) {
return Boolean(data && typeof data === 'object' && Array.isArray(data.anomalies));
}
const MONITORED_ZONES = [
{ name: 'Ukraine', lat: 48.4, lon: 31.2 },
{ name: 'Middle East', lat: 33.0, lon: 44.0 },
{ name: 'Sahel', lat: 14.0, lon: 0.0 },
{ name: 'Horn of Africa', lat: 8.0, lon: 42.0 },
{ name: 'South Asia', lat: 25.0, lon: 78.0 },
{ name: 'California', lat: 36.8, lon: -119.4 },
{ name: 'Amazon', lat: -3.4, lon: -60.0 },
{ name: 'Australia', lat: -25.0, lon: 134.0 },
{ name: 'Mediterranean', lat: 38.0, lon: 20.0 },
{ name: 'Taiwan Strait', lat: 24.0, lon: 120.0 },
{ name: 'Myanmar', lat: 19.8, lon: 96.7 },
{ name: 'Central Africa', lat: 4.0, lon: 22.0 },
{ name: 'Southern Africa', lat: -25.0, lon: 28.0 },
{ name: 'Central Asia', lat: 42.0, lon: 65.0 },
{ name: 'Caribbean', lat: 19.0, lon: -72.0 },
];
function classifySeverity(tempDelta, precipDelta) {
const absTemp = Math.abs(tempDelta);
const absPrecip = Math.abs(precipDelta);
if (absTemp >= 5 || absPrecip >= 80) return 'extreme';
if (absTemp >= 3 || absPrecip >= 40) return 'moderate';
return 'normal';
}
function classifyType(tempDelta, precipDelta) {
const absTemp = Math.abs(tempDelta);
const absPrecip = Math.abs(precipDelta);
if (absTemp >= absPrecip / 20) {
if (tempDelta > 0 && precipDelta < -20) return 'mixed';
if (tempDelta > 3) return 'warm';
if (tempDelta < -3) return 'cold';
}
if (precipDelta > 40) return 'wet';
if (precipDelta < -40) return 'dry';
if (tempDelta > 0) return 'warm';
return 'cold';
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
}
const ip = getClientIp(req);
if (!rateLimiter.check(ip)) {
return Response.json({ error: 'Rate limited' }, {
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
});
}
const now = Date.now();
const cached = await getCachedJson(CACHE_KEY);
if (isValidResult(cached)) {
recordCacheTelemetry('/api/climate-anomalies', 'REDIS-HIT');
return Response.json(cached, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
});
}
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
recordCacheTelemetry('/api/climate-anomalies', 'MEMORY-HIT');
return Response.json(fallbackCache.data, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
});
}
try {
const endDate = new Date();
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000);
const start = startDate.toISOString().split('T')[0];
const end = endDate.toISOString().split('T')[0];
const fetchZone = async (zone) => {
try {
const params = new URLSearchParams({
latitude: String(zone.lat),
longitude: String(zone.lon),
start_date: start,
end_date: end,
daily: 'temperature_2m_mean,precipitation_sum',
timezone: 'UTC',
});
const resp = await fetch(`https://archive-api.open-meteo.com/v1/archive?${params}`, {
headers: { Accept: 'application/json' },
});
if (!resp.ok) return null;
const data = await resp.json();
const temps = data.daily?.temperature_2m_mean || [];
const precips = data.daily?.precipitation_sum || [];
if (temps.length < 14) return null;
const validTemps = temps.filter(t => t !== null);
const validPrecips = precips.filter(p => p !== null);
const last7Temps = validTemps.slice(-7);
const baseline30Temps = validTemps.slice(0, -7);
const last7Precips = validPrecips.slice(-7);
const baseline30Precips = validPrecips.slice(0, -7);
const avg = arr => arr.length ? arr.reduce((s, v) => s + v, 0) / arr.length : 0;
const tempDelta = avg(last7Temps) - avg(baseline30Temps);
const precipDelta = avg(last7Precips) - avg(baseline30Precips);
const severity = classifySeverity(tempDelta, precipDelta);
return {
zone: zone.name,
lat: zone.lat,
lon: zone.lon,
tempDelta: Math.round(tempDelta * 10) / 10,
precipDelta: Math.round(precipDelta * 10) / 10,
severity,
type: classifyType(tempDelta, precipDelta),
period: `${start} to ${end}`,
};
} catch {
return null;
}
};
const results = await Promise.allSettled(MONITORED_ZONES.map(fetchZone));
const anomalies = results
.filter(r => r.status === 'fulfilled' && r.value)
.map(r => r.value);
const result = {
success: true,
anomalies,
timestamp: new Date().toISOString(),
};
fallbackCache = { data: result, timestamp: now };
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/climate-anomalies', 'MISS');
return Response.json(result, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
});
} catch (error) {
if (isValidResult(fallbackCache.data)) {
recordCacheTelemetry('/api/climate-anomalies', 'STALE');
return Response.json(fallbackCache.data, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
});
}
recordCacheTelemetry('/api/climate-anomalies', 'ERROR');
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, anomalies: [] }, {
status: 500, headers: corsHeaders,
});
}
}
-64
View File
@@ -1,64 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
function clampLimit(rawLimit) {
const parsed = Number.parseInt(rawLimit || '', 10);
if (!Number.isFinite(parsed)) return 50;
return Math.max(1, Math.min(100, parsed));
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
const url = new URL(req.url);
const dateRange = url.searchParams.get('dateRange') || '7d';
const limit = clampLimit(url.searchParams.get('limit'));
const token = process.env.CLOUDFLARE_API_TOKEN;
if (!token) {
// Signal to client that outages feature is not configured
return new Response(JSON.stringify({ configured: false }), {
status: 200,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
try {
const response = await fetch(
`https://api.cloudflare.com/client/v4/radar/annotations/outages?dateRange=${dateRange}&limit=${limit}`,
{ headers: { 'Authorization': `Bearer ${token}` } }
);
const data = await response.text();
return new Response(data, {
status: response.status,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=120, s-maxage=120, stale-while-revalidate=60', ...corsHeaders },
});
} catch (error) {
// Return empty result on error so client circuit breaker doesn't trigger unnecessarily
return new Response(JSON.stringify({ success: true, result: { annotations: [] } }), {
status: 200,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
}
-154
View File
@@ -1,154 +0,0 @@
export const config = { runtime: 'edge' };
import { getCachedJson, hashString, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
const ALLOWED_CURRENCIES = ['usd', 'eur', 'gbp', 'jpy', 'cny', 'btc', 'eth'];
const MAX_COIN_IDS = 20;
const COIN_ID_PATTERN = /^[a-z0-9-]+$/;
const CACHE_TTL_SECONDS = 120; // 2 minutes
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
const RESPONSE_CACHE_CONTROL = 'public, max-age=120, s-maxage=120, stale-while-revalidate=60';
const CACHE_VERSION = 'v2';
// In-memory fallback cache for the current instance.
let fallbackCache = { key: '', payload: null, timestamp: 0 };
function validateCoinIds(idsParam) {
if (!idsParam) return 'bitcoin,ethereum,solana';
const ids = idsParam.split(',')
.map(id => id.trim().toLowerCase())
.filter(id => COIN_ID_PATTERN.test(id) && id.length <= 50)
.slice(0, MAX_COIN_IDS);
return ids.length > 0 ? ids.join(',') : 'bitcoin,ethereum,solana';
}
function validateCurrency(val) {
const currency = (val || 'usd').toLowerCase();
return ALLOWED_CURRENCIES.includes(currency) ? currency : 'usd';
}
function validateBoolean(val, defaultVal) {
if (val === 'true' || val === 'false') return val;
return defaultVal;
}
function getHeaders(cors, xCache, cacheControl = RESPONSE_CACHE_CONTROL) {
return {
'Content-Type': 'application/json',
...cors,
'Cache-Control': cacheControl,
'X-Cache': xCache,
};
}
function isValidPayload(payload) {
return Boolean(
payload &&
typeof payload === 'object' &&
typeof payload.body === 'string' &&
Number.isFinite(payload.status)
);
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const url = new URL(req.url);
const ids = validateCoinIds(url.searchParams.get('ids'));
const vsCurrencies = validateCurrency(url.searchParams.get('vs_currencies'));
const include24hrChange = validateBoolean(url.searchParams.get('include_24hr_change'), 'true');
const now = Date.now();
const cacheKey = `${ids}:${vsCurrencies}:${include24hrChange}`;
const redisKey = `coingecko:${CACHE_VERSION}:${hashString(cacheKey)}`;
const redisCached = await getCachedJson(redisKey);
if (isValidPayload(redisCached)) {
recordCacheTelemetry('/api/coingecko', 'REDIS-HIT');
return new Response(redisCached.body, {
status: redisCached.status,
headers: getHeaders(cors, 'REDIS-HIT'),
});
}
if (
isValidPayload(fallbackCache.payload) &&
fallbackCache.key === cacheKey &&
now - fallbackCache.timestamp < CACHE_TTL_MS
) {
recordCacheTelemetry('/api/coingecko', 'MEMORY-HIT');
return new Response(fallbackCache.payload.body, {
status: fallbackCache.payload.status,
headers: getHeaders(cors, 'MEMORY-HIT'),
});
}
const endpoint = url.searchParams.get('endpoint');
try {
let geckoUrl;
if (endpoint === 'markets') {
geckoUrl = `https://api.coingecko.com/api/v3/coins/markets?vs_currency=${vsCurrencies}&ids=${ids}&order=market_cap_desc&sparkline=true&price_change_percentage=24h`;
} else {
geckoUrl = `https://api.coingecko.com/api/v3/simple/price?ids=${ids}&vs_currencies=${vsCurrencies}&include_24hr_change=${include24hrChange}`;
}
const response = await fetch(geckoUrl, {
headers: {
'Accept': 'application/json',
},
});
// If rate limited, return cached data if available
if (
response.status === 429 &&
isValidPayload(fallbackCache.payload) &&
fallbackCache.key === cacheKey
) {
recordCacheTelemetry('/api/coingecko', 'STALE');
return new Response(fallbackCache.payload.body, {
status: fallbackCache.payload.status,
headers: getHeaders(cors, 'STALE'),
});
}
const data = await response.text();
// Cache successful responses
if (response.ok) {
const payload = { body: data, status: response.status };
fallbackCache = { key: cacheKey, payload, timestamp: Date.now() };
void setCachedJson(redisKey, payload, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/coingecko', 'MISS');
} else {
recordCacheTelemetry('/api/coingecko', 'UPSTREAM-ERROR');
}
return new Response(data, {
status: response.status,
headers: getHeaders(cors, 'MISS'),
});
} catch (error) {
// Return cached data on error if available
if (isValidPayload(fallbackCache.payload) && fallbackCache.key === cacheKey) {
recordCacheTelemetry('/api/coingecko', 'ERROR-FALLBACK');
return new Response(fallbackCache.payload.body, {
status: fallbackCache.payload.status,
headers: getHeaders(cors, 'ERROR-FALLBACK', 'public, max-age=120'),
});
}
recordCacheTelemetry('/api/coingecko', 'ERROR');
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
status: 500,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
}
-191
View File
@@ -1,191 +0,0 @@
/**
* Country Intelligence Brief Endpoint
* Generates AI-powered country situation briefs using Groq
* Redis cached (2h TTL) for cross-user deduplication
*/
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
const MODEL = 'llama-3.1-8b-instant';
const CACHE_TTL_SECONDS = 7200; // 2 hours
const CACHE_VERSION = 'ci-v2';
export default async function handler(request) {
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
if (request.method !== 'POST') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
const apiKey = process.env.GROQ_API_KEY;
if (!apiKey) {
return new Response(JSON.stringify({ intel: null, fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
if (contentLength > 51200) {
return new Response(JSON.stringify({ error: 'Payload too large' }), {
status: 413,
headers: { 'Content-Type': 'application/json' },
});
}
try {
const { country, code, context } = await request.json();
if (!country || !code) {
return new Response(JSON.stringify({ error: 'country and code required' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
// Cache key includes country code + context hash (context changes as data updates)
const contextHash = context ? hashString(JSON.stringify(context)).slice(0, 8) : 'no-ctx';
const cacheKey = `${CACHE_VERSION}:${code}:${contextHash}`;
const cached = await getCachedJson(cacheKey);
if (cached && typeof cached === 'object' && cached.brief) {
console.log('[CountryIntel] Cache hit:', code);
return new Response(JSON.stringify({ ...cached, cached: true }), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
// Build data context section
const dataLines = [];
if (context?.score != null) {
const changeStr = context.change24h ? ` (${context.change24h > 0 ? '+' : ''}${context.change24h} in 24h)` : '';
dataLines.push(`Instability Score: ${context.score}/100 (${context.level || 'unknown'}) — trend: ${context.trend || 'unknown'}${changeStr}`);
}
if (context?.components) {
const c = context.components;
dataLines.push(`Score Components: Unrest ${c.unrest ?? '?'}/100, Security ${c.security ?? '?'}/100, Information ${c.information ?? '?'}/100`);
}
if (context?.protests != null) dataLines.push(`Active protests in/near country (7d): ${context.protests}`);
if (context?.militaryFlights != null) dataLines.push(`Military aircraft detected in/near country: ${context.militaryFlights}`);
if (context?.militaryVessels != null) dataLines.push(`Military vessels detected in/near country: ${context.militaryVessels}`);
if (context?.outages != null) dataLines.push(`Internet outages: ${context.outages}`);
if (context?.earthquakes != null) dataLines.push(`Recent earthquakes: ${context.earthquakes}`);
if (context?.stockIndex) dataLines.push(`Stock Market Index: ${context.stockIndex}`);
if (context?.convergenceScore != null) {
dataLines.push(`Signal convergence score: ${context.convergenceScore}/100 (multiple signal types detected: ${(context.signalTypes || []).join(', ')})`);
}
if (context?.regionalConvergence?.length > 0) {
dataLines.push(`\nRegional convergence alerts:`);
context.regionalConvergence.forEach(r => dataLines.push(`- ${r}`));
}
if (context?.headlines?.length > 0) {
dataLines.push(`\nRecent headlines mentioning ${country} (${context.headlines.length} found):`);
context.headlines.slice(0, 15).forEach((h, i) => dataLines.push(`${i + 1}. ${h}`));
}
const dataSection = dataLines.length > 0
? `\nCURRENT SENSOR DATA:\n${dataLines.join('\n')}`
: '\nNo real-time sensor data available for this country.';
const dateStr = new Date().toISOString().split('T')[0];
const systemPrompt = `You are a senior intelligence analyst providing comprehensive country situation briefs. Current date: ${dateStr}. Donald Trump is the current US President (second term, inaugurated Jan 2025).
Write a thorough, data-driven intelligence brief for the requested country. Structure:
1. **Current Situation** — What is happening right now. Reference specific data: instability scores, protest counts, military presence, outages. Explain what the numbers mean in context.
2. **Military & Security Posture** — Analyze military activity in/near the country. What forces are present? What does the positioning suggest? What are foreign nations doing in this theater?
3. **Key Risk Factors** — What drives instability or stability. Connect the dots between different signals (protests + outages = potential crackdown? military buildup + diplomatic tensions = escalation risk?). Reference specific headlines.
4. **Regional Context** — How does this country's situation affect or relate to its neighbors and the broader region? Reference any convergence alerts.
5. **Outlook & Watch Items** — What to monitor in the near term. Be specific about indicators that would signal escalation or de-escalation.
Rules:
- Be specific and analytical. Reference the data provided (scores, counts, headlines, convergence).
- If data shows low activity, say so — don't manufacture threats.
- Connect signals: explain what combinations of data points suggest.
- 5-6 paragraphs, 300-400 words.
- No speculation beyond what the data supports.
- Use plain language, not jargon.
- If military assets are 0, don't speculate about military presence — say monitoring shows no current military activity.
- When referencing a specific headline from the numbered list, cite it as [N] where N is the headline number (e.g. "tensions escalated [3]"). Only cite headlines you directly reference.`;
const userPrompt = `Country: ${country} (${code})${dataSection}`;
const groqRes = await fetch(GROQ_API_URL, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: MODEL,
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userPrompt },
],
temperature: 0.4,
max_tokens: 900,
}),
});
if (!groqRes.ok) {
const errText = await groqRes.text();
console.error('[CountryIntel] Groq error:', groqRes.status, errText);
return new Response(JSON.stringify({ error: 'AI service error', fallback: true }), {
status: 502,
headers: { 'Content-Type': 'application/json' },
});
}
const groqData = await groqRes.json();
const brief = groqData.choices?.[0]?.message?.content || '';
const result = {
brief,
country,
code,
model: MODEL,
generatedAt: new Date().toISOString(),
};
if (brief) {
await setCachedJson(cacheKey, result, CACHE_TTL_SECONDS);
}
return new Response(JSON.stringify(result), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
} catch (err) {
console.error('[CountryIntel] Error:', err);
return new Response(JSON.stringify({ error: 'Internal error' }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
-1061
View File
File diff suppressed because it is too large Load Diff
-371
View File
@@ -1,371 +0,0 @@
import { strict as assert } from 'node:assert';
import test from 'node:test';
import handler, {
__resetCyberThreatsState,
__testDedupeThreats,
__testParseFeodoRecords,
} from './cyber-threats.js';
const ORIGINAL_FETCH = globalThis.fetch;
const ORIGINAL_URLHAUS_KEY = process.env.URLHAUS_AUTH_KEY;
const ORIGINAL_OTX_KEY = process.env.OTX_API_KEY;
const ORIGINAL_ABUSEIPDB_KEY = process.env.ABUSEIPDB_API_KEY;
function makeRequest(path = '/api/cyber-threats', ip = '198.51.100.10') {
const headers = new Headers();
headers.set('x-forwarded-for', ip);
return new Request(`https://worldmonitor.app${path}`, { headers });
}
function jsonResponse(body, status = 200) {
return new Response(JSON.stringify(body), {
status,
headers: { 'content-type': 'application/json' },
});
}
function textResponse(body, status = 200) {
return new Response(body, {
status,
headers: { 'content-type': 'text/plain' },
});
}
// Mock that handles all 5 source URLs + geo enrichment
function createMockFetch({ feodo, urlhaus, c2intel, otx, abuseipdb, geo } = {}) {
return async (url) => {
const target = String(url);
if (target.includes('feodotracker.abuse.ch') && feodo) return feodo(target);
if (target.includes('urlhaus-api.abuse.ch') && urlhaus) return urlhaus(target);
if (target.includes('raw.githubusercontent.com') && target.includes('C2IntelFeeds') && c2intel) return c2intel(target);
if (target.includes('otx.alienvault.com') && otx) return otx(target);
if (target.includes('api.abuseipdb.com') && abuseipdb) return abuseipdb(target);
if ((target.includes('ipwho.is') || target.includes('ipapi.co')) && geo) return geo(target);
// Default: return 404 for unconfigured sources
return new Response('not found', { status: 404 });
};
}
test.afterEach(() => {
globalThis.fetch = ORIGINAL_FETCH;
process.env.URLHAUS_AUTH_KEY = ORIGINAL_URLHAUS_KEY;
process.env.OTX_API_KEY = ORIGINAL_OTX_KEY;
process.env.ABUSEIPDB_API_KEY = ORIGINAL_ABUSEIPDB_KEY;
__resetCyberThreatsState();
});
test('Feodo parser accepts online and recent offline entries, filters stale', () => {
const nowMs = Date.parse('2026-02-15T12:00:00.000Z');
const records = [
{
ip_address: '1.2.3.4',
status: 'online',
first_seen: '2026-02-14 10:00:00 UTC',
last_online: '2026-02-15 10:00:00 UTC',
malware: 'QakBot',
},
{
ip_address: '5.6.7.8',
status: 'offline',
first_seen: '2026-02-14 10:00:00 UTC',
last_online: '2026-02-15 10:00:00 UTC',
malware: 'Emotet',
},
{
ip_address: '9.9.9.9',
status: 'online',
first_seen: '2025-10-01 10:00:00 UTC',
last_online: '2025-10-02 10:00:00 UTC',
malware: 'generic',
},
{
ip_address: '2.2.2.2',
first_seen: '2026-02-14 10:00:00 UTC',
last_online: '2026-02-15 10:00:00 UTC',
malware: 'generic',
},
];
const parsed = __testParseFeodoRecords(records, { nowMs, days: 14 });
// online + offline (recent) + no-status (recent) = 3; stale (9.9.9.9) filtered
assert.equal(parsed.length, 3);
assert.equal(parsed[0].indicator, '1.2.3.4');
assert.equal(parsed[0].severity, 'critical');
assert.equal(parsed[1].indicator, '5.6.7.8');
assert.equal(parsed[1].severity, 'medium');
assert.equal(parsed[0].firstSeen?.endsWith('Z'), true);
assert.equal(parsed[0].lastSeen?.endsWith('Z'), true);
});
test('dedupes by source + indicatorType + indicator', () => {
const deduped = __testDedupeThreats([
{
id: 'a',
source: 'feodo',
type: 'c2_server',
indicatorType: 'ip',
indicator: '1.2.3.4',
severity: 'high',
tags: ['a'],
firstSeen: '2026-02-10T00:00:00.000Z',
lastSeen: '2026-02-11T00:00:00.000Z',
},
{
id: 'b',
source: 'feodo',
type: 'c2_server',
indicatorType: 'ip',
indicator: '1.2.3.4',
severity: 'critical',
tags: ['b'],
firstSeen: '2026-02-12T00:00:00.000Z',
lastSeen: '2026-02-13T00:00:00.000Z',
},
{
id: 'c',
source: 'urlhaus',
type: 'malicious_url',
indicatorType: 'domain',
indicator: 'bad.example',
severity: 'medium',
tags: [],
firstSeen: '2026-02-11T00:00:00.000Z',
lastSeen: '2026-02-11T01:00:00.000Z',
},
]);
assert.equal(deduped.length, 2);
const feodo = deduped.find((item) => item.source === 'feodo');
assert.equal(feodo?.severity, 'critical');
assert.equal(feodo?.tags.includes('a'), true);
assert.equal(feodo?.tags.includes('b'), true);
});
test('API aggregates from all 5 sources', async () => {
process.env.URLHAUS_AUTH_KEY = 'test-key';
process.env.OTX_API_KEY = 'test-otx';
process.env.ABUSEIPDB_API_KEY = 'test-abuse';
globalThis.fetch = createMockFetch({
feodo: () => jsonResponse([
{
ip_address: '1.2.3.4',
status: 'online',
last_online: '2026-02-15T10:00:00.000Z',
first_seen: '2026-02-14T10:00:00.000Z',
malware: 'QakBot',
country: 'GB',
lat: 51.5,
lon: -0.12,
},
]),
urlhaus: () => jsonResponse({
urls: [{
url: 'http://5.5.5.5/malware.exe',
host: '5.5.5.5',
url_status: 'online',
threat: 'malware_download',
tags: ['malware'],
dateadded: '2026-02-14T08:00:00.000Z',
latitude: 48.86,
longitude: 2.35,
country: 'FR',
}],
}),
c2intel: () => textResponse(
'#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP\n10.10.10.11,Possible Metasploit C2 IP',
),
otx: () => jsonResponse({
results: [{
indicator: '20.20.20.20',
title: 'APT threat',
tags: ['apt', 'c2'],
created: '2026-02-13T00:00:00.000Z',
modified: '2026-02-14T00:00:00.000Z',
}],
}),
abuseipdb: () => jsonResponse({
data: [{
ipAddress: '30.30.30.30',
abuseConfidenceScore: 98,
lastReportedAt: '2026-02-15T06:00:00.000Z',
countryCode: 'CN',
latitude: 39.9,
longitude: 116.4,
}],
}),
geo: () => jsonResponse({ success: true, latitude: 40.0, longitude: -74.0, country_code: 'US' }),
});
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.20'));
assert.equal(response.status, 200);
const body = await response.json();
assert.equal(body.success, true);
assert.equal(body.sources.feodo.ok, true);
assert.equal(body.sources.urlhaus.ok, true);
assert.equal(body.sources.c2intel.ok, true);
assert.equal(body.sources.otx.ok, true);
assert.equal(body.sources.abuseipdb.ok, true);
// 5 sources, all with coords (3 native + 3 via geo enrichment mock)
assert.equal(body.data.length >= 5, true);
});
test('API works with only free sources when keys missing', async () => {
delete process.env.URLHAUS_AUTH_KEY;
delete process.env.OTX_API_KEY;
delete process.env.ABUSEIPDB_API_KEY;
globalThis.fetch = createMockFetch({
feodo: () => jsonResponse([
{
ip_address: '1.2.3.4',
status: 'online',
last_online: '2026-02-15T10:00:00.000Z',
first_seen: '2026-02-14T10:00:00.000Z',
malware: 'QakBot',
country: 'GB',
lat: 51.5,
lon: -0.12,
},
]),
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
});
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.11'));
assert.equal(response.status, 200);
assert.equal(response.headers.get('X-Cache'), 'MISS');
const body = await response.json();
assert.equal(body.success, true);
assert.equal(body.partial, false);
assert.equal(body.sources.feodo.ok, true);
assert.equal(body.sources.c2intel.ok, true);
assert.equal(body.sources.urlhaus.ok, false);
assert.equal(body.sources.urlhaus.reason, 'missing_auth_key');
assert.equal(body.sources.otx.ok, false);
assert.equal(body.sources.otx.reason, 'missing_api_key');
assert.equal(body.sources.abuseipdb.ok, false);
assert.equal(body.sources.abuseipdb.reason, 'missing_api_key');
assert.equal(Array.isArray(body.data), true);
});
test('API marks partial=true when URLhaus is enabled but fails', async () => {
process.env.URLHAUS_AUTH_KEY = 'test-key';
delete process.env.OTX_API_KEY;
delete process.env.ABUSEIPDB_API_KEY;
globalThis.fetch = createMockFetch({
feodo: () => jsonResponse([
{
ip_address: '1.2.3.4',
status: 'online',
last_online: '2026-02-15T10:00:00.000Z',
first_seen: '2026-02-14T10:00:00.000Z',
malware: 'QakBot',
country: 'GB',
lat: 51.5,
lon: -0.12,
},
]),
urlhaus: () => new Response('boom', { status: 500 }),
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
});
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.12'));
assert.equal(response.status, 200);
const body = await response.json();
assert.equal(body.success, true);
assert.equal(body.partial, true);
assert.equal(body.sources.urlhaus.ok, false);
assert.equal(body.sources.urlhaus.reason, 'urlhaus_http_500');
});
test('API returns memory cache hit on repeated request', async () => {
delete process.env.URLHAUS_AUTH_KEY;
delete process.env.OTX_API_KEY;
delete process.env.ABUSEIPDB_API_KEY;
let feodoCalls = 0;
globalThis.fetch = createMockFetch({
feodo: () => {
feodoCalls += 1;
return jsonResponse([
{
ip_address: '1.2.3.4',
status: 'online',
last_online: '2026-02-15T10:00:00.000Z',
first_seen: '2026-02-14T10:00:00.000Z',
malware: 'QakBot',
country: 'GB',
lat: 51.5,
lon: -0.12,
},
]);
},
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
});
const first = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.13'));
assert.equal(first.status, 200);
assert.equal(first.headers.get('X-Cache'), 'MISS');
assert.equal(feodoCalls, 1);
globalThis.fetch = async () => {
throw new Error('network should not be hit for memory cache');
};
const second = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.13'));
assert.equal(second.status, 200);
assert.equal(second.headers.get('X-Cache'), 'MEMORY-HIT');
assert.equal(feodoCalls, 1);
});
test('API returns stale fallback when upstream fails after fresh cache TTL', async () => {
delete process.env.URLHAUS_AUTH_KEY;
delete process.env.OTX_API_KEY;
delete process.env.ABUSEIPDB_API_KEY;
const baseNow = Date.parse('2026-02-15T12:00:00.000Z');
const originalDateNow = Date.now;
Date.now = () => baseNow;
try {
globalThis.fetch = createMockFetch({
feodo: () => jsonResponse([
{
ip_address: '1.2.3.4',
status: 'online',
last_online: '2026-02-15T10:00:00.000Z',
first_seen: '2026-02-14T10:00:00.000Z',
malware: 'QakBot',
country: 'GB',
lat: 51.5,
lon: -0.12,
},
]),
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
});
const first = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.14'));
assert.equal(first.status, 200);
assert.equal(first.headers.get('X-Cache'), 'MISS');
Date.now = () => baseNow + (11 * 60 * 1000);
globalThis.fetch = async () => {
throw new Error('forced upstream failure');
};
const stale = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.14'));
assert.equal(stale.status, 200);
assert.equal(stale.headers.get('X-Cache'), 'STALE');
const body = await stale.json();
assert.equal(body.success, true);
assert.equal(Array.isArray(body.data), true);
assert.equal(body.data.length >= 1, true);
} finally {
Date.now = originalDateNow;
}
});
+27 -359
View File
@@ -1,77 +1,16 @@
// Tech Events API - Parses Techmeme ICS feed and dev.events RSS, returns structured events
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
/**
* Comprehensive city geocoding database (500+ cities worldwide).
* Extracted from the legacy api/tech-events.js endpoint.
*/
const ICS_URL = 'https://www.techmeme.com/newsy_events.ics';
const DEV_EVENTS_RSS = 'https://dev.events/rss.xml';
export interface CityCoord {
lat: number;
lng: number;
country: string;
virtual?: boolean;
}
// Curated major tech events that may fall off limited RSS feeds
// These are manually maintained for important conferences
const CURATED_EVENTS = [
{
id: 'step-dubai-2026',
title: 'STEP Dubai 2026',
type: 'conference',
location: 'Dubai Internet City, Dubai',
coords: { lat: 25.0956, lng: 55.1548, country: 'UAE', original: 'Dubai Internet City, Dubai' },
startDate: '2026-02-11',
endDate: '2026-02-12',
url: 'https://dubai.stepconference.com',
source: 'curated',
description: 'Intelligence Everywhere: The AI Economy - 8,000+ attendees, 400+ startups',
},
{
id: 'gitex-global-2026',
title: 'GITEX Global 2026',
type: 'conference',
location: 'Dubai World Trade Centre, Dubai',
coords: { lat: 25.2285, lng: 55.2867, country: 'UAE', original: 'Dubai World Trade Centre, Dubai' },
startDate: '2026-12-07',
endDate: '2026-12-11',
url: 'https://www.gitex.com',
source: 'curated',
description: 'World\'s largest tech & startup show',
},
{
id: 'token2049-dubai-2026',
title: 'TOKEN2049 Dubai 2026',
type: 'conference',
location: 'Dubai, UAE',
coords: { lat: 25.2048, lng: 55.2708, country: 'UAE', original: 'Dubai, UAE' },
startDate: '2026-04-29',
endDate: '2026-04-30',
url: 'https://www.token2049.com',
source: 'curated',
description: 'Premier crypto event in Dubai',
},
{
id: 'collision-2026',
title: 'Collision 2026',
type: 'conference',
location: 'Toronto, Canada',
coords: { lat: 43.6532, lng: -79.3832, country: 'Canada', original: 'Toronto, Canada' },
startDate: '2026-06-22',
endDate: '2026-06-25',
url: 'https://collisionconf.com',
source: 'curated',
description: 'North America\'s fastest growing tech conference',
},
{
id: 'web-summit-2026',
title: 'Web Summit 2026',
type: 'conference',
location: 'Lisbon, Portugal',
coords: { lat: 38.7223, lng: -9.1393, country: 'Portugal', original: 'Lisbon, Portugal' },
startDate: '2026-11-02',
endDate: '2026-11-05',
url: 'https://websummit.com',
source: 'curated',
description: 'The world\'s premier tech conference',
},
];
// Comprehensive city geocoding database (500+ cities worldwide)
const CITY_COORDS = {
export const CITY_COORDS: Record<string, CityCoord> = {
// North America - USA
'san francisco': { lat: 37.7749, lng: -122.4194, country: 'USA' },
'san jose': { lat: 37.3382, lng: -121.8863, country: 'USA' },
@@ -164,7 +103,7 @@ const CITY_COORDS = {
'tijuana': { lat: 32.5149, lng: -117.0382, country: 'Mexico' },
'cancun': { lat: 21.1619, lng: -86.8515, country: 'Mexico' },
'panama city': { lat: 8.9824, lng: -79.5199, country: 'Panama' },
'san jose': { lat: 9.9281, lng: -84.0907, country: 'Costa Rica' },
'san jose cr': { lat: 9.9281, lng: -84.0907, country: 'Costa Rica' },
// South America
'sao paulo': { lat: -23.5505, lng: -46.6333, country: 'Brazil' },
@@ -176,9 +115,9 @@ const CITY_COORDS = {
'buenos aires': { lat: -34.6037, lng: -58.3816, country: 'Argentina' },
'santiago': { lat: -33.4489, lng: -70.6693, country: 'Chile' },
'bogota': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
'bogotá': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
'bogot\u00e1': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
'medellin': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
'medellín': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
'medell\u00edn': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
'lima': { lat: -12.0464, lng: -77.0428, country: 'Peru' },
'caracas': { lat: 10.4806, lng: -66.9036, country: 'Venezuela' },
'montevideo': { lat: -34.9011, lng: -56.1645, country: 'Uruguay' },
@@ -186,7 +125,7 @@ const CITY_COORDS = {
// Europe - UK & Ireland
'london': { lat: 51.5074, lng: -0.1278, country: 'UK' },
'cambridge': { lat: 52.2053, lng: 0.1218, country: 'UK' },
'cambridge uk': { lat: 52.2053, lng: 0.1218, country: 'UK' },
'oxford': { lat: 51.7520, lng: -1.2577, country: 'UK' },
'manchester': { lat: 53.4808, lng: -2.2426, country: 'UK' },
'birmingham': { lat: 52.4862, lng: -1.8904, country: 'UK' },
@@ -214,12 +153,12 @@ const CITY_COORDS = {
'monaco': { lat: 43.7384, lng: 7.4246, country: 'Monaco' },
'berlin': { lat: 52.5200, lng: 13.4050, country: 'Germany' },
'munich': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
'münchen': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
'm\u00fcnchen': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
'frankfurt': { lat: 50.1109, lng: 8.6821, country: 'Germany' },
'hamburg': { lat: 53.5511, lng: 9.9937, country: 'Germany' },
'cologne': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
'köln': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
'düsseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
'k\u00f6ln': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
'd\u00fcsseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
'dusseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
'stuttgart': { lat: 48.7758, lng: 9.1829, country: 'Germany' },
'hanover': { lat: 52.3759, lng: 9.7320, country: 'Germany' },
@@ -237,9 +176,9 @@ const CITY_COORDS = {
'ghent': { lat: 51.0543, lng: 3.7174, country: 'Belgium' },
'luxembourg': { lat: 49.6116, lng: 6.1319, country: 'Luxembourg' },
'zurich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
'zürich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
'z\u00fcrich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
'geneva': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
'genève': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
'gen\u00e8ve': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
'basel': { lat: 47.5596, lng: 7.5886, country: 'Switzerland' },
'bern': { lat: 46.9480, lng: 7.4474, country: 'Switzerland' },
'lausanne': { lat: 46.5197, lng: 6.6323, country: 'Switzerland' },
@@ -257,7 +196,7 @@ const CITY_COORDS = {
'seville': { lat: 37.3891, lng: -5.9845, country: 'Spain' },
'sevilla': { lat: 37.3891, lng: -5.9845, country: 'Spain' },
'malaga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
'málaga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
'm\u00e1laga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
'bilbao': { lat: 43.2630, lng: -2.9350, country: 'Spain' },
'lisbon': { lat: 38.7223, lng: -9.1393, country: 'Portugal' },
'lisboa': { lat: 38.7223, lng: -9.1393, country: 'Portugal' },
@@ -283,11 +222,11 @@ const CITY_COORDS = {
// Europe - Northern
'stockholm': { lat: 59.3293, lng: 18.0686, country: 'Sweden' },
'gothenburg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
'göteborg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
'malmö': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
'g\u00f6teborg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
'malm\u00f6': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
'malmo': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
'copenhagen': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
'københavn': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
'k\u00f8benhavn': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
'aarhus': { lat: 56.1629, lng: 10.2039, country: 'Denmark' },
'oslo': { lat: 59.9139, lng: 10.7522, country: 'Norway' },
'bergen': { lat: 60.3913, lng: 5.3221, country: 'Norway' },
@@ -300,16 +239,16 @@ const CITY_COORDS = {
'warsaw': { lat: 52.2297, lng: 21.0122, country: 'Poland' },
'warszawa': { lat: 52.2297, lng: 21.0122, country: 'Poland' },
'krakow': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
'kraków': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
'krak\u00f3w': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
'wroclaw': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
'wrocław': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
'wroc\u0142aw': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
'gdansk': { lat: 54.3520, lng: 18.6466, country: 'Poland' },
'prague': { lat: 50.0755, lng: 14.4378, country: 'Czech Republic' },
'praha': { lat: 50.0755, lng: 14.4378, country: 'Czech Republic' },
'brno': { lat: 49.1951, lng: 16.6068, country: 'Czech Republic' },
'budapest': { lat: 47.4979, lng: 19.0402, country: 'Hungary' },
'bucharest': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
'bucurești': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
'bucure\u0219ti': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
'cluj-napoca': { lat: 46.7712, lng: 23.6236, country: 'Romania' },
'sofia': { lat: 42.6977, lng: 23.3219, country: 'Bulgaria' },
'belgrade': { lat: 44.7866, lng: 20.4489, country: 'Serbia' },
@@ -464,274 +403,3 @@ const CITY_COORDS = {
'virtual': { lat: 0, lng: 0, country: 'Virtual', virtual: true },
'hybrid': { lat: 0, lng: 0, country: 'Virtual', virtual: true },
};
function normalizeLocation(location) {
if (!location) return null;
// Clean up the location string
let normalized = location.toLowerCase().trim();
// Remove common suffixes/prefixes
normalized = normalized.replace(/^hybrid:\s*/i, '');
normalized = normalized.replace(/,\s*(usa|us|uk|canada)$/i, '');
// Direct lookup
if (CITY_COORDS[normalized]) {
return { ...CITY_COORDS[normalized], original: location };
}
// Try removing state/country suffix
const parts = normalized.split(',');
if (parts.length > 1) {
const city = parts[0].trim();
if (CITY_COORDS[city]) {
return { ...CITY_COORDS[city], original: location };
}
}
// Try fuzzy match (contains)
for (const [key, coords] of Object.entries(CITY_COORDS)) {
if (normalized.includes(key) || key.includes(normalized)) {
return { ...coords, original: location };
}
}
return null;
}
function parseICS(icsText) {
const events = [];
const eventBlocks = icsText.split('BEGIN:VEVENT').slice(1);
for (const block of eventBlocks) {
const summaryMatch = block.match(/SUMMARY:(.+)/);
const locationMatch = block.match(/LOCATION:(.+)/);
const dtstartMatch = block.match(/DTSTART;VALUE=DATE:(\d+)/);
const dtendMatch = block.match(/DTEND;VALUE=DATE:(\d+)/);
const urlMatch = block.match(/URL:(.+)/);
const uidMatch = block.match(/UID:(.+)/);
if (summaryMatch && dtstartMatch) {
const summary = summaryMatch[1].trim();
const location = locationMatch ? locationMatch[1].trim() : null;
const startDate = dtstartMatch[1];
const endDate = dtendMatch ? dtendMatch[1] : startDate;
const url = urlMatch ? urlMatch[1].trim() : null;
const uid = uidMatch ? uidMatch[1].trim() : null;
// Determine event type
let type = 'other';
if (summary.startsWith('Earnings:')) type = 'earnings';
else if (summary.startsWith('IPO')) type = 'ipo';
else if (location) type = 'conference';
// Parse coordinates if location exists
const coords = normalizeLocation(location);
events.push({
id: uid,
title: summary,
type,
location: location,
coords: coords,
startDate: `${startDate.slice(0, 4)}-${startDate.slice(4, 6)}-${startDate.slice(6, 8)}`,
endDate: `${endDate.slice(0, 4)}-${endDate.slice(4, 6)}-${endDate.slice(6, 8)}`,
url: url,
source: 'techmeme',
});
}
}
return events.sort((a, b) => a.startDate.localeCompare(b.startDate));
}
function parseDevEventsRSS(rssText) {
const events = [];
// Simple regex-based RSS parsing for edge runtime
const itemMatches = rssText.matchAll(/<item>([\s\S]*?)<\/item>/g);
for (const match of itemMatches) {
const item = match[1];
const titleMatch = item.match(/<title><!\[CDATA\[(.*?)\]\]><\/title>|<title>(.*?)<\/title>/);
const linkMatch = item.match(/<link>(.*?)<\/link>/);
const descMatch = item.match(/<description><!\[CDATA\[(.*?)\]\]><\/description>|<description>(.*?)<\/description>/s);
const guidMatch = item.match(/<guid[^>]*>(.*?)<\/guid>/);
const title = titleMatch ? (titleMatch[1] || titleMatch[2]) : null;
const link = linkMatch ? linkMatch[1] : null;
const description = descMatch ? (descMatch[1] || descMatch[2]) : '';
const guid = guidMatch ? guidMatch[1] : null;
if (!title) continue;
// Parse date from description: "EventName is happening on Month Day, Year"
const dateMatch = description.match(/on\s+(\w+\s+\d{1,2},?\s+\d{4})/i);
let startDate = null;
if (dateMatch) {
const parsed = new Date(dateMatch[1]);
if (!isNaN(parsed.getTime())) {
startDate = parsed.toISOString().split('T')[0];
}
}
// Parse location from description: various formats
let location = null;
const locationMatch = description.match(/(?:in|at)\s+([A-Za-z\s]+,\s*[A-Za-z\s]+)(?:\.|$)/i) ||
description.match(/Location:\s*([^<\n]+)/i);
if (locationMatch) {
location = locationMatch[1].trim();
}
// Check for "Online" events
if (description.toLowerCase().includes('online')) {
location = 'Online';
}
// Skip events without valid dates or in the past
if (!startDate) continue;
const eventDate = new Date(startDate);
const now = new Date();
now.setHours(0, 0, 0, 0);
if (eventDate < now) continue;
const coords = location && location !== 'Online' ? normalizeLocation(location) : null;
if (location === 'Online') {
// Mark as virtual
if (coords) coords.virtual = true;
}
events.push({
id: guid || `dev-events-${title.slice(0, 20)}`,
title: title,
type: 'conference',
location: location,
coords: coords || (location === 'Online' ? { virtual: true, original: 'Online' } : null),
startDate: startDate,
endDate: startDate, // RSS doesn't have end date
url: link,
source: 'dev.events',
});
}
return events;
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const url = new URL(req.url);
const type = url.searchParams.get('type'); // 'all', 'conferences', 'earnings', 'ipo'
const mappable = url.searchParams.get('mappable') === 'true'; // Only return events with coords
const limit = parseInt(url.searchParams.get('limit')) || 0; // Max events (0 = unlimited)
const days = parseInt(url.searchParams.get('days')) || 0; // Events within N days (0 = unlimited)
try {
// Fetch both sources in parallel
const [icsResponse, rssResponse] = await Promise.allSettled([
fetch(ICS_URL, {
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0)' },
}),
fetch(DEV_EVENTS_RSS, {
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0)' },
}),
]);
let events = [];
// Parse Techmeme ICS
if (icsResponse.status === 'fulfilled' && icsResponse.value.ok) {
const icsText = await icsResponse.value.text();
events.push(...parseICS(icsText));
} else {
console.warn('Failed to fetch Techmeme ICS');
}
// Parse dev.events RSS
if (rssResponse.status === 'fulfilled' && rssResponse.value.ok) {
const rssText = await rssResponse.value.text();
const devEvents = parseDevEventsRSS(rssText);
events.push(...devEvents);
} else {
console.warn('Failed to fetch dev.events RSS');
}
// Add curated events (major conferences that may fall off limited RSS feeds)
const now = new Date();
now.setHours(0, 0, 0, 0);
for (const curated of CURATED_EVENTS) {
const eventDate = new Date(curated.startDate);
if (eventDate >= now) {
events.push(curated);
}
}
// Deduplicate by title similarity (rough match)
const seen = new Set();
events = events.filter(e => {
const key = e.title.toLowerCase().replace(/[^a-z0-9]/g, '').slice(0, 30);
if (seen.has(key)) return false;
seen.add(key);
return true;
});
// Sort by date
events.sort((a, b) => a.startDate.localeCompare(b.startDate));
// Filter by type if specified
if (type && type !== 'all') {
events = events.filter(e => e.type === type);
}
// Filter to only mappable events if requested
if (mappable) {
events = events.filter(e => e.coords && !e.coords.virtual);
}
// Filter by time range if specified
if (days > 0) {
const cutoff = new Date();
cutoff.setDate(cutoff.getDate() + days);
events = events.filter(e => new Date(e.startDate) <= cutoff);
}
// Apply limit if specified
if (limit > 0) {
events = events.slice(0, limit);
}
// Add metadata
const conferences = events.filter(e => e.type === 'conference');
const mappableCount = conferences.filter(e => e.coords && !e.coords.virtual).length;
return new Response(JSON.stringify({
success: true,
count: events.length,
conferenceCount: conferences.length,
mappableCount,
lastUpdated: new Date().toISOString(),
events,
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
},
});
} catch (error) {
console.error('Tech events error:', error);
return new Response(JSON.stringify({
success: false,
error: error.message,
}), {
status: 500,
headers: {
'Content-Type': 'application/json',
...cors,
},
});
}
}
File diff suppressed because one or more lines are too long
-11
View File
@@ -1,11 +0,0 @@
export const config = { runtime: 'edge' };
export default async function handler() {
return new Response(JSON.stringify({ error: 'Not found' }), {
status: 404,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
},
});
}
+26 -1
View File
@@ -1,3 +1,4 @@
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
export const config = { runtime: 'edge' };
const RELEASES_URL = 'https://api.github.com/repos/koala73/worldmonitor/releases/latest';
@@ -11,9 +12,30 @@ const PLATFORM_PATTERNS = {
'linux-appimage': (name) => name.endsWith('_amd64.AppImage'),
};
const VARIANT_PREFIXES = {
full: ['world-monitor'],
world: ['world-monitor'],
tech: ['tech-monitor'],
finance: ['finance-monitor'],
};
function findAssetForVariant(assets, variant, platformMatcher) {
const prefixes = VARIANT_PREFIXES[variant] ?? null;
if (!prefixes) return null;
return assets.find((asset) => {
const assetName = String(asset?.name || '').toLowerCase();
const hasVariantPrefix = prefixes.some((prefix) =>
assetName.startsWith(`${prefix.toLowerCase()}_`) || assetName.startsWith(`${prefix.toLowerCase()}-`)
);
return hasVariantPrefix && platformMatcher(String(asset?.name || ''));
}) ?? null;
}
export default async function handler(req) {
const url = new URL(req.url);
const platform = url.searchParams.get('platform');
const variant = (url.searchParams.get('variant') || '').toLowerCase();
if (!platform || !PLATFORM_PATTERNS[platform]) {
return Response.redirect(RELEASES_PAGE, 302);
@@ -33,7 +55,10 @@ export default async function handler(req) {
const release = await res.json();
const matcher = PLATFORM_PATTERNS[platform];
const asset = release.assets?.find((a) => matcher(a.name));
const assets = Array.isArray(release.assets) ? release.assets : [];
const asset = variant
? findAssetForVariant(assets, variant, matcher)
: assets.find((a) => matcher(String(a?.name || '')));
if (!asset) {
return Response.redirect(RELEASES_PAGE, 302);
-35
View File
@@ -1,35 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(request) {
const cors = getCorsHeaders(request);
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
try {
const response = await fetch(
'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson',
{
headers: {
'Accept': 'application/json',
},
}
);
const data = await response.text();
return new Response(data, {
status: response.status,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
},
});
} catch (error) {
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
status: 500,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
}
-163
View File
@@ -1,163 +0,0 @@
export const config = { runtime: 'edge' };
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
const CACHE_TTL = 900;
let cachedResponse = null;
let cacheTimestamp = 0;
const ETF_LIST = [
{ ticker: 'IBIT', issuer: 'BlackRock' },
{ ticker: 'FBTC', issuer: 'Fidelity' },
{ ticker: 'ARKB', issuer: 'ARK/21Shares' },
{ ticker: 'BITB', issuer: 'Bitwise' },
{ ticker: 'GBTC', issuer: 'Grayscale' },
{ ticker: 'HODL', issuer: 'VanEck' },
{ ticker: 'BRRR', issuer: 'Valkyrie' },
{ ticker: 'EZBC', issuer: 'Franklin' },
{ ticker: 'BTCO', issuer: 'Invesco' },
{ ticker: 'BTCW', issuer: 'WisdomTree' },
];
async function fetchChart(ticker) {
const url = `https://query1.finance.yahoo.com/v8/finance/chart/${ticker}?range=5d&interval=1d`;
const controller = new AbortController();
const id = setTimeout(() => controller.abort(), 8000);
try {
const res = await fetch(url, { signal: controller.signal });
if (!res.ok) return null;
return await res.json();
} catch {
return null;
} finally {
clearTimeout(id);
}
}
function parseChartData(chart, ticker, issuer) {
try {
const result = chart?.chart?.result?.[0];
if (!result) return null;
const quote = result.indicators?.quote?.[0];
const closes = quote?.close || [];
const volumes = quote?.volume || [];
const validCloses = closes.filter(p => p != null);
const validVolumes = volumes.filter(v => v != null);
if (validCloses.length < 2) return null;
const latestPrice = validCloses[validCloses.length - 1];
const prevPrice = validCloses[validCloses.length - 2];
const priceChange = prevPrice ? ((latestPrice - prevPrice) / prevPrice * 100) : 0;
const latestVolume = validVolumes.length > 0 ? validVolumes[validVolumes.length - 1] : 0;
const avgVolume = validVolumes.length > 1
? validVolumes.slice(0, -1).reduce((a, b) => a + b, 0) / (validVolumes.length - 1)
: latestVolume;
// Estimate flow direction from price change + volume
const volumeRatio = avgVolume > 0 ? latestVolume / avgVolume : 1;
const direction = priceChange > 0.1 ? 'inflow' : priceChange < -0.1 ? 'outflow' : 'neutral';
const estFlowMagnitude = latestVolume * latestPrice * (priceChange > 0 ? 1 : -1) * 0.1;
return {
ticker,
issuer,
price: +latestPrice.toFixed(2),
priceChange: +priceChange.toFixed(2),
volume: latestVolume,
avgVolume: Math.round(avgVolume),
volumeRatio: +volumeRatio.toFixed(2),
direction,
estFlow: Math.round(estFlowMagnitude),
};
} catch {
return null;
}
}
function buildFallbackResult() {
return {
timestamp: new Date().toISOString(),
summary: {
etfCount: 0,
totalVolume: 0,
totalEstFlow: 0,
netDirection: 'UNAVAILABLE',
inflowCount: 0,
outflowCount: 0,
},
etfs: [],
unavailable: true,
};
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: cors });
}
return new Response(null, { status: 204, headers: cors });
}
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
}
const now = Date.now();
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
return new Response(JSON.stringify(cachedResponse), {
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=1800` },
});
}
try {
const charts = await Promise.allSettled(
ETF_LIST.map(etf => fetchChart(etf.ticker))
);
const etfs = [];
for (let i = 0; i < ETF_LIST.length; i++) {
const chart = charts[i].status === 'fulfilled' ? charts[i].value : null;
if (chart) {
const parsed = parseChartData(chart, ETF_LIST[i].ticker, ETF_LIST[i].issuer);
if (parsed) etfs.push(parsed);
}
}
const totalVolume = etfs.reduce((sum, e) => sum + e.volume, 0);
const totalEstFlow = etfs.reduce((sum, e) => sum + e.estFlow, 0);
const inflowCount = etfs.filter(e => e.direction === 'inflow').length;
const outflowCount = etfs.filter(e => e.direction === 'outflow').length;
const result = {
timestamp: new Date().toISOString(),
summary: {
etfCount: etfs.length,
totalVolume,
totalEstFlow,
netDirection: totalEstFlow > 0 ? 'NET INFLOW' : totalEstFlow < 0 ? 'NET OUTFLOW' : 'NEUTRAL',
inflowCount,
outflowCount,
},
etfs: etfs.sort((a, b) => b.volume - a.volume),
};
cachedResponse = result;
cacheTimestamp = now;
return new Response(JSON.stringify(result), {
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=1800` },
});
} catch (err) {
const fallback = cachedResponse || buildFallbackResult();
cachedResponse = fallback;
cacheTimestamp = now;
return new Response(JSON.stringify(fallback), {
status: 200,
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
});
}
}
-29
View File
@@ -1,29 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
try {
const response = await fetch('https://nasstatus.faa.gov/api/airport-status-information', {
headers: { 'Accept': 'application/xml' },
});
const data = await response.text();
return new Response(data, {
status: response.status,
headers: {
'Content-Type': 'application/xml',
...cors,
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
},
});
} catch (error) {
return new Response(`<error>${error.message}</error>`, {
status: 500,
headers: { 'Content-Type': 'application/xml' },
});
}
}
-115
View File
@@ -1,115 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
const SYMBOL_PATTERN = /^[A-Za-z0-9.^]+$/;
const MAX_SYMBOLS = 20;
const MAX_SYMBOL_LENGTH = 10;
function validateSymbols(symbolsParam) {
if (!symbolsParam) return null;
const symbols = symbolsParam
.split(',')
.map(s => s.trim().toUpperCase())
.filter(s => s.length <= MAX_SYMBOL_LENGTH && SYMBOL_PATTERN.test(s))
.slice(0, MAX_SYMBOLS);
return symbols.length > 0 ? symbols : null;
}
async function fetchQuote(symbol, apiKey) {
const url = `https://finnhub.io/api/v1/quote?symbol=${encodeURIComponent(symbol)}&token=${apiKey}`;
const response = await fetch(url, {
headers: { 'Accept': 'application/json' },
});
if (!response.ok) {
return { symbol, error: `HTTP ${response.status}` };
}
const data = await response.json();
// Finnhub returns { c, d, dp, h, l, o, pc, t } where:
// c = current price, d = change, dp = percent change
// h = high, l = low, o = open, pc = previous close, t = timestamp
if (data.c === 0 && data.h === 0 && data.l === 0) {
return { symbol, error: 'No data available' };
}
return {
symbol,
price: data.c,
change: data.d,
changePercent: data.dp,
high: data.h,
low: data.l,
open: data.o,
previousClose: data.pc,
timestamp: data.t,
};
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
const apiKey = process.env.FINNHUB_API_KEY;
if (!apiKey) {
return new Response(JSON.stringify({ quotes: [], skipped: true, reason: 'FINNHUB_API_KEY not configured' }), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30', ...corsHeaders },
});
}
const url = new URL(req.url);
const symbols = validateSymbols(url.searchParams.get('symbols'));
if (!symbols) {
return new Response(JSON.stringify({ error: 'Invalid or missing symbols parameter' }), {
status: 400,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
try {
// Fetch all quotes in parallel (Finnhub allows 60 req/min on free tier)
const quotes = await Promise.all(
symbols.map(symbol => fetchQuote(symbol, apiKey))
);
return new Response(JSON.stringify({ quotes }), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
...corsHeaders,
},
});
} catch (error) {
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
status: 500,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
}
-145
View File
@@ -1,145 +0,0 @@
/**
* NASA FIRMS Satellite Fire Detection API
* Proxies requests to NASA FIRMS to avoid CORS and protect API key
* Returns parsed fire data for monitored conflict regions
*
* GET ?region=Ukraine&days=1 fires for one region
* GET ?days=1 fires for all monitored regions
*/
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const FIRMS_API_KEY = process.env.NASA_FIRMS_API_KEY || process.env.FIRMS_API_KEY || '';
const FIRMS_BASE = 'https://firms.modaps.eosdis.nasa.gov/api/area/csv';
const SOURCE = 'VIIRS_SNPP_NRT';
// Bounding boxes as west,south,east,north
const MONITORED_REGIONS = {
'Ukraine': { bbox: '22,44,40,53' },
'Russia': { bbox: '20,50,180,82' },
'Iran': { bbox: '44,25,63,40' },
'Israel/Gaza': { bbox: '34,29,36,34' },
'Syria': { bbox: '35,32,42,37' },
'Taiwan': { bbox: '119,21,123,26' },
'North Korea': { bbox: '124,37,131,43' },
'Saudi Arabia': { bbox: '34,16,56,32' },
'Turkey': { bbox: '26,36,45,42' },
};
// Map VIIRS confidence letters to numeric
function parseConfidence(c) {
if (c === 'h') return 95;
if (c === 'n') return 50;
if (c === 'l') return 20;
return parseInt(c) || 0;
}
function parseCSV(csv) {
const lines = csv.trim().split('\n');
if (lines.length < 2) return [];
const headers = lines[0].split(',').map(h => h.trim());
const results = [];
for (let i = 1; i < lines.length; i++) {
const vals = lines[i].split(',').map(v => v.trim());
if (vals.length < headers.length) continue;
const row = {};
headers.forEach((h, idx) => { row[h] = vals[idx]; });
results.push({
lat: parseFloat(row.latitude),
lon: parseFloat(row.longitude),
brightness: parseFloat(row.bright_ti4) || 0,
scan: parseFloat(row.scan) || 0,
track: parseFloat(row.track) || 0,
acq_date: row.acq_date || '',
acq_time: row.acq_time || '',
satellite: row.satellite || '',
confidence: parseConfidence(row.confidence),
bright_t31: parseFloat(row.bright_ti5) || 0,
frp: parseFloat(row.frp) || 0,
daynight: row.daynight || '',
});
}
return results;
}
export default async function handler(request) {
const cors = getCorsHeaders(request);
if (request.method === 'OPTIONS') return new Response(null, { status: 204, headers: cors });
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
if (!FIRMS_API_KEY) {
return json({ regions: {}, totalCount: 0, skipped: true, reason: 'NASA_FIRMS_API_KEY not configured', source: SOURCE, days: 0, timestamp: new Date().toISOString() });
}
try {
const { searchParams } = new URL(request.url);
const regionName = searchParams.get('region');
const days = Math.min(parseInt(searchParams.get('days')) || 1, 5);
const regions = regionName
? { [regionName]: MONITORED_REGIONS[regionName] }
: MONITORED_REGIONS;
if (regionName && !MONITORED_REGIONS[regionName]) {
return json({ error: `Unknown region: ${regionName}` }, 400);
}
const allFires = {};
let totalCount = 0;
// Fetch regions in parallel (max 10)
const entries = Object.entries(regions);
const results = await Promise.allSettled(
entries.map(async ([name, { bbox }]) => {
const url = `${FIRMS_BASE}/${FIRMS_API_KEY}/${SOURCE}/${bbox}/${days}`;
const res = await fetch(url, {
headers: { 'Accept': 'text/csv' },
});
if (!res.ok) throw new Error(`FIRMS ${res.status} for ${name}`);
const csv = await res.text();
return { name, fires: parseCSV(csv) };
})
);
for (const result of results) {
if (result.status === 'fulfilled') {
const { name, fires } = result.value;
allFires[name] = fires;
totalCount += fires.length;
} else {
console.error('[FIRMS]', result.reason?.message);
}
}
return json({
regions: allFires,
totalCount,
source: SOURCE,
days,
timestamp: new Date().toISOString(),
});
} catch (err) {
console.error('[FIRMS] Error:', err);
return json({ error: 'Failed to fetch fire data' }, 500);
}
}
function json(data, status = 200) {
return new Response(JSON.stringify(data), {
status,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', // 10 min cache
},
});
}
-89
View File
@@ -1,89 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
const url = new URL(req.url);
const seriesId = url.searchParams.get('series_id');
const observationStart = url.searchParams.get('observation_start');
const observationEnd = url.searchParams.get('observation_end');
if (!seriesId) {
return new Response(JSON.stringify({ error: 'Missing series_id parameter' }), {
status: 400,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
const apiKey = process.env.FRED_API_KEY;
if (!apiKey) {
return new Response(JSON.stringify({
observations: [],
skipped: true,
reason: 'FRED_API_KEY not configured',
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
...corsHeaders,
},
});
}
try {
const params = new URLSearchParams({
series_id: seriesId,
api_key: apiKey,
file_type: 'json',
sort_order: 'desc',
limit: '10',
});
if (observationStart) params.set('observation_start', observationStart);
if (observationEnd) params.set('observation_end', observationEnd);
const fredUrl = `https://api.stlouisfed.org/fred/series/observations?${params}`;
const response = await fetch(fredUrl, {
headers: { 'Accept': 'application/json' },
});
const data = await response.json();
return new Response(JSON.stringify(data), {
status: response.status,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600',
...corsHeaders,
},
});
} catch (error) {
return new Response(JSON.stringify({ error: error.message }), {
status: 500,
headers: { 'Content-Type': 'application/json', ...corsHeaders },
});
}
}
+1
View File
@@ -1,3 +1,4 @@
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
-68
View File
@@ -1,68 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
const MAX_RECORDS = 20;
const DEFAULT_RECORDS = 10;
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const url = new URL(req.url);
const query = url.searchParams.get('query');
const maxrecords = Math.min(
parseInt(url.searchParams.get('maxrecords') || DEFAULT_RECORDS, 10),
MAX_RECORDS
);
const timespan = url.searchParams.get('timespan') || '72h';
if (!query || query.length < 2) {
return new Response(JSON.stringify({ error: 'Query parameter required' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
try {
const gdeltUrl = new URL('https://api.gdeltproject.org/api/v2/doc/doc');
gdeltUrl.searchParams.set('query', query);
gdeltUrl.searchParams.set('mode', 'artlist');
gdeltUrl.searchParams.set('maxrecords', maxrecords.toString());
gdeltUrl.searchParams.set('format', 'json');
gdeltUrl.searchParams.set('sort', 'date');
gdeltUrl.searchParams.set('timespan', timespan);
const response = await fetch(gdeltUrl.toString());
if (!response.ok) {
throw new Error(`GDELT returned ${response.status}`);
}
const data = await response.json();
const articles = (data.articles || []).map(article => ({
title: article.title,
url: article.url,
source: article.domain || article.source?.domain,
date: article.seendate,
image: article.socialimage,
language: article.language,
tone: article.tone,
}));
return new Response(JSON.stringify({ articles, query }), {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
},
});
} catch (error) {
return new Response(JSON.stringify({ error: error.message, articles: [] }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
-86
View File
@@ -1,86 +0,0 @@
// GDELT Geo API proxy with security hardening
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
const ALLOWED_FORMATS = ['geojson', 'json', 'csv'];
const MAX_RECORDS = 500;
const MIN_RECORDS = 1;
const ALLOWED_TIMESPANS = ['1d', '7d', '14d', '30d', '60d', '90d'];
function validateMaxRecords(val) {
const num = parseInt(val, 10);
if (isNaN(num)) return 250;
return Math.max(MIN_RECORDS, Math.min(MAX_RECORDS, num));
}
function validateFormat(val) {
return ALLOWED_FORMATS.includes(val) ? val : 'geojson';
}
function validateTimespan(val) {
return ALLOWED_TIMESPANS.includes(val) ? val : '7d';
}
function sanitizeQuery(val) {
if (!val || typeof val !== 'string') return 'protest';
return val.slice(0, 200).replace(/[<>\"']/g, '');
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
if (req.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: cors });
}
if (req.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
const url = new URL(req.url);
const query = sanitizeQuery(url.searchParams.get('query'));
const format = validateFormat(url.searchParams.get('format') || 'geojson');
const maxrecords = validateMaxRecords(url.searchParams.get('maxrecords') || '250');
const timespan = validateTimespan(url.searchParams.get('timespan') || '7d');
try {
const response = await fetch(
`https://api.gdeltproject.org/api/v2/geo/geo?query=${encodeURIComponent(query)}&format=${format}&maxrecords=${maxrecords}&timespan=${timespan}`
);
if (!response.ok) {
return new Response(JSON.stringify({ error: 'Upstream service unavailable' }), {
status: 502,
headers: {
'Content-Type': 'application/json',
...cors,
},
});
}
const data = await response.text();
return new Response(data, {
status: 200,
headers: {
'Content-Type': format === 'csv' ? 'text/csv' : 'application/json',
...cors,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
},
});
} catch (error) {
console.error('[GDELT] Fetch error:', error.message);
return new Response(JSON.stringify({ error: 'Failed to fetch GDELT data' }), {
status: 500,
headers: {
'Content-Type': 'application/json',
...cors,
},
});
}
}
-90
View File
@@ -1,90 +0,0 @@
export const config = { runtime: 'edge' };
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
// Fetch trending GitHub repositories
// Uses unofficial GitHub trending scraper API
export default async function handler(request) {
const cors = getCorsHeaders(request);
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
try {
const { searchParams } = new URL(request.url);
const language = searchParams.get('language') || 'python'; // python, javascript, typescript, etc.
const since = searchParams.get('since') || 'daily'; // daily, weekly, monthly
const spoken_language = searchParams.get('spoken_language') || ''; // en, zh, etc.
// Using GitHub trending API (unofficial)
// Alternative: https://gh-trending-api.herokuapp.com/repositories
const baseUrl = 'https://api.gitterapp.com/repositories';
const queryParams = new URLSearchParams({
language: language,
since: since,
});
if (spoken_language) {
queryParams.append('spoken_language_code', spoken_language);
}
const apiUrl = `${baseUrl}?${queryParams.toString()}`;
const response = await fetch(apiUrl, {
headers: {
'Accept': 'application/json',
'User-Agent': 'WorldMonitor/1.0 (Tech Tracker)',
},
signal: AbortSignal.timeout(10000), // 10 second timeout
});
if (!response.ok) {
// Fallback: try alternative API
const fallbackUrl = `https://gh-trending-api.herokuapp.com/repositories/${language}?since=${since}`;
const fallbackResponse = await fetch(fallbackUrl, {
headers: {
'Accept': 'application/json',
},
signal: AbortSignal.timeout(10000),
});
if (!fallbackResponse.ok) {
throw new Error(`GitHub trending API returned ${fallbackResponse.status}`);
}
const data = await fallbackResponse.json();
return new Response(JSON.stringify(data), {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300', // 30 min cache
},
});
}
const data = await response.json();
return new Response(JSON.stringify(data), {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300', // 30 min cache
},
});
} catch (error) {
return new Response(
JSON.stringify({
error: 'Failed to fetch GitHub trending data',
message: error.message
}),
{
status: 500,
headers: {
'Content-Type': 'application/json',
...cors,
},
}
);
}
}
-296
View File
@@ -1,296 +0,0 @@
/**
* Groq API Summarization Endpoint with Redis Caching
* Uses Llama 3.1 8B Instant for high-throughput summarization
* Free tier: 14,400 requests/day (14x more than 70B model)
* Server-side Redis cache for cross-user deduplication
*/
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
const MODEL = 'llama-3.1-8b-instant'; // 14.4K RPD vs 1K for 70b
const CACHE_TTL_SECONDS = 86400; // 24 hours
const CACHE_VERSION = 'v3';
function getCacheKey(headlines, mode, geoContext = '', variant = 'full', lang = 'en') {
const sorted = headlines.slice(0, 8).sort().join('|');
const geoHash = geoContext ? ':g' + hashString(geoContext).slice(0, 6) : '';
const hash = hashString(`${mode}:${sorted}`);
const normalizedVariant = typeof variant === 'string' && variant ? variant.toLowerCase() : 'full';
const normalizedLang = typeof lang === 'string' && lang ? lang.toLowerCase() : 'en';
if (mode === 'translate') {
const targetLang = normalizedVariant || normalizedLang;
return `summary:${CACHE_VERSION}:${mode}:${targetLang}:${hash}${geoHash}`;
}
return `summary:${CACHE_VERSION}:${mode}:${normalizedVariant}:${normalizedLang}:${hash}${geoHash}`;
}
// Deduplicate similar headlines (same story from different sources)
function deduplicateHeadlines(headlines) {
const seen = new Set();
const unique = [];
for (const headline of headlines) {
// Normalize: lowercase, remove punctuation, collapse whitespace
const normalized = headline.toLowerCase()
.replace(/[^\w\s]/g, '')
.replace(/\s+/g, ' ')
.trim();
// Extract key words (4+ chars) for similarity check
const words = new Set(normalized.split(' ').filter(w => w.length >= 4));
// Check if this headline is too similar to any we've seen
let isDuplicate = false;
for (const seenWords of seen) {
const intersection = [...words].filter(w => seenWords.has(w));
const similarity = intersection.length / Math.min(words.size, seenWords.size);
if (similarity > 0.6) {
isDuplicate = true;
break;
}
}
if (!isDuplicate) {
seen.add(words);
unique.push(headline);
}
}
return unique;
}
export default async function handler(request) {
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
if (request.method !== 'POST') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
const apiKey = process.env.GROQ_API_KEY;
if (!apiKey) {
return new Response(JSON.stringify({ summary: null, fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
if (contentLength > 51200) {
return new Response(JSON.stringify({ error: 'Payload too large' }), {
status: 413,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
try {
const { headlines, mode = 'brief', geoContext = '', variant = 'full', lang = 'en' } = await request.json();
if (!headlines || !Array.isArray(headlines) || headlines.length === 0) {
return new Response(JSON.stringify({ error: 'Headlines array required' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
// Check cache first
const cacheKey = getCacheKey(headlines, mode, geoContext, variant, lang);
const cached = await getCachedJson(cacheKey);
if (cached && typeof cached === 'object' && cached.summary) {
console.log('[Groq] Cache hit:', cacheKey);
return new Response(JSON.stringify({
summary: cached.summary,
model: cached.model || MODEL,
provider: 'cache',
cached: true,
}), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
// Deduplicate similar headlines (same story from multiple sources)
const uniqueHeadlines = deduplicateHeadlines(headlines.slice(0, 8));
const headlineText = uniqueHeadlines.map((h, i) => `${i + 1}. ${h}`).join('\n');
let systemPrompt, userPrompt;
// Include intelligence synthesis context in prompt if available
const intelSection = geoContext ? `\n\n${geoContext}` : '';
// Current date context for LLM (models may have outdated knowledge)
const isTechVariant = variant === 'tech';
const dateContext = `Current date: ${new Date().toISOString().split('T')[0]}.${isTechVariant ? '' : ' Donald Trump is the current US President (second term, inaugurated Jan 2025).'}`;
// Language instruction
const langInstruction = lang && lang !== 'en' ? `\nIMPORTANT: Output the summary in ${lang.toUpperCase()} language.` : '';
if (mode === 'brief') {
if (isTechVariant) {
// Tech variant: focus on startups, AI, funding, product launches
systemPrompt = `${dateContext}
Summarize the key tech/startup development in 2-3 sentences.
Rules:
- Focus ONLY on technology, startups, AI, funding, product launches, or developer news
- IGNORE political news, trade policy, tariffs, government actions unless directly about tech regulation
- Lead with the company/product/technology name
- Start directly: "OpenAI announced...", "A new $50M Series B...", "GitHub released..."
- No bullet points, no meta-commentary${langInstruction}`;
} else {
// Full variant: geopolitical focus
systemPrompt = `${dateContext}
Summarize the key development in 2-3 sentences.
Rules:
- Lead with WHAT happened and WHERE - be specific
- NEVER start with "Breaking news", "Good evening", "Tonight", or TV-style openings
- Start directly with the subject: "Iran's regime...", "The US Treasury...", "Protests in..."
- CRITICAL FOCAL POINTS are the main actors - mention them by name
- If focal points show news + signals convergence, that's the lead
- No bullet points, no meta-commentary${langInstruction}`;
}
userPrompt = `Summarize the top story:\n${headlineText}${intelSection}`;
} else if (mode === 'analysis') {
if (isTechVariant) {
systemPrompt = `${dateContext}
Analyze the tech/startup trend in 2-3 sentences.
Rules:
- Focus ONLY on technology implications: funding trends, AI developments, market shifts, product strategy
- IGNORE political implications, trade wars, government unless directly about tech policy
- Lead with the insight for tech industry
- Connect to startup ecosystem, VC trends, or technical implications`;
} else {
systemPrompt = `${dateContext}
Provide analysis in 2-3 sentences. Be direct and specific.
Rules:
- Lead with the insight - what's significant and why
- NEVER start with "Breaking news", "Tonight", "The key/dominant narrative is"
- Start with substance: "Iran faces...", "The escalation in...", "Multiple signals suggest..."
- CRITICAL FOCAL POINTS are your main actors - explain WHY they matter
- If focal points show news-signal correlation, flag as escalation
- Connect dots, be specific about implications`;
}
userPrompt = isTechVariant
? `What's the key tech trend or development?\n${headlineText}${intelSection}`
: `What's the key pattern or risk?\n${headlineText}${intelSection}`;
} else if (mode === 'translate') {
const targetLang = variant; // In translate mode, variant param holds the target language code (e.g., 'fr', 'es')
systemPrompt = `You are a professional news translator. Translate the following news headlines/summaries into ${targetLang}.
Rules:
- Maintain the original tone and journalistic style.
- Do NOT add any conversational filler (e.g., "Here is the translation").
- Output ONLY the translated text.
- If the text is already in ${targetLang}, return it as is.`;
userPrompt = `Translate to ${targetLang}:\n${headlines[0]}`;
} else {
systemPrompt = isTechVariant
? `${dateContext}\n\nSynthesize tech news in 2 sentences. Focus on startups, AI, funding, products. Ignore politics unless directly about tech regulation.${langInstruction}`
: `${dateContext}\n\nSynthesize in 2 sentences max. Lead with substance. NEVER start with "Breaking news" or "Tonight" - just state the insight directly. CRITICAL focal points with news-signal convergence are significant.${langInstruction}`;
userPrompt = `Key takeaway:\n${headlineText}${intelSection}`;
}
const response = await fetch(GROQ_API_URL, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: MODEL,
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userPrompt },
],
temperature: 0.3,
max_tokens: 150,
top_p: 0.9,
}),
});
if (!response.ok) {
const errorText = await response.text();
console.error('[Groq] API error:', response.status, errorText);
// Return fallback signal for rate limiting
if (response.status === 429) {
return new Response(JSON.stringify({ error: 'Rate limited', fallback: true }), {
status: 429,
headers: { 'Content-Type': 'application/json' },
});
}
return new Response(JSON.stringify({ error: 'Groq API error', fallback: true }), {
status: response.status,
headers: { 'Content-Type': 'application/json' },
});
}
const data = await response.json();
const summary = data.choices?.[0]?.message?.content?.trim();
if (!summary) {
return new Response(JSON.stringify({ error: 'Empty response', fallback: true }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
// Store in cache
await setCachedJson(cacheKey, {
summary,
model: MODEL,
timestamp: Date.now(),
}, CACHE_TTL_SECONDS);
return new Response(JSON.stringify({
summary,
model: MODEL,
provider: 'groq',
cached: false,
tokens: data.usage?.total_tokens || 0,
}), {
status: 200,
headers: {
...corsHeaders,
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
},
});
} catch (error) {
console.error('[Groq] Error:', error.name, error.message, error.stack?.split('\n')[1]);
return new Response(JSON.stringify({
error: error.message,
errorType: error.name,
fallback: true
}), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
-98
View File
@@ -1,98 +0,0 @@
export const config = { runtime: 'edge' };
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
// Fetch Hacker News front page stories
// Uses official HackerNews Firebase API
const ALLOWED_STORY_TYPES = new Set(['top', 'new', 'best', 'ask', 'show', 'job']);
const DEFAULT_LIMIT = 30;
const MAX_LIMIT = 60;
const MAX_CONCURRENCY = 10;
function parseLimit(rawLimit) {
const parsed = Number.parseInt(rawLimit || '', 10);
if (!Number.isFinite(parsed)) return DEFAULT_LIMIT;
return Math.max(1, Math.min(MAX_LIMIT, parsed));
}
export default async function handler(request) {
const cors = getCorsHeaders(request);
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
try {
const { searchParams } = new URL(request.url);
const requestedType = searchParams.get('type') || 'top';
const storyType = ALLOWED_STORY_TYPES.has(requestedType) ? requestedType : 'top';
const limit = parseLimit(searchParams.get('limit'));
// HackerNews official Firebase API
const storiesUrl = `https://hacker-news.firebaseio.com/v0/${storyType}stories.json`;
// Fetch story IDs
const storiesResponse = await fetch(storiesUrl, {
signal: AbortSignal.timeout(10000),
});
if (!storiesResponse.ok) {
throw new Error(`HackerNews API returned ${storiesResponse.status}`);
}
const storyIds = await storiesResponse.json();
if (!Array.isArray(storyIds)) {
throw new Error('HackerNews API returned unexpected payload');
}
const limitedIds = storyIds.slice(0, limit);
// Fetch story details in bounded batches to avoid unbounded fan-out.
const stories = [];
for (let i = 0; i < limitedIds.length; i += MAX_CONCURRENCY) {
const batchIds = limitedIds.slice(i, i + MAX_CONCURRENCY);
const storyPromises = batchIds.map(async (id) => {
const storyUrl = `https://hacker-news.firebaseio.com/v0/item/${id}.json`;
try {
const response = await fetch(storyUrl, {
signal: AbortSignal.timeout(5000),
});
if (response.ok) {
return await response.json();
}
return null;
} catch (error) {
console.error(`Failed to fetch story ${id}:`, error);
return null;
}
});
const batchResults = await Promise.all(storyPromises);
stories.push(...batchResults.filter((story) => story !== null));
}
return new Response(JSON.stringify({
type: storyType,
stories: stories,
total: stories.length,
timestamp: new Date().toISOString()
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', // 5 min cache
},
});
} catch (error) {
return new Response(
JSON.stringify({
error: 'Failed to fetch Hacker News data',
message: error.message
}),
{
status: 500,
headers: {
'Content-Type': 'application/json',
...cors,
},
}
);
}
}
-148
View File
@@ -1,148 +0,0 @@
// HDX HAPI (Humanitarian API) proxy
// Returns aggregated conflict event counts per country
// Source: ACLED data aggregated monthly by HDX
export const config = { runtime: 'edge' };
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
const CACHE_KEY = 'hapi:conflict-events:v2';
const CACHE_TTL_SECONDS = 6 * 60 * 60; // 6 hours
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
const RESPONSE_CACHE_CONTROL = 'public, max-age=1800';
// In-memory fallback when Redis is unavailable.
let fallbackCache = { data: null, timestamp: 0 };
function isValidResult(data) {
return Boolean(
data &&
typeof data === 'object' &&
Array.isArray(data.countries)
);
}
function toErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'unknown error');
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const now = Date.now();
const cached = await getCachedJson(CACHE_KEY);
if (isValidResult(cached)) {
recordCacheTelemetry('/api/hapi', 'REDIS-HIT');
return Response.json(cached, {
status: 200,
headers: {
...cors,
'Cache-Control': RESPONSE_CACHE_CONTROL,
'X-Cache': 'REDIS-HIT',
},
});
}
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
recordCacheTelemetry('/api/hapi', 'MEMORY-HIT');
return Response.json(fallbackCache.data, {
status: 200,
headers: {
...cors,
'Cache-Control': RESPONSE_CACHE_CONTROL,
'X-Cache': 'MEMORY-HIT',
},
});
}
try {
const appId = btoa('worldmonitor:monitor@worldmonitor.app');
const response = await fetch(
`https://hapi.humdata.org/api/v2/coordination-context/conflict-events?output_format=json&limit=1000&offset=0&app_identifier=${appId}`,
{
headers: {
'Accept': 'application/json',
},
}
);
if (!response.ok) {
throw new Error(`HAPI API error: ${response.status}`);
}
const rawData = await response.json();
const records = rawData.data || [];
// Each record is (country, event_type, month) — aggregate across event types per country
// Keep only the most recent month per country
const byCountry = {};
for (const r of records) {
const iso3 = r.location_code || '';
if (!iso3) continue;
const month = r.reference_period_start || '';
const eventType = (r.event_type || '').toLowerCase();
const events = r.events || 0;
const fatalities = r.fatalities || 0;
if (!byCountry[iso3]) {
byCountry[iso3] = { iso3, locationName: r.location_name || '', month, eventsTotal: 0, eventsPoliticalViolence: 0, eventsCivilianTargeting: 0, eventsDemonstrations: 0, fatalitiesTotalPoliticalViolence: 0, fatalitiesTotalCivilianTargeting: 0 };
}
const c = byCountry[iso3];
if (month > c.month) {
// Newer month — reset
c.month = month;
c.eventsTotal = 0; c.eventsPoliticalViolence = 0; c.eventsCivilianTargeting = 0; c.eventsDemonstrations = 0; c.fatalitiesTotalPoliticalViolence = 0; c.fatalitiesTotalCivilianTargeting = 0;
}
if (month === c.month) {
c.eventsTotal += events;
if (eventType.includes('political_violence')) { c.eventsPoliticalViolence += events; c.fatalitiesTotalPoliticalViolence += fatalities; }
if (eventType.includes('civilian_targeting')) { c.eventsCivilianTargeting += events; c.fatalitiesTotalCivilianTargeting += fatalities; }
if (eventType.includes('demonstration')) { c.eventsDemonstrations += events; }
}
}
const result = {
success: true,
count: Object.keys(byCountry).length,
countries: Object.values(byCountry),
cached_at: new Date().toISOString(),
};
fallbackCache = { data: result, timestamp: now };
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/hapi', 'MISS');
return Response.json(result, {
status: 200,
headers: {
...cors,
'Cache-Control': RESPONSE_CACHE_CONTROL,
'X-Cache': 'MISS',
},
});
} catch (error) {
if (isValidResult(fallbackCache.data)) {
recordCacheTelemetry('/api/hapi', 'STALE');
return Response.json(fallbackCache.data, {
status: 200,
headers: {
...cors,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'X-Cache': 'STALE',
},
});
}
recordCacheTelemetry('/api/hapi', 'ERROR');
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, countries: [] }, {
status: 500,
headers: { ...cors },
});
}
}
-284
View File
@@ -1,284 +0,0 @@
export const config = { runtime: 'edge' };
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
const CACHE_TTL = 300;
let cachedResponse = null;
let cacheTimestamp = 0;
async function fetchJSON(url, timeout = 8000) {
const controller = new AbortController();
const id = setTimeout(() => controller.abort(), timeout);
try {
const res = await fetch(url, { signal: controller.signal });
if (!res.ok) throw new Error(`HTTP ${res.status}`);
return await res.json();
} finally {
clearTimeout(id);
}
}
function rateOfChange(prices, days) {
if (!prices || prices.length < days + 1) return null;
const recent = prices[prices.length - 1];
const past = prices[prices.length - 1 - days];
if (!past || past === 0) return null;
return ((recent - past) / past) * 100;
}
function sma(prices, period) {
if (!prices || prices.length < period) return null;
const slice = prices.slice(-period);
return slice.reduce((a, b) => a + b, 0) / period;
}
function extractClosePrices(chart) {
try {
const result = chart?.chart?.result?.[0];
return result?.indicators?.quote?.[0]?.close?.filter(p => p != null) || [];
} catch {
return [];
}
}
function extractVolumes(chart) {
try {
const result = chart?.chart?.result?.[0];
return result?.indicators?.quote?.[0]?.volume?.filter(v => v != null) || [];
} catch {
return [];
}
}
function extractAlignedPriceVolume(chart) {
try {
const result = chart?.chart?.result?.[0];
const closes = result?.indicators?.quote?.[0]?.close || [];
const volumes = result?.indicators?.quote?.[0]?.volume || [];
const pairs = [];
for (let i = 0; i < closes.length; i++) {
if (closes[i] != null && volumes[i] != null) {
pairs.push({ price: closes[i], volume: volumes[i] });
}
}
return pairs;
} catch {
return [];
}
}
function buildFallbackResult() {
return {
timestamp: new Date().toISOString(),
verdict: 'UNKNOWN',
bullishCount: 0,
totalCount: 0,
signals: {
liquidity: { status: 'UNKNOWN', value: null, sparkline: [] },
flowStructure: { status: 'UNKNOWN', btcReturn5: null, qqqReturn5: null },
macroRegime: { status: 'UNKNOWN', qqqRoc20: null, xlpRoc20: null },
technicalTrend: {
status: 'UNKNOWN',
btcPrice: null,
sma50: null,
sma200: null,
vwap30d: null,
mayerMultiple: null,
sparkline: [],
},
hashRate: { status: 'UNKNOWN', change30d: null },
miningCost: { status: 'UNKNOWN' },
fearGreed: { status: 'UNKNOWN', value: null, history: [] },
},
meta: { qqqSparkline: [] },
unavailable: true,
};
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: cors });
}
return new Response(null, { status: 204, headers: cors });
}
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
}
const now = Date.now();
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
return new Response(JSON.stringify(cachedResponse), {
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=600` },
});
}
try {
const yahooBase = 'https://query1.finance.yahoo.com/v8/finance/chart';
const [jpyChart, btcChart, qqqChart, xlpChart, fearGreed, mempoolHash] = await Promise.allSettled([
fetchJSON(`${yahooBase}/JPY=X?range=1y&interval=1d`),
fetchJSON(`${yahooBase}/BTC-USD?range=1y&interval=1d`),
fetchJSON(`${yahooBase}/QQQ?range=1y&interval=1d`),
fetchJSON(`${yahooBase}/XLP?range=1y&interval=1d`),
fetchJSON('https://api.alternative.me/fng/?limit=30&format=json'),
fetchJSON('https://mempool.space/api/v1/mining/hashrate/1m'),
]);
const jpyPrices = jpyChart.status === 'fulfilled' ? extractClosePrices(jpyChart.value) : [];
const btcPrices = btcChart.status === 'fulfilled' ? extractClosePrices(btcChart.value) : [];
const btcVolumes = btcChart.status === 'fulfilled' ? extractVolumes(btcChart.value) : [];
const btcAligned = btcChart.status === 'fulfilled' ? extractAlignedPriceVolume(btcChart.value) : [];
const qqqPrices = qqqChart.status === 'fulfilled' ? extractClosePrices(qqqChart.value) : [];
const xlpPrices = xlpChart.status === 'fulfilled' ? extractClosePrices(xlpChart.value) : [];
// 1. Liquidity Signal (JPY 30d ROC)
const jpyRoc30 = rateOfChange(jpyPrices, 30);
const liquidityStatus = jpyRoc30 !== null
? (jpyRoc30 < -2 ? 'SQUEEZE' : 'NORMAL')
: 'UNKNOWN';
// 2. Flow Structure (BTC vs QQQ 5d return)
const btcReturn5 = rateOfChange(btcPrices, 5);
const qqqReturn5 = rateOfChange(qqqPrices, 5);
let flowStatus = 'UNKNOWN';
if (btcReturn5 !== null && qqqReturn5 !== null) {
const gap = btcReturn5 - qqqReturn5;
flowStatus = Math.abs(gap) > 5 ? 'PASSIVE GAP' : 'ALIGNED';
}
// 3. Macro Regime (QQQ/XLP 20d ROC)
const qqqRoc20 = rateOfChange(qqqPrices, 20);
const xlpRoc20 = rateOfChange(xlpPrices, 20);
let regimeStatus = 'UNKNOWN';
if (qqqRoc20 !== null && xlpRoc20 !== null) {
regimeStatus = qqqRoc20 > xlpRoc20 ? 'RISK-ON' : 'DEFENSIVE';
}
// 4. Technical Trend (BTC vs SMA50 + VWAP)
const btcSma50 = sma(btcPrices, 50);
const btcSma200 = sma(btcPrices, 200);
const btcCurrent = btcPrices.length > 0 ? btcPrices[btcPrices.length - 1] : null;
// Compute VWAP from aligned price/volume pairs (30d)
let btcVwap = null;
if (btcAligned.length >= 30) {
const last30 = btcAligned.slice(-30);
let sumPV = 0, sumV = 0;
for (const { price, volume } of last30) {
sumPV += price * volume;
sumV += volume;
}
if (sumV > 0) btcVwap = +(sumPV / sumV).toFixed(0);
}
let trendStatus = 'UNKNOWN';
let mayerMultiple = null;
if (btcCurrent && btcSma50) {
const aboveSma = btcCurrent > btcSma50 * 1.02;
const belowSma = btcCurrent < btcSma50 * 0.98;
const aboveVwap = btcVwap ? btcCurrent > btcVwap : null;
if (aboveSma && aboveVwap !== false) trendStatus = 'BULLISH';
else if (belowSma && aboveVwap !== true) trendStatus = 'BEARISH';
else trendStatus = 'NEUTRAL';
}
if (btcCurrent && btcSma200) {
mayerMultiple = +(btcCurrent / btcSma200).toFixed(2);
}
// 5. Hash Rate
let hashStatus = 'UNKNOWN';
let hashChange = null;
if (mempoolHash.status === 'fulfilled') {
const hr = mempoolHash.value?.hashrates || mempoolHash.value;
if (Array.isArray(hr) && hr.length >= 2) {
const recent = hr[hr.length - 1]?.avgHashrate || hr[hr.length - 1];
const older = hr[0]?.avgHashrate || hr[0];
if (recent && older && older > 0) {
hashChange = +((recent - older) / older * 100).toFixed(1);
hashStatus = hashChange > 3 ? 'GROWING' : hashChange < -3 ? 'DECLINING' : 'STABLE';
}
}
}
// 6. Mining Cost (hashrate-based model)
let miningStatus = 'UNKNOWN';
if (btcCurrent && hashChange !== null) {
miningStatus = btcCurrent > 60000 ? 'PROFITABLE' : btcCurrent > 40000 ? 'TIGHT' : 'SQUEEZE';
}
// 7. Fear & Greed
let fgValue = null;
let fgLabel = 'UNKNOWN';
let fgHistory = [];
if (fearGreed.status === 'fulfilled' && fearGreed.value?.data) {
const data = fearGreed.value.data;
const parsed = parseInt(data[0]?.value, 10);
fgValue = Number.isFinite(parsed) ? parsed : null;
fgLabel = data[0]?.value_classification || 'UNKNOWN';
fgHistory = data.slice(0, 30).map(d => ({
value: parseInt(d.value, 10),
date: new Date(parseInt(d.timestamp, 10) * 1000).toISOString().slice(0, 10),
})).reverse();
}
// Sparkline data
const btcSparkline = btcPrices.slice(-30);
const qqqSparkline = qqqPrices.slice(-30);
const jpySparkline = jpyPrices.slice(-30);
// Overall Verdict
let bullishCount = 0;
let totalCount = 0;
const signals = [
{ name: 'Liquidity', status: liquidityStatus, bullish: liquidityStatus === 'NORMAL' },
{ name: 'Flow Structure', status: flowStatus, bullish: flowStatus === 'ALIGNED' },
{ name: 'Macro Regime', status: regimeStatus, bullish: regimeStatus === 'RISK-ON' },
{ name: 'Technical Trend', status: trendStatus, bullish: trendStatus === 'BULLISH' },
{ name: 'Hash Rate', status: hashStatus, bullish: hashStatus === 'GROWING' },
{ name: 'Mining Cost', status: miningStatus, bullish: miningStatus === 'PROFITABLE' },
{ name: 'Fear & Greed', status: fgLabel, bullish: fgValue !== null && fgValue > 50 },
];
for (const s of signals) {
if (s.status !== 'UNKNOWN') {
totalCount++;
if (s.bullish) bullishCount++;
}
}
const verdict = totalCount === 0 ? 'UNKNOWN' : (bullishCount / totalCount >= 0.57 ? 'BUY' : 'CASH');
const result = {
timestamp: new Date().toISOString(),
verdict,
bullishCount,
totalCount,
signals: {
liquidity: { status: liquidityStatus, value: jpyRoc30 !== null ? +jpyRoc30.toFixed(2) : null, sparkline: jpySparkline },
flowStructure: { status: flowStatus, btcReturn5: btcReturn5 !== null ? +btcReturn5.toFixed(2) : null, qqqReturn5: qqqReturn5 !== null ? +qqqReturn5.toFixed(2) : null },
macroRegime: { status: regimeStatus, qqqRoc20: qqqRoc20 !== null ? +qqqRoc20.toFixed(2) : null, xlpRoc20: xlpRoc20 !== null ? +xlpRoc20.toFixed(2) : null },
technicalTrend: { status: trendStatus, btcPrice: btcCurrent, sma50: btcSma50 ? +btcSma50.toFixed(0) : null, sma200: btcSma200 ? +btcSma200.toFixed(0) : null, vwap30d: btcVwap, mayerMultiple, sparkline: btcSparkline },
hashRate: { status: hashStatus, change30d: hashChange },
miningCost: { status: miningStatus },
fearGreed: { status: fgLabel, value: fgValue, history: fgHistory },
},
meta: { qqqSparkline },
};
cachedResponse = result;
cacheTimestamp = now;
return new Response(JSON.stringify(result), {
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=600` },
});
} catch (err) {
const fallback = cachedResponse || buildFallbackResult();
cachedResponse = fallback;
cacheTimestamp = now;
return new Response(JSON.stringify(fallback), {
status: 200,
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
});
}
}
-25
View File
@@ -1,25 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
try {
const response = await fetch(
'https://msi.nga.mil/api/publications/broadcast-warn?output=json&status=A'
);
const data = await response.text();
return new Response(data, {
status: response.status,
headers: { 'Content-Type': 'application/json', ...cors, 'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60' },
});
} catch (error) {
return new Response(JSON.stringify({ error: error.message }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
+7 -1
View File
@@ -1,3 +1,4 @@
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
/**
* Dynamic OG Image Generator for Story Sharing
* Returns an SVG image (1200x630) rich intelligence card for social previews.
@@ -24,12 +25,17 @@ const LEVEL_LABELS = {
low: 'LOW RISK',
};
function normalizeLevel(rawLevel) {
const level = String(rawLevel || '').toLowerCase();
return Object.prototype.hasOwnProperty.call(LEVEL_COLORS, level) ? level : 'normal';
}
export default function handler(req, res) {
const url = new URL(req.url, `https://${req.headers.host}`);
const countryCode = (url.searchParams.get('c') || '').toUpperCase();
const type = url.searchParams.get('t') || 'ciianalysis';
const score = url.searchParams.get('s');
const level = url.searchParams.get('l') || 'normal';
const level = normalizeLevel(url.searchParams.get('l'));
const countryName = COUNTRY_NAMES[countryCode] || countryCode || 'Global';
const levelColor = LEVEL_COLORS[level] || '#eab308';
+48
View File
@@ -0,0 +1,48 @@
import { strict as assert } from 'node:assert';
import test from 'node:test';
import handler from './og-story.js';
function renderOgStory(query = '') {
const req = {
url: `https://worldmonitor.app/api/og-story${query ? `?${query}` : ''}`,
headers: { host: 'worldmonitor.app' },
};
let statusCode = 0;
let body = '';
const headers = {};
const res = {
setHeader(name, value) {
headers[String(name).toLowerCase()] = String(value);
},
status(code) {
statusCode = code;
return this;
},
send(payload) {
body = String(payload);
},
};
handler(req, res);
return { statusCode, body, headers };
}
test('normalizes unsupported level values to prevent SVG script injection', () => {
const injectedLevel = encodeURIComponent('</text><script>alert(1)</script><text>');
const response = renderOgStory(`c=US&s=50&l=${injectedLevel}`);
assert.equal(response.statusCode, 200);
assert.equal(/<script/i.test(response.body), false);
assert.match(response.body, />NORMAL<\/text>/);
});
test('uses a known level when it is allowlisted', () => {
const response = renderOgStory('c=US&s=88&l=critical');
assert.equal(response.statusCode, 200);
assert.match(response.body, />CRITICAL<\/text>/);
assert.match(response.body, /#ef4444/);
});
-294
View File
@@ -1,294 +0,0 @@
/**
* OpenRouter API Summarization Endpoint with Redis Caching
* Fallback when Groq is rate-limited
* Uses OpenRouter auto-routed free model
* Free tier: 50 requests/day (20/min)
* Server-side Redis cache for cross-user deduplication
*/
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const OPENROUTER_API_URL = 'https://openrouter.ai/api/v1/chat/completions';
const MODEL = 'openrouter/free';
const CACHE_TTL_SECONDS = 86400; // 24 hours
const CACHE_VERSION = 'v3';
function getCacheKey(headlines, mode, geoContext = '', variant = 'full', lang = 'en') {
const sorted = headlines.slice(0, 8).sort().join('|');
const geoHash = geoContext ? ':g' + hashString(geoContext).slice(0, 6) : '';
const hash = hashString(`${mode}:${sorted}`);
const normalizedVariant = typeof variant === 'string' && variant ? variant.toLowerCase() : 'full';
const normalizedLang = typeof lang === 'string' && lang ? lang.toLowerCase() : 'en';
if (mode === 'translate') {
const targetLang = normalizedVariant || normalizedLang;
return `summary:${CACHE_VERSION}:${mode}:${targetLang}:${hash}${geoHash}`;
}
return `summary:${CACHE_VERSION}:${mode}:${normalizedVariant}:${normalizedLang}:${hash}${geoHash}`;
}
// Deduplicate similar headlines (same story from different sources)
function deduplicateHeadlines(headlines) {
const seen = new Set();
const unique = [];
for (const headline of headlines) {
// Normalize: lowercase, remove punctuation, collapse whitespace
const normalized = headline.toLowerCase()
.replace(/[^\w\s]/g, '')
.replace(/\s+/g, ' ')
.trim();
// Extract key words (4+ chars) for similarity check
const words = new Set(normalized.split(' ').filter(w => w.length >= 4));
// Check if this headline is too similar to any we've seen
let isDuplicate = false;
for (const seenWords of seen) {
const intersection = [...words].filter(w => seenWords.has(w));
const similarity = intersection.length / Math.min(words.size, seenWords.size);
if (similarity > 0.6) {
isDuplicate = true;
break;
}
}
if (!isDuplicate) {
seen.add(words);
unique.push(headline);
}
}
return unique;
}
export default async function handler(request) {
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
if (request.method !== 'POST') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
const apiKey = process.env.OPENROUTER_API_KEY;
if (!apiKey) {
return new Response(JSON.stringify({ summary: null, fallback: true, skipped: true, reason: 'OPENROUTER_API_KEY not configured' }), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
if (contentLength > 51200) {
return new Response(JSON.stringify({ error: 'Payload too large' }), {
status: 413,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
try {
const { headlines, mode = 'brief', geoContext = '', variant = 'full', lang = 'en' } = await request.json();
if (!headlines || !Array.isArray(headlines) || headlines.length === 0) {
return new Response(JSON.stringify({ error: 'Headlines array required' }), {
status: 400,
headers: { 'Content-Type': 'application/json' },
});
}
// Check cache first (shared with Groq endpoint)
const cacheKey = getCacheKey(headlines, mode, geoContext, variant, lang);
const cached = await getCachedJson(cacheKey);
if (cached && typeof cached === 'object' && cached.summary) {
console.log('[OpenRouter] Cache hit:', cacheKey);
return new Response(JSON.stringify({
summary: cached.summary,
model: cached.model || MODEL,
provider: 'cache',
cached: true,
}), {
status: 200,
headers: { 'Content-Type': 'application/json' },
});
}
// Deduplicate similar headlines (same story from different sources)
const uniqueHeadlines = deduplicateHeadlines(headlines.slice(0, 8));
const headlineText = uniqueHeadlines.map((h, i) => `${i + 1}. ${h}`).join('\n');
let systemPrompt, userPrompt;
// Include intelligence synthesis context in prompt if available
const intelSection = geoContext ? `\n\n${geoContext}` : '';
// Current date context for LLM (models may have outdated knowledge)
const isTechVariant = variant === 'tech';
const dateContext = `Current date: ${new Date().toISOString().split('T')[0]}.${isTechVariant ? '' : ' Donald Trump is the current US President (second term, inaugurated Jan 2025).'}`;
// Language instruction
const langInstruction = lang && lang !== 'en' ? `\nIMPORTANT: Output the summary in ${lang.toUpperCase()} language.` : '';
if (mode === 'brief') {
if (isTechVariant) {
// Tech variant: focus on startups, AI, funding, product launches
systemPrompt = `${dateContext}
Summarize the key tech/startup development in 2-3 sentences.
Rules:
- Focus ONLY on technology, startups, AI, funding, product launches, or developer news
- IGNORE political news, trade policy, tariffs, government actions unless directly about tech regulation
- Lead with the company/product/technology name
- Start directly: "OpenAI announced...", "A new $50M Series B...", "GitHub released..."
- No bullet points, no meta-commentary${langInstruction}`;
} else {
// Full variant: geopolitical focus
systemPrompt = `${dateContext}
Summarize the key development in 2-3 sentences.
Rules:
- Lead with WHAT happened and WHERE - be specific
- NEVER start with "Breaking news", "Good evening", "Tonight", or TV-style openings
- Start directly with the subject: "Iran's regime...", "The US Treasury...", "Protests in..."
- CRITICAL FOCAL POINTS are the main actors - mention them by name
- If focal points show news + signals convergence, that's the lead
- No bullet points, no meta-commentary${langInstruction}`;
}
userPrompt = `Summarize the top story:\n${headlineText}${intelSection}`;
} else if (mode === 'analysis') {
if (isTechVariant) {
systemPrompt = `${dateContext}
Analyze the tech/startup trend in 2-3 sentences.
Rules:
- Focus ONLY on technology implications: funding trends, AI developments, market shifts, product strategy
- IGNORE political implications, trade wars, government unless directly about tech policy
- Lead with the insight for tech industry
- Connect to startup ecosystem, VC trends, or technical implications`;
} else {
systemPrompt = `${dateContext}
Provide analysis in 2-3 sentences. Be direct and specific.
Rules:
- Lead with the insight - what's significant and why
- NEVER start with "Breaking news", "Tonight", "The key/dominant narrative is"
- Start with substance: "Iran faces...", "The escalation in...", "Multiple signals suggest..."
- CRITICAL FOCAL POINTS are your main actors - explain WHY they matter
- If focal points show news-signal correlation, flag as escalation
- Connect dots, be specific about implications`;
}
userPrompt = isTechVariant
? `What's the key tech trend or development?\n${headlineText}${intelSection}`
: `What's the key pattern or risk?\n${headlineText}${intelSection}`;
} else if (mode === 'translate') {
const targetLang = variant; // In translate mode, variant param holds the target language code
systemPrompt = `You are a professional news translator. Translate the following news headlines/summaries into ${targetLang}.
Rules:
- Maintain the original tone and journalistic style.
- Do NOT add any conversational filler.
- Output ONLY the translated text.`;
userPrompt = `Translate to ${targetLang}:\n${headlines[0]}`;
} else {
systemPrompt = isTechVariant
? `${dateContext}\n\nSynthesize tech news in 2 sentences. Focus on startups, AI, funding, products. Ignore politics unless directly about tech regulation.${langInstruction}`
: `${dateContext}\n\nSynthesize in 2 sentences max. Lead with substance. NEVER start with "Breaking news" or "Tonight" - just state the insight directly. CRITICAL focal points with news-signal convergence are significant.${langInstruction}`;
userPrompt = `Key takeaway:\n${headlineText}${intelSection}`;
}
const response = await fetch(OPENROUTER_API_URL, {
method: 'POST',
headers: {
'Authorization': `Bearer ${apiKey}`,
'Content-Type': 'application/json',
'HTTP-Referer': 'https://worldmonitor.app',
'X-Title': 'WorldMonitor',
},
body: JSON.stringify({
model: MODEL,
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userPrompt },
],
temperature: 0.3,
max_tokens: 150,
top_p: 0.9,
}),
});
if (!response.ok) {
const errorText = await response.text();
console.error('[OpenRouter] API error:', response.status, errorText);
// Return fallback signal for rate limiting
if (response.status === 429) {
return new Response(JSON.stringify({ error: 'Rate limited', fallback: true }), {
status: 429,
headers: { 'Content-Type': 'application/json' },
});
}
return new Response(JSON.stringify({ error: 'OpenRouter API error', fallback: true }), {
status: response.status,
headers: { 'Content-Type': 'application/json' },
});
}
const data = await response.json();
const summary = data.choices?.[0]?.message?.content?.trim();
if (!summary) {
return new Response(JSON.stringify({ error: 'Empty response', fallback: true }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
// Store in cache (shared with Groq endpoint)
await setCachedJson(cacheKey, {
summary,
model: MODEL,
timestamp: Date.now(),
}, CACHE_TTL_SECONDS);
return new Response(JSON.stringify({
summary,
model: MODEL,
provider: 'openrouter',
cached: false,
tokens: data.usage?.total_tokens || 0,
}), {
status: 200,
headers: {
...corsHeaders,
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
},
});
} catch (error) {
console.error('[OpenRouter] Error:', error);
return new Response(JSON.stringify({ error: error.message, fallback: true }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
-73
View File
@@ -1,73 +0,0 @@
// OpenSky Network API proxy - v3
// Note: OpenSky seems to block some cloud provider IPs
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const url = new URL(req.url);
// Build OpenSky API URL with bounding box params
const params = new URLSearchParams();
['lamin', 'lomin', 'lamax', 'lomax'].forEach(key => {
const val = url.searchParams.get(key);
if (val) params.set(key, val);
});
const openskyUrl = `https://opensky-network.org/api/states/all${params.toString() ? '?' + params.toString() : ''}`;
try {
// Try fetching with different headers to avoid blocks
const response = await fetch(openskyUrl, {
headers: {
'Accept': 'application/json',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
'Accept-Language': 'en-US,en;q=0.9',
'Accept-Encoding': 'gzip, deflate, br',
'Connection': 'keep-alive',
},
});
if (response.status === 429) {
return Response.json({ error: 'Rate limited', time: Date.now(), states: null }, {
status: 429,
headers: cors,
});
}
// Check if response is OK
if (!response.ok) {
const text = await response.text();
return Response.json({
error: `OpenSky HTTP ${response.status}: ${text.substring(0, 200)}`,
time: Date.now(),
states: null
}, {
status: response.status,
headers: cors,
});
}
const data = await response.json();
return Response.json(data, {
status: response.status,
headers: {
...cors,
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
},
});
} catch (error) {
return Response.json({
error: `Fetch failed: ${error.name} - ${error.message}`,
time: Date.now(),
states: null
}, {
status: 500,
headers: cors,
});
}
}
-36
View File
@@ -1,36 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from '../_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(request) {
const cors = getCorsHeaders(request);
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
try {
const response = await fetch('https://www.pizzint.watch/api/dashboard-data', {
headers: {
'Accept': 'application/json',
'User-Agent': 'WorldMonitor/1.0',
},
});
if (!response.ok) {
throw new Error(`Upstream returned ${response.status}`);
}
const data = await response.text();
return new Response(data, {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
},
});
} catch (error) {
return new Response(JSON.stringify({ error: 'Failed to fetch PizzINT data', details: error.message }), {
status: 502,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
}
-46
View File
@@ -1,46 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const url = new URL(req.url);
const pairs = url.searchParams.get('pairs') || 'usa_russia,russia_ukraine,usa_china,china_taiwan,usa_iran,usa_venezuela';
const dateStart = url.searchParams.get('dateStart');
const dateEnd = url.searchParams.get('dateEnd');
const method = url.searchParams.get('method') || 'gpr';
let targetUrl = `https://www.pizzint.watch/api/gdelt/batch?pairs=${encodeURIComponent(pairs)}&method=${method}`;
if (dateStart) targetUrl += `&dateStart=${dateStart}`;
if (dateEnd) targetUrl += `&dateEnd=${dateEnd}`;
try {
const response = await fetch(targetUrl, {
headers: {
'Accept': 'application/json',
'User-Agent': 'WorldMonitor/1.0',
},
});
if (!response.ok) {
throw new Error(`Upstream returned ${response.status}`);
}
const data = await response.text();
return new Response(data, {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
},
});
} catch (error) {
return new Response(JSON.stringify({ error: 'Failed to fetch GDELT data', details: error.message }), {
status: 502,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
}
-108
View File
@@ -1,108 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
const GAMMA_BASE = 'https://gamma-api.polymarket.com';
const ALLOWED_ORDER = ['volume', 'liquidity', 'startDate', 'endDate', 'spread'];
const MAX_LIMIT = 100;
const MIN_LIMIT = 1;
function validateBoolean(val, defaultVal) {
if (val === 'true' || val === 'false') return val;
return defaultVal;
}
function validateLimit(val) {
const num = parseInt(val, 10);
if (isNaN(num)) return 50;
return Math.max(MIN_LIMIT, Math.min(MAX_LIMIT, num));
}
function validateOrder(val) {
return ALLOWED_ORDER.includes(val) ? val : 'volume';
}
function sanitizeTagSlug(val) {
if (!val) return null;
return val.replace(/[^a-z0-9-]/gi, '').slice(0, 100) || null;
}
async function tryFetch(url, timeoutMs = 8000) {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), timeoutMs);
try {
const response = await fetch(url, {
headers: { 'Accept': 'application/json' },
signal: controller.signal,
});
clearTimeout(timer);
if (!response.ok) {
throw new Error(`HTTP ${response.status}`);
}
return await response.text();
} catch (err) {
clearTimeout(timer);
throw err;
}
}
function buildUrl(base, endpoint, params) {
if (endpoint === 'events') {
return `${base}/events?${params}`;
}
return `${base}/markets?${params}`;
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const url = new URL(req.url);
const endpoint = url.searchParams.get('endpoint') || 'markets';
const closed = validateBoolean(url.searchParams.get('closed'), 'false');
const order = validateOrder(url.searchParams.get('order'));
const ascending = validateBoolean(url.searchParams.get('ascending'), 'false');
const limit = validateLimit(url.searchParams.get('limit'));
const params = new URLSearchParams({
closed,
order,
ascending,
limit: String(limit),
});
if (endpoint === 'events') {
const tag = sanitizeTagSlug(url.searchParams.get('tag'));
if (tag) params.set('tag_slug', tag);
}
// Gamma API is behind Cloudflare which blocks server-side TLS connections
// (JA3 fingerprint detection). Only browser-originated requests succeed.
// We still try in case Cloudflare policy changes, but gracefully return empty on failure.
try {
const data = await tryFetch(buildUrl(GAMMA_BASE, endpoint, params));
return new Response(data, {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=120, s-maxage=120, stale-while-revalidate=60',
'X-Polymarket-Source': 'gamma',
},
});
} catch (err) {
// Expected: Cloudflare blocks non-browser TLS connections
return new Response(JSON.stringify([]), {
status: 200,
headers: {
'Content-Type': 'application/json',
...cors,
'X-Polymarket-Error': err.message,
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
},
});
}
}
+97
View File
@@ -0,0 +1,97 @@
export const config = { runtime: 'edge' };
import { ConvexHttpClient } from 'convex/browser';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
const EMAIL_RE = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
const MAX_EMAIL_LENGTH = 320;
const rateLimitMap = new Map();
const RATE_LIMIT = 5;
const RATE_WINDOW_MS = 60 * 60 * 1000;
function isRateLimited(ip) {
const now = Date.now();
const entry = rateLimitMap.get(ip);
if (!entry || now - entry.windowStart > RATE_WINDOW_MS) {
rateLimitMap.set(ip, { windowStart: now, count: 1 });
return false;
}
entry.count += 1;
return entry.count > RATE_LIMIT;
}
export default async function handler(req) {
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
const cors = getCorsHeaders(req, 'POST, OPTIONS');
if (req.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: cors });
}
if (req.method !== 'POST') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
const ip = req.headers.get('x-forwarded-for')?.split(',')[0]?.trim() || 'unknown';
if (isRateLimited(ip)) {
return new Response(JSON.stringify({ error: 'Too many requests' }), {
status: 429,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
let body;
try {
body = await req.json();
} catch {
return new Response(JSON.stringify({ error: 'Invalid JSON' }), {
status: 400,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
const { email, source, appVersion } = body;
if (!email || typeof email !== 'string' || email.length > MAX_EMAIL_LENGTH || !EMAIL_RE.test(email)) {
return new Response(JSON.stringify({ error: 'Invalid email address' }), {
status: 400,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
const convexUrl = process.env.CONVEX_URL;
if (!convexUrl) {
return new Response(JSON.stringify({ error: 'Registration service unavailable' }), {
status: 503,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
try {
const client = new ConvexHttpClient(convexUrl);
const result = await client.mutation('registerInterest:register', {
email,
source: source || 'unknown',
appVersion: appVersion || 'unknown',
});
return new Response(JSON.stringify(result), {
status: 200,
headers: { 'Content-Type': 'application/json', ...cors },
});
} catch (err) {
console.error('[register-interest] Convex error:', err);
return new Response(JSON.stringify({ error: 'Registration failed' }), {
status: 500,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
}
-355
View File
@@ -1,355 +0,0 @@
/**
* Risk Scores API - Cached CII and Strategic Risk computation
* Eliminates 15-minute "learning mode" for users by pre-computing scores
* Uses Upstash Redis for cross-user caching (10-minute TTL)
*/
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const CACHE_TTL_SECONDS = 600; // 10 minutes
const STALE_CACHE_TTL_SECONDS = 3600; // 1 hour - serve stale when API fails
const CACHE_KEY = 'risk:scores:v2';
const STALE_CACHE_KEY = 'risk:scores:stale:v2';
// Tier 1 countries for CII
const TIER1_COUNTRIES = {
US: 'United States', RU: 'Russia', CN: 'China', UA: 'Ukraine', IR: 'Iran',
IL: 'Israel', TW: 'Taiwan', KP: 'North Korea', SA: 'Saudi Arabia', TR: 'Turkey',
PL: 'Poland', DE: 'Germany', FR: 'France', GB: 'United Kingdom', IN: 'India',
PK: 'Pakistan', SY: 'Syria', YE: 'Yemen', MM: 'Myanmar', VE: 'Venezuela',
};
// Baseline geopolitical risk (0-50)
const BASELINE_RISK = {
US: 5, RU: 35, CN: 25, UA: 50, IR: 40, IL: 45, TW: 30, KP: 45,
SA: 20, TR: 25, PL: 10, DE: 5, FR: 10, GB: 5, IN: 20, PK: 35,
SY: 50, YE: 50, MM: 45, VE: 40,
};
// Event significance multipliers
const EVENT_MULTIPLIER = {
US: 0.3, RU: 2.0, CN: 2.5, UA: 0.8, IR: 2.0, IL: 0.7, TW: 1.5, KP: 3.0,
SA: 2.0, TR: 1.2, PL: 0.8, DE: 0.5, FR: 0.6, GB: 0.5, IN: 0.8, PK: 1.5,
SY: 0.7, YE: 0.7, MM: 1.8, VE: 1.8,
};
// Country keywords for matching
const COUNTRY_KEYWORDS = {
US: ['united states', 'usa', 'america', 'washington', 'biden', 'trump', 'pentagon'],
RU: ['russia', 'moscow', 'kremlin', 'putin'],
CN: ['china', 'beijing', 'xi jinping', 'prc'],
UA: ['ukraine', 'kyiv', 'zelensky', 'donbas'],
IR: ['iran', 'tehran', 'khamenei', 'irgc'],
IL: ['israel', 'tel aviv', 'netanyahu', 'idf', 'gaza'],
TW: ['taiwan', 'taipei'],
KP: ['north korea', 'pyongyang', 'kim jong'],
SA: ['saudi arabia', 'riyadh'],
TR: ['turkey', 'ankara', 'erdogan'],
PL: ['poland', 'warsaw'],
DE: ['germany', 'berlin'],
FR: ['france', 'paris', 'macron'],
GB: ['britain', 'uk', 'london'],
IN: ['india', 'delhi', 'modi'],
PK: ['pakistan', 'islamabad'],
SY: ['syria', 'damascus'],
YE: ['yemen', 'sanaa', 'houthi'],
MM: ['myanmar', 'burma'],
VE: ['venezuela', 'caracas', 'maduro'],
};
function normalizeCountryName(text) {
const lower = text.toLowerCase();
for (const [code, keywords] of Object.entries(COUNTRY_KEYWORDS)) {
if (keywords.some(kw => lower.includes(kw))) return code;
}
return null;
}
function getScoreLevel(score) {
if (score >= 70) return 'critical';
if (score >= 55) return 'high';
if (score >= 40) return 'elevated';
if (score >= 25) return 'normal';
return 'low';
}
async function fetchACLEDProtests() {
try {
// Fetch recent protests from ACLED (last 7 days)
const endDate = new Date().toISOString().split('T')[0];
const startDate = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 15000); // 15s timeout
// ACLED API now requires authentication - new endpoint as of Jan 2026
const token = process.env.ACLED_ACCESS_TOKEN;
const headers = { 'Accept': 'application/json' };
if (token) {
headers['Authorization'] = `Bearer ${token}`;
}
// Updated endpoint: acleddata.com/api/ instead of api.acleddata.com
const response = await fetch(
`https://acleddata.com/api/acled/read?_format=json&event_type=Protests&event_type=Riots&event_date=${startDate}|${endDate}&event_date_where=BETWEEN&limit=500`,
{
headers,
signal: controller.signal,
}
);
clearTimeout(timeoutId);
if (!response.ok) {
const text = await response.text().catch(() => '');
console.warn('[RiskScores] ACLED fetch failed:', response.status, text.slice(0, 200));
// Check for auth errors specifically
if (response.status === 401 || response.status === 403) {
throw new Error('ACLED API requires valid authentication token');
}
throw new Error(`ACLED API error: ${response.status}`);
}
const data = await response.json();
// Check for API-level error in response
if (data.message) {
console.warn('[RiskScores] ACLED API returned message:', data.message);
throw new Error(data.message);
}
if (data.error || data.success === false) {
console.warn('[RiskScores] ACLED API returned error:', data.error || 'unknown');
throw new Error(data.error || 'ACLED API error');
}
return data.data || [];
} catch (error) {
console.warn('[RiskScores] ACLED error:', error.message);
throw error; // Re-throw to trigger stale cache fallback
}
}
function computeCIIScores(protests) {
const countryEvents = new Map();
// Count events per country
for (const event of protests) {
const country = event.country;
const code = normalizeCountryName(country);
if (code && TIER1_COUNTRIES[code]) {
const count = countryEvents.get(code) || { protests: 0, riots: 0 };
if (event.event_type === 'Riots') {
count.riots++;
} else {
count.protests++;
}
countryEvents.set(code, count);
}
}
// Compute scores for all Tier 1 countries
const scores = [];
const now = new Date();
for (const [code, name] of Object.entries(TIER1_COUNTRIES)) {
const events = countryEvents.get(code) || { protests: 0, riots: 0 };
const baseline = BASELINE_RISK[code] || 20;
const multiplier = EVENT_MULTIPLIER[code] || 1.0;
// Unrest component: protests + riots (riots weighted 2x)
const unrestRaw = (events.protests + events.riots * 2) * multiplier;
const unrest = Math.min(100, Math.round(unrestRaw * 2));
// Security component: baseline + riot contribution
const security = Math.min(100, baseline + events.riots * multiplier * 5);
// Information component: based on event count (proxy for news coverage)
const totalEvents = events.protests + events.riots;
const information = Math.min(100, totalEvents * multiplier * 3);
// Composite score: weighted average + baseline
const composite = Math.min(100, Math.round(
baseline +
(unrest * 0.4 + security * 0.35 + information * 0.25) * 0.5
));
scores.push({
code,
name,
score: composite,
level: getScoreLevel(composite),
trend: 'stable', // Would need historical data for real trend
change24h: 0,
components: { unrest, security, information },
lastUpdated: now.toISOString(),
});
}
// Sort by score descending
scores.sort((a, b) => b.score - a.score);
return scores;
}
function computeStrategicRisk(ciiScores) {
// Top 5 CII scores weighted average
const top5 = ciiScores.slice(0, 5);
const weights = top5.map((_, i) => 1 - (i * 0.15)); // [1.0, 0.85, 0.70, 0.55, 0.40]
const totalWeight = weights.reduce((sum, w) => sum + w, 0); // 3.5
const weightedSum = top5.reduce((sum, s, i) => sum + s.score * weights[i], 0);
const ciiComponent = weightedSum / totalWeight;
// Overall strategic risk
const overallScore = Math.round(ciiComponent * 0.7 + 15); // 30% baseline
return {
score: Math.min(100, overallScore),
level: getScoreLevel(overallScore),
trend: 'stable',
lastUpdated: new Date().toISOString(),
contributors: top5.map(s => ({
country: s.name,
code: s.code,
score: s.score,
level: s.level,
})),
};
}
export default async function handler(request) {
const corsHeaders = getCorsHeaders(request, 'GET, OPTIONS');
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
if (request.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
}
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
if (!process.env.ACLED_ACCESS_TOKEN) {
const baselineScores = computeCIIScores([]);
const baselineStrategic = computeStrategicRisk(baselineScores);
return new Response(JSON.stringify({
cii: baselineScores,
strategicRisk: baselineStrategic,
protestCount: 0,
computedAt: new Date().toISOString(),
baseline: true,
error: 'ACLED token not configured - showing baseline risk assessments',
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
},
});
}
// Check cache first
const cached = await getCachedJson(CACHE_KEY);
if (cached && typeof cached === 'object') {
console.log('[RiskScores] Cache hit');
return new Response(JSON.stringify({
...cached,
cached: true,
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
},
});
}
try {
// Fetch ACLED protests
console.log('[RiskScores] Computing scores...');
const protests = await fetchACLEDProtests();
// Compute CII scores
const ciiScores = computeCIIScores(protests);
// Compute strategic risk
const strategicRisk = computeStrategicRisk(ciiScores);
const result = {
cii: ciiScores,
strategicRisk,
protestCount: protests.length,
computedAt: new Date().toISOString(),
};
// Cache (both regular and stale backup)
await Promise.all([
setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS),
setCachedJson(STALE_CACHE_KEY, result, STALE_CACHE_TTL_SECONDS),
]);
return new Response(JSON.stringify({
...result,
cached: false,
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
},
});
} catch (error) {
console.error('[RiskScores] Error:', error);
// Try to return stale cached data
const stale = await getCachedJson(STALE_CACHE_KEY);
if (stale && typeof stale === 'object') {
console.log('[RiskScores] Returning stale cache due to error');
return new Response(JSON.stringify({
...stale,
cached: true,
stale: true,
error: 'Using cached data - ACLED temporarily unavailable',
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
},
});
}
// Final fallback: return baseline scores without unrest data
console.log('[RiskScores] Returning baseline scores (no ACLED data)');
const baselineScores = computeCIIScores([]); // Empty protests = baseline only
const baselineStrategic = computeStrategicRisk(baselineScores);
return new Response(JSON.stringify({
cii: baselineScores,
strategicRisk: baselineStrategic,
protestCount: 0,
computedAt: new Date().toISOString(),
baseline: true,
error: 'ACLED unavailable - showing baseline risk assessments',
}), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
},
});
}
}
+18 -2
View File
@@ -1,3 +1,4 @@
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
@@ -63,6 +64,11 @@ const ALLOWED_DOMAINS = [
'www.brookings.edu',
'layoffs.fyi',
'www.defensenews.com',
'www.militarytimes.com',
'taskandpurpose.com',
'news.usni.org',
'www.oryxspioenkop.com',
'www.gov.uk',
'www.foreignaffairs.com',
'www.atlanticcouncil.org',
// Tech variant domains
@@ -171,6 +177,16 @@ const ALLOWED_DOMAINS = [
'www.fao.org',
'worldbank.org',
'www.imf.org',
// Regional locale feeds (tr, pl, ru, th, vi)
'www.hurriyet.com.tr',
'tvn24.pl',
'www.polsatnews.pl',
'www.rp.pl',
'meduza.io',
'novayagazeta.eu',
'www.bangkokpost.com',
'vnexpress.net',
'www.abc.net.au',
// Additional
'news.ycombinator.com',
// Finance variant
@@ -244,7 +260,7 @@ export default async function handler(req) {
status: redirectResponse.status,
headers: {
'Content-Type': 'application/xml',
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'Cache-Control': 'public, max-age=300, s-maxage=600, stale-while-revalidate=300',
...corsHeaders,
},
});
@@ -262,7 +278,7 @@ export default async function handler(req) {
status: response.status,
headers: {
'Content-Type': 'application/xml',
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=300',
...corsHeaders,
},
});
-296
View File
@@ -1,296 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = { runtime: 'edge' };
// Major tech services and their status page endpoints
// Most use Statuspage.io which has a standard /api/v2/status.json endpoint
const SERVICES = [
// Cloud Providers
{ id: 'aws', name: 'AWS', statusPage: 'https://health.aws.amazon.com/health/status', customParser: 'aws', category: 'cloud' },
{ id: 'azure', name: 'Azure', statusPage: 'https://azure.status.microsoft/en-us/status/feed/', customParser: 'rss', category: 'cloud' },
{ id: 'gcp', name: 'Google Cloud', statusPage: 'https://status.cloud.google.com/incidents.json', customParser: 'gcp', category: 'cloud' },
{ id: 'cloudflare', name: 'Cloudflare', statusPage: 'https://www.cloudflarestatus.com/api/v2/status.json', category: 'cloud' },
{ id: 'vercel', name: 'Vercel', statusPage: 'https://www.vercel-status.com/api/v2/status.json', category: 'cloud' },
{ id: 'netlify', name: 'Netlify', statusPage: 'https://www.netlifystatus.com/api/v2/status.json', category: 'cloud' },
{ id: 'digitalocean', name: 'DigitalOcean', statusPage: 'https://status.digitalocean.com/api/v2/status.json', category: 'cloud' },
{ id: 'render', name: 'Render', statusPage: 'https://status.render.com/api/v2/status.json', category: 'cloud' },
{ id: 'railway', name: 'Railway', statusPage: 'https://railway.instatus.com/summary.json', customParser: 'instatus', category: 'cloud' },
// Developer Tools
{ id: 'github', name: 'GitHub', statusPage: 'https://www.githubstatus.com/api/v2/status.json', category: 'dev' },
{ id: 'gitlab', name: 'GitLab', statusPage: 'https://status.gitlab.com/1.0/status/5b36dc6502d06804c08349f7', customParser: 'statusio', category: 'dev' },
{ id: 'npm', name: 'npm', statusPage: 'https://status.npmjs.org/api/v2/status.json', category: 'dev' },
{ id: 'docker', name: 'Docker Hub', statusPage: 'https://www.dockerstatus.com/1.0/status/533c6539221ae15e3f000031', customParser: 'statusio', category: 'dev' },
{ id: 'bitbucket', name: 'Bitbucket', statusPage: 'https://bitbucket.status.atlassian.com/api/v2/status.json', category: 'dev' },
{ id: 'circleci', name: 'CircleCI', statusPage: 'https://status.circleci.com/api/v2/status.json', category: 'dev' },
{ id: 'jira', name: 'Jira', statusPage: 'https://jira-software.status.atlassian.com/api/v2/status.json', category: 'dev' },
{ id: 'confluence', name: 'Confluence', statusPage: 'https://confluence.status.atlassian.com/api/v2/status.json', category: 'dev' },
{ id: 'linear', name: 'Linear', statusPage: 'https://linearstatus.com/api/v2/status.json', customParser: 'incidentio', category: 'dev' },
// Communication
{ id: 'slack', name: 'Slack', statusPage: 'https://slack-status.com/api/v2.0.0/current', customParser: 'slack', category: 'comm' },
{ id: 'discord', name: 'Discord', statusPage: 'https://discordstatus.com/api/v2/status.json', category: 'comm' },
{ id: 'zoom', name: 'Zoom', statusPage: 'https://www.zoomstatus.com/api/v2/status.json', category: 'comm' },
{ id: 'notion', name: 'Notion', statusPage: 'https://www.notion-status.com/api/v2/status.json', category: 'comm' },
// AI Services (incident.io powered)
{ id: 'openai', name: 'OpenAI', statusPage: 'https://status.openai.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
{ id: 'anthropic', name: 'Anthropic', statusPage: 'https://status.claude.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
{ id: 'replicate', name: 'Replicate', statusPage: 'https://www.replicatestatus.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
// SaaS
{ id: 'stripe', name: 'Stripe', statusPage: 'https://status.stripe.com/current', customParser: 'stripe', category: 'saas' },
{ id: 'twilio', name: 'Twilio', statusPage: 'https://status.twilio.com/api/v2/status.json', category: 'saas' },
{ id: 'datadog', name: 'Datadog', statusPage: 'https://status.datadoghq.com/api/v2/status.json', category: 'saas' },
{ id: 'sentry', name: 'Sentry', statusPage: 'https://status.sentry.io/api/v2/status.json', category: 'saas' },
{ id: 'supabase', name: 'Supabase', statusPage: 'https://status.supabase.com/api/v2/status.json', category: 'saas' },
];
// Statuspage.io API returns status like: none, minor, major, critical
function normalizeStatus(indicator) {
if (!indicator) return 'unknown';
const val = indicator.toLowerCase();
// Check for operational indicators
if (val === 'none' || val === 'operational' || val.includes('all systems operational')) {
return 'operational';
}
// Check for degraded indicators
if (val === 'minor' || val === 'degraded_performance' || val === 'partial_outage' || val.includes('degraded')) {
return 'degraded';
}
// Check for outage indicators
if (val === 'major' || val === 'major_outage' || val === 'critical' || val.includes('outage')) {
return 'outage';
}
return 'unknown';
}
async function checkStatusPage(service) {
if (!service.statusPage) {
return { ...service, status: 'unknown', description: 'No API available' };
}
try {
// Use browser-like headers to avoid being blocked
const headers = {
'Accept': service.customParser === 'rss' ? 'application/xml, text/xml' : 'application/json, text/plain, */*',
'Accept-Language': 'en-US,en;q=0.9',
'Cache-Control': 'no-cache',
};
// Don't send User-Agent for incident.io - they may block bots
if (service.customParser !== 'incidentio') {
headers['User-Agent'] = 'Mozilla/5.0 (compatible; WorldMonitor/1.0)';
}
const response = await fetch(service.statusPage, {
headers,
signal: AbortSignal.timeout(10000),
});
if (!response.ok) {
return { ...service, status: 'unknown', description: `HTTP ${response.status}` };
}
// Handle custom parsers
if (service.customParser === 'gcp') {
const data = await response.json();
// GCP incidents.json returns array of incidents
const activeIncidents = Array.isArray(data) ? data.filter(i =>
i.end === undefined || new Date(i.end) > new Date()
) : [];
if (activeIncidents.length === 0) {
return { ...service, status: 'operational', description: 'All services operational' };
}
const severity = activeIncidents.some(i => i.severity === 'high') ? 'outage' : 'degraded';
return { ...service, status: severity, description: `${activeIncidents.length} active incident(s)` };
}
if (service.customParser === 'aws') {
// AWS status page is complex HTML - assume operational if reachable
return { ...service, status: 'operational', description: 'Status page reachable' };
}
if (service.customParser === 'rss') {
// Azure RSS feed - check if there are recent items (incidents)
const text = await response.text();
const hasRecentIncident = text.includes('<item>') &&
(text.includes('degradation') || text.includes('outage') || text.includes('incident'));
return {
...service,
status: hasRecentIncident ? 'degraded' : 'operational',
description: hasRecentIncident ? 'Recent incidents reported' : 'No recent incidents'
};
}
if (service.customParser === 'instatus') {
// Instatus format (Railway, etc.)
const data = await response.json();
const pageStatus = data.page?.status;
if (pageStatus === 'UP') {
return { ...service, status: 'operational', description: 'All systems operational' };
} else if (pageStatus === 'HASISSUES') {
return { ...service, status: 'degraded', description: 'Some issues reported' };
} else {
return { ...service, status: 'unknown', description: pageStatus || 'Unknown' };
}
}
if (service.customParser === 'statusio') {
// Status.io format (GitLab, Docker Hub)
const data = await response.json();
const overall = data.result?.status_overall;
const statusCode = overall?.status_code;
if (statusCode === 100) {
return { ...service, status: 'operational', description: overall.status || 'All systems operational' };
} else if (statusCode >= 300 && statusCode < 500) {
return { ...service, status: 'degraded', description: overall.status || 'Degraded performance' };
} else if (statusCode >= 500) {
return { ...service, status: 'outage', description: overall.status || 'Service disruption' };
}
return { ...service, status: 'unknown', description: overall?.status || 'Unknown status' };
}
if (service.customParser === 'slack') {
// Slack custom API format
const data = await response.json();
if (data.status === 'ok') {
return { ...service, status: 'operational', description: 'All systems operational' };
} else if (data.status === 'active' || data.active_incidents?.length > 0) {
const count = data.active_incidents?.length || 1;
return { ...service, status: 'degraded', description: `${count} active incident(s)` };
}
return { ...service, status: 'unknown', description: data.status || 'Unknown' };
}
if (service.customParser === 'stripe') {
// Stripe custom API format at /current
const data = await response.json();
if (data.largestatus === 'up') {
return { ...service, status: 'operational', description: data.message || 'All systems operational' };
} else if (data.largestatus === 'degraded') {
return { ...service, status: 'degraded', description: data.message || 'Degraded performance' };
} else if (data.largestatus === 'down') {
return { ...service, status: 'outage', description: data.message || 'Service disruption' };
}
return { ...service, status: 'unknown', description: data.message || 'Unknown' };
}
if (service.customParser === 'incidentio') {
// incident.io status pages (OpenAI, Linear, Replicate, Anthropic)
const text = await response.text();
// Check for HTML response (blocked)
if (text.startsWith('<!') || text.startsWith('<html')) {
// Try parsing HTML for status - incident.io pages have status in HTML
const operationalMatch = text.match(/All Systems Operational|fully operational|no issues/i);
if (operationalMatch) {
return { ...service, status: 'operational', description: 'All systems operational' };
}
const degradedMatch = text.match(/degraded|partial outage|experiencing issues/i);
if (degradedMatch) {
return { ...service, status: 'degraded', description: 'Some issues reported' };
}
return { ...service, status: 'unknown', description: 'Could not parse status' };
}
// Parse JSON response
try {
const data = JSON.parse(text);
const indicator = data.status?.indicator || '';
const description = data.status?.description || '';
if (indicator === 'none' || description.toLowerCase().includes('operational')) {
return { ...service, status: 'operational', description: description || 'All systems operational' };
} else if (indicator === 'minor' || indicator === 'maintenance') {
return { ...service, status: 'degraded', description: description || 'Minor issues' };
} else if (indicator === 'major' || indicator === 'critical') {
return { ...service, status: 'outage', description: description || 'Major outage' };
}
return { ...service, status: 'operational', description: description || 'Status OK' };
} catch {
return { ...service, status: 'unknown', description: 'Invalid response' };
}
}
const text = await response.text();
// Check if we got HTML instead of JSON (blocked/redirected)
if (text.startsWith('<!') || text.startsWith('<html')) {
return { ...service, status: 'unknown', description: 'Blocked by service' };
}
let data;
try {
data = JSON.parse(text);
} catch {
return { ...service, status: 'unknown', description: 'Invalid JSON response' };
}
// Handle different API formats
let status, description;
if (data.status?.indicator !== undefined) {
// Standard Statuspage.io format
status = normalizeStatus(data.status.indicator);
description = data.status.description || '';
} else if (data.status?.status) {
// Slack format
status = data.status.status === 'ok' ? 'operational' : 'degraded';
description = data.status.description || '';
} else if (data.page && data.status) {
// Alternative Statuspage format - check if status object exists
status = normalizeStatus(data.status.indicator || data.status.description);
description = data.status.description || 'Status available';
} else {
status = 'unknown';
description = 'Unknown format';
}
return { ...service, status, description };
} catch (error) {
return { ...service, status: 'unknown', description: error.message || 'Request failed' };
}
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const url = new URL(req.url);
const category = url.searchParams.get('category'); // cloud, dev, comm, ai, saas, or all
let servicesToCheck = SERVICES;
if (category && category !== 'all') {
servicesToCheck = SERVICES.filter(s => s.category === category);
}
// Check all services in parallel
const results = await Promise.all(servicesToCheck.map(checkStatusPage));
// Sort by status (outages first, then degraded, then operational)
const statusOrder = { outage: 0, degraded: 1, unknown: 2, operational: 3 };
results.sort((a, b) => statusOrder[a.status] - statusOrder[b.status]);
const summary = {
operational: results.filter(r => r.status === 'operational').length,
degraded: results.filter(r => r.status === 'degraded').length,
outage: results.filter(r => r.status === 'outage').length,
unknown: results.filter(r => r.status === 'unknown').length,
};
return new Response(JSON.stringify({
success: true,
timestamp: new Date().toISOString(),
summary,
services: results.map(r => ({
id: r.id,
name: r.name,
category: r.category,
status: r.status,
description: r.description,
})),
}), {
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30', // 1 min cache
},
});
}
-130
View File
@@ -1,130 +0,0 @@
export const config = { runtime: 'edge' };
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
const CACHE_TTL = 120;
let cachedResponse = null;
let cacheTimestamp = 0;
const DEFAULT_COINS = 'tether,usd-coin,dai,first-digital-usd,ethena-usde';
function buildFallbackResult() {
return {
timestamp: new Date().toISOString(),
summary: {
totalMarketCap: 0,
totalVolume24h: 0,
coinCount: 0,
depeggedCount: 0,
healthStatus: 'UNAVAILABLE',
},
stablecoins: [],
unavailable: true,
};
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: cors });
}
return new Response(null, { status: 204, headers: cors });
}
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
}
const now = Date.now();
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
return new Response(JSON.stringify(cachedResponse), {
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=300` },
});
}
const url = new URL(req.url);
const rawCoins = url.searchParams.get('coins') || DEFAULT_COINS;
const coins = rawCoins.split(',').filter(c => /^[a-z0-9-]+$/.test(c)).join(',') || DEFAULT_COINS;
try {
const controller = new AbortController();
const id = setTimeout(() => controller.abort(), 10000);
const apiUrl = `https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&ids=${coins}&order=market_cap_desc&sparkline=false&price_change_percentage=7d`;
const res = await fetch(apiUrl, {
signal: controller.signal,
headers: { 'Accept': 'application/json' },
});
clearTimeout(id);
if (res.status === 429) {
if (cachedResponse) {
return new Response(JSON.stringify(cachedResponse), {
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
});
}
return new Response(JSON.stringify({ error: 'Rate limited', timestamp: new Date().toISOString() }), {
status: 429,
headers: { ...cors, 'Content-Type': 'application/json' },
});
}
if (!res.ok) throw new Error(`CoinGecko HTTP ${res.status}`);
const data = await res.json();
const stablecoins = data.map(coin => {
const price = coin.current_price || 0;
const deviation = Math.abs(price - 1.0);
let pegStatus;
if (deviation <= 0.005) pegStatus = 'ON PEG';
else if (deviation <= 0.01) pegStatus = 'SLIGHT DEPEG';
else pegStatus = 'DEPEGGED';
return {
id: coin.id,
symbol: (coin.symbol || '').toUpperCase(),
name: coin.name,
price,
deviation: +(deviation * 100).toFixed(3),
pegStatus,
marketCap: coin.market_cap || 0,
volume24h: coin.total_volume || 0,
change24h: coin.price_change_percentage_24h || 0,
change7d: coin.price_change_percentage_7d_in_currency || 0,
image: coin.image,
};
});
const totalMarketCap = stablecoins.reduce((sum, c) => sum + c.marketCap, 0);
const totalVolume24h = stablecoins.reduce((sum, c) => sum + c.volume24h, 0);
const depeggedCount = stablecoins.filter(c => c.pegStatus === 'DEPEGGED').length;
const result = {
timestamp: new Date().toISOString(),
summary: {
totalMarketCap,
totalVolume24h,
coinCount: stablecoins.length,
depeggedCount,
healthStatus: depeggedCount === 0 ? 'HEALTHY' : depeggedCount === 1 ? 'CAUTION' : 'WARNING',
},
stablecoins,
};
cachedResponse = result;
cacheTimestamp = now;
return new Response(JSON.stringify(result), {
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=300` },
});
} catch (err) {
const fallback = cachedResponse || buildFallbackResult();
cachedResponse = fallback;
cacheTimestamp = now;
return new Response(JSON.stringify(fallback), {
status: 200,
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
});
}
}
-172
View File
@@ -1,172 +0,0 @@
/**
* Stock Market Index Endpoint
* Fetches weekly % change for a country's primary stock index via Yahoo Finance
* Redis cached (1h TTL)
*/
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
export const config = {
runtime: 'edge',
};
const CACHE_TTL_SECONDS = 3600; // 1 hour
const CACHE_VERSION = 'stock-v1';
const COUNTRY_INDEX = {
US: { symbol: '^GSPC', name: 'S&P 500' },
GB: { symbol: '^FTSE', name: 'FTSE 100' },
DE: { symbol: '^GDAXI', name: 'DAX' },
FR: { symbol: '^FCHI', name: 'CAC 40' },
JP: { symbol: '^N225', name: 'Nikkei 225' },
CN: { symbol: '000001.SS', name: 'SSE Composite' },
HK: { symbol: '^HSI', name: 'Hang Seng' },
IN: { symbol: '^BSESN', name: 'BSE Sensex' },
KR: { symbol: '^KS11', name: 'KOSPI' },
TW: { symbol: '^TWII', name: 'TAIEX' },
AU: { symbol: '^AXJO', name: 'ASX 200' },
BR: { symbol: '^BVSP', name: 'Bovespa' },
CA: { symbol: '^GSPTSE', name: 'TSX Composite' },
MX: { symbol: '^MXX', name: 'IPC Mexico' },
AR: { symbol: '^MERV', name: 'MERVAL' },
RU: { symbol: 'IMOEX.ME', name: 'MOEX' },
ZA: { symbol: '^J203.JO', name: 'JSE All Share' },
SA: { symbol: '^TASI.SR', name: 'Tadawul' },
AE: { symbol: 'DFMGI.AE', name: 'DFM General' },
IL: { symbol: '^TA125.TA', name: 'TA-125' },
TR: { symbol: 'XU100.IS', name: 'BIST 100' },
PL: { symbol: '^WIG20', name: 'WIG 20' },
NL: { symbol: '^AEX', name: 'AEX' },
CH: { symbol: '^SSMI', name: 'SMI' },
ES: { symbol: '^IBEX', name: 'IBEX 35' },
IT: { symbol: 'FTSEMIB.MI', name: 'FTSE MIB' },
SE: { symbol: '^OMX', name: 'OMX Stockholm 30' },
NO: { symbol: '^OSEAX', name: 'Oslo All Share' },
SG: { symbol: '^STI', name: 'STI' },
TH: { symbol: '^SET.BK', name: 'SET' },
MY: { symbol: '^KLSE', name: 'KLCI' },
ID: { symbol: '^JKSE', name: 'Jakarta Composite' },
PH: { symbol: 'PSEI.PS', name: 'PSEi' },
NZ: { symbol: '^NZ50', name: 'NZX 50' },
EG: { symbol: '^EGX30.CA', name: 'EGX 30' },
CL: { symbol: '^IPSA', name: 'IPSA' },
PE: { symbol: '^SPBLPGPT', name: 'S&P Lima' },
AT: { symbol: '^ATX', name: 'ATX' },
BE: { symbol: '^BFX', name: 'BEL 20' },
FI: { symbol: '^OMXH25', name: 'OMX Helsinki 25' },
DK: { symbol: '^OMXC25', name: 'OMX Copenhagen 25' },
IE: { symbol: '^ISEQ', name: 'ISEQ Overall' },
PT: { symbol: '^PSI20', name: 'PSI 20' },
CZ: { symbol: '^PX', name: 'PX Prague' },
HU: { symbol: '^BUX', name: 'BUX' },
};
export default async function handler(request) {
const cors = getCorsHeaders(request);
if (request.method === 'OPTIONS') return new Response(null, { status: 204, headers: cors });
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
if (request.method !== 'GET') {
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
const url = new URL(request.url);
const code = (url.searchParams.get('code') || '').toUpperCase();
if (!code) {
return new Response(JSON.stringify({ error: 'code parameter required' }), {
status: 400,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
const index = COUNTRY_INDEX[code];
if (!index) {
return new Response(JSON.stringify({ error: 'No stock index for country', code, available: false }), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
const cacheKey = `${CACHE_VERSION}:${code}`;
const cached = await getCachedJson(cacheKey);
if (cached && typeof cached === 'object' && cached.indexName) {
return new Response(JSON.stringify({ ...cached, cached: true }), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
try {
const encodedSymbol = encodeURIComponent(index.symbol);
// Use 1mo range to handle markets with different trading weeks (e.g. Sun-Thu Middle East)
const yahooUrl = `https://query1.finance.yahoo.com/v8/finance/chart/${encodedSymbol}?range=1mo&interval=1d`;
const res = await fetch(yahooUrl, {
headers: {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)',
},
});
if (!res.ok) {
console.error('[StockIndex] Yahoo error:', res.status, index.symbol);
return new Response(JSON.stringify({ error: 'Upstream error', available: false }), {
status: 502,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
const data = await res.json();
const result = data?.chart?.result?.[0];
if (!result) {
return new Response(JSON.stringify({ error: 'No data', available: false }), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
const allCloses = result.indicators?.quote?.[0]?.close?.filter(v => v != null);
if (!allCloses || allCloses.length < 2) {
return new Response(JSON.stringify({ error: 'Insufficient data', available: false }), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
// Take last ~5 trading days worth of data
const closes = allCloses.slice(-6);
const latest = closes[closes.length - 1];
const oldest = closes[0];
const weekChange = ((latest - oldest) / oldest) * 100;
const meta = result.meta || {};
const payload = {
available: true,
code,
symbol: index.symbol,
indexName: index.name,
price: latest.toFixed(2),
weekChangePercent: weekChange.toFixed(2),
currency: meta.currency || 'USD',
fetchedAt: new Date().toISOString(),
};
await setCachedJson(cacheKey, payload, CACHE_TTL_SECONDS);
return new Response(JSON.stringify(payload), {
status: 200,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
} catch (err) {
console.error('[StockIndex] Error:', err);
return new Response(JSON.stringify({ error: 'Internal error', available: false }), {
status: 500,
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
}
+1
View File
@@ -1,3 +1,4 @@
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
/**
* Story Page for Social Crawlers
* Returns HTML with proper og:image and twitter:card meta tags.
-176
View File
@@ -1,176 +0,0 @@
/**
* Temporal Baseline Anomaly Detection API
* Stores and queries activity baselines using Welford's online algorithm
* Backed by Upstash Redis for cross-user persistence
*
* GET ?type=military_flights&region=global&count=47 check anomaly
* POST { updates: [{ type, region, count }] } batch update baselines
*/
import { getCachedJson, setCachedJson, mget } from './_upstash-cache.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const BASELINE_TTL = 7776000; // 90 days in seconds
const MIN_SAMPLES = 10;
const Z_THRESHOLD_LOW = 1.5;
const Z_THRESHOLD_MEDIUM = 2.0;
const Z_THRESHOLD_HIGH = 3.0;
const VALID_TYPES = ['military_flights', 'vessels', 'protests', 'news', 'ais_gaps', 'satellite_fires'];
function makeKey(type, region, weekday, month) {
return `baseline:${type}:${region}:${weekday}:${month}`;
}
function getSeverity(zScore) {
if (zScore >= Z_THRESHOLD_HIGH) return 'critical';
if (zScore >= Z_THRESHOLD_MEDIUM) return 'high';
if (zScore >= Z_THRESHOLD_LOW) return 'medium';
return 'normal';
}
export default async function handler(request) {
const corsHeaders = getCorsHeaders(request, 'GET, POST, OPTIONS');
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
status: 403,
headers: { 'Content-Type': 'application/json' },
});
}
try {
if (request.method === 'GET') {
return await handleGet(request);
} else if (request.method === 'POST') {
return await handlePost(request);
}
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
status: 405,
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
});
} catch (err) {
console.error('[TemporalBaseline] Error:', err);
return new Response(JSON.stringify({ error: 'Internal error' }), {
status: 500,
headers: { 'Content-Type': 'application/json' },
});
}
}
async function handleGet(request) {
const { searchParams } = new URL(request.url);
const type = searchParams.get('type');
const region = searchParams.get('region') || 'global';
const count = parseFloat(searchParams.get('count'));
if (!type || !VALID_TYPES.includes(type) || isNaN(count)) {
return json({ error: 'Missing or invalid params: type, count required' }, 400);
}
const now = new Date();
const weekday = now.getUTCDay();
const month = now.getUTCMonth() + 1;
const key = makeKey(type, region, weekday, month);
const baseline = await getCachedJson(key);
if (!baseline || baseline.sampleCount < MIN_SAMPLES) {
return json({
anomaly: null,
learning: true,
sampleCount: baseline?.sampleCount || 0,
samplesNeeded: MIN_SAMPLES,
});
}
const variance = Math.max(0, baseline.m2 / (baseline.sampleCount - 1));
const stdDev = Math.sqrt(variance);
const zScore = stdDev > 0 ? Math.abs((count - baseline.mean) / stdDev) : 0;
const severity = getSeverity(zScore);
const multiplier = baseline.mean > 0
? Math.round((count / baseline.mean) * 100) / 100
: count > 0 ? 999 : 1;
return json({
anomaly: zScore >= Z_THRESHOLD_LOW ? {
zScore: Math.round(zScore * 100) / 100,
severity,
multiplier,
} : null,
baseline: {
mean: Math.round(baseline.mean * 100) / 100,
stdDev: Math.round(stdDev * 100) / 100,
sampleCount: baseline.sampleCount,
},
learning: false,
});
}
async function handlePost(request) {
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
if (contentLength > 51200) {
return json({ error: 'Payload too large' }, 413);
}
const body = await request.json();
const updates = body?.updates;
if (!Array.isArray(updates) || updates.length === 0) {
return json({ error: 'Body must have updates array' }, 400);
}
const batch = updates.slice(0, 20);
const now = new Date();
const weekday = now.getUTCDay();
const month = now.getUTCMonth() + 1;
const keys = batch.map(u => makeKey(u.type, u.region || 'global', weekday, month));
const existing = await mget(...keys);
const writes = [];
for (let i = 0; i < batch.length; i++) {
const { type, region = 'global', count } = batch[i];
if (!VALID_TYPES.includes(type) || typeof count !== 'number' || isNaN(count)) continue;
const prev = existing[i] || { mean: 0, m2: 0, sampleCount: 0 };
const n = prev.sampleCount + 1;
const delta = count - prev.mean;
const newMean = prev.mean + delta / n;
const delta2 = count - newMean;
const newM2 = prev.m2 + delta * delta2;
writes.push(setCachedJson(keys[i], {
mean: newMean,
m2: newM2,
sampleCount: n,
lastUpdated: now.toISOString(),
}, BASELINE_TTL));
}
if (writes.length > 0) {
await Promise.all(writes);
}
return json({ updated: writes.length });
}
function json(data, status = 200) {
return new Response(JSON.stringify(data), {
status,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'no-store',
},
});
}
-611
View File
@@ -1,611 +0,0 @@
/**
* Theater Posture API - Aggregates military aircraft by theater
* Caches results in Upstash Redis for cross-user efficiency
* TTL: 5 minutes (matches OpenSky refresh rate)
*/
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const CACHE_TTL_SECONDS = 300; // 5 minutes
const STALE_CACHE_TTL_SECONDS = 86400; // 24 hours - serve stale data when API is down
const BACKUP_CACHE_TTL_SECONDS = 604800; // 7 days - last resort backup
const CACHE_KEY = 'theater-posture:v4';
const STALE_CACHE_KEY = 'theater-posture:stale:v4';
const BACKUP_CACHE_KEY = 'theater-posture:backup:v4';
// Theater definitions (matches client-side POSTURE_THEATERS)
const POSTURE_THEATERS = [
{
id: 'iran-theater',
name: 'Iran Theater',
shortName: 'IRAN',
targetNation: 'Iran',
bounds: { north: 42, south: 20, east: 65, west: 30 },
thresholds: { elevated: 8, critical: 20 },
strikeIndicators: { minTankers: 2, minAwacs: 1, minFighters: 5 },
},
{
id: 'taiwan-theater',
name: 'Taiwan Strait',
shortName: 'TAIWAN',
targetNation: 'Taiwan',
bounds: { north: 30, south: 18, east: 130, west: 115 },
thresholds: { elevated: 6, critical: 15 },
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 4 },
},
{
id: 'baltic-theater',
name: 'Baltic Theater',
shortName: 'BALTIC',
targetNation: null,
bounds: { north: 65, south: 52, east: 32, west: 10 },
thresholds: { elevated: 5, critical: 12 },
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
},
{
id: 'blacksea-theater',
name: 'Black Sea',
shortName: 'BLACK SEA',
targetNation: null,
bounds: { north: 48, south: 40, east: 42, west: 26 },
thresholds: { elevated: 4, critical: 10 },
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
},
{
id: 'korea-theater',
name: 'Korean Peninsula',
shortName: 'KOREA',
targetNation: 'North Korea',
bounds: { north: 43, south: 33, east: 132, west: 124 },
thresholds: { elevated: 5, critical: 12 },
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
},
{
id: 'south-china-sea',
name: 'South China Sea',
shortName: 'SCS',
targetNation: null,
bounds: { north: 25, south: 5, east: 121, west: 105 },
thresholds: { elevated: 6, critical: 15 },
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 4 },
},
{
id: 'east-med-theater',
name: 'Eastern Mediterranean',
shortName: 'E.MED',
targetNation: null,
bounds: { north: 37, south: 33, east: 37, west: 25 },
thresholds: { elevated: 4, critical: 10 },
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
},
{
id: 'israel-gaza-theater',
name: 'Israel/Gaza',
shortName: 'GAZA',
targetNation: 'Gaza',
bounds: { north: 33, south: 29, east: 36, west: 33 },
thresholds: { elevated: 3, critical: 8 },
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
},
{
id: 'yemen-redsea-theater',
name: 'Yemen/Red Sea',
shortName: 'RED SEA',
targetNation: 'Yemen',
bounds: { north: 22, south: 11, east: 54, west: 32 },
thresholds: { elevated: 4, critical: 10 },
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
},
];
// Military hex database from ADS-B Exchange (updated daily at adsbexchange.com)
// Contains ~20k verified military aircraft hex IDs
import { MILITARY_HEX_LIST } from './data/military-hex-db.js';
// Create Set for O(1) lookup
const MILITARY_HEX_SET = new Set(MILITARY_HEX_LIST.map(h => h.toLowerCase()));
console.log(`[TheaterPosture] Loaded ${MILITARY_HEX_SET.size} military hex IDs from ADS-B Exchange`);
// Check if ICAO hex is in military database
function isMilitaryHex(hexId) {
if (!hexId) return false;
// Handle both string and number, remove ~ prefix if present
const cleanHex = String(hexId).replace(/^~/, '').toLowerCase();
return MILITARY_HEX_SET.has(cleanHex);
}
// Military callsign prefixes for identification
const MILITARY_PREFIXES = [
// US Military
'RCH', 'REACH', 'MOOSE', 'EVAC', 'DUSTOFF', 'PEDRO', // Transport/medevac
'DUKE', 'HAVOC', 'KNIFE', 'WARHAWK', 'VIPER', 'RAGE', 'FURY', // Fighters
'SHELL', 'TEXACO', 'ARCO', 'ESSO', 'PETRO', // Tankers
'SENTRY', 'AWACS', 'MAGIC', 'DISCO', 'DARKSTAR', // AWACS/ISR
'COBRA', 'PYTHON', 'RAPTOR', 'EAGLE', 'HAWK', 'TALON', // Various
'BOXER', 'OMNI', 'TOPCAT', 'SKULL', 'REAPER', 'HUNTER', // More callsigns
'ARMY', 'NAVY', 'USAF', 'USMC', 'USCG', // Service prefixes
'AE', 'CNV', 'PAT', 'SAM', 'EXEC', // Special missions
'OPS', 'CTF', 'TF', // Operations/Task Force
// NATO
'NATO', 'GAF', 'RRF', 'RAF', 'FAF', 'IAF', 'RNLAF', 'BAF', 'DAF', 'HAF', 'PAF',
'SWORD', 'LANCE', 'ARROW', 'SPARTAN', // NATO tactical
// Middle East (avoid UAE - conflicts with Emirates airline)
'RSAF', 'EMIRI', 'UAEAF', 'KAF', 'QAF', 'BAHAF', 'OMAAF', // Gulf states
'IRIAF', 'IRG', 'IRGC', // Iran (IAF already in NATO section covers Israel)
'TAF', 'TUAF', // Turkey
// Russia
'RSD', 'RF', 'RFF', 'VKS',
// China (NOTE: CCA is Air China airline, not military)
'CHN', 'PLAAF', 'PLA',
];
// Airline ICAO codes to exclude from military detection (Set for O(1) lookup)
const AIRLINE_CODES = new Set([
// Middle East
'SVA', 'QTR', 'THY', 'UAE', 'ETD', 'GFA', 'MEA', 'RJA', 'KAC', 'ELY',
'IAW', 'IRA', 'MSR', 'SYR', 'PGT', 'AXB', 'FDB', 'KNE', 'FAD', 'ADY', 'OMA',
'ABQ', 'ABY', 'NIA', 'FJA', 'SWR', 'HZA', 'OMS', 'EGF', 'NOS', 'SXD',
// Europe
'BAW', 'AFR', 'DLH', 'KLM', 'AUA', 'SAS', 'FIN', 'LOT', 'AZA', 'TAP', 'IBE',
'VLG', 'RYR', 'EZY', 'WZZ', 'NOZ', 'BEL', 'AEE', 'ROT',
// Asia
'AIC', 'CPA', 'SIA', 'MAS', 'THA', 'VNM', 'JAL', 'ANA', 'KAL', 'AAR', 'EVA',
'CAL', 'CCA', 'CES', 'CSN', 'HDA', 'CHH', 'CXA', 'GIA', 'PAL', 'SLK',
// Americas
'AAL', 'DAL', 'UAL', 'SWA', 'JBU', 'FFT', 'ASA', 'NKS', 'WJA', 'ACA',
// Cargo
'FDX', 'UPS', 'GTI', 'ABW', 'CLX', 'MPH',
// Generic
'AIR', 'SKY', 'JET',
]);
// Aircraft type detection from callsign patterns
function detectAircraftType(callsign) {
if (!callsign) return 'unknown';
const cs = callsign.toUpperCase().trim();
// Tankers
if (/^(SHELL|TEXACO|ARCO|ESSO|PETRO)/.test(cs)) return 'tanker';
if (/^(KC|STRAT)/.test(cs)) return 'tanker';
// AWACS
if (/^(SENTRY|AWACS|MAGIC|DISCO|DARKSTAR)/.test(cs)) return 'awacs';
if (/^(E3|E8|E6)/.test(cs)) return 'awacs';
// Transport
if (/^(RCH|REACH|MOOSE|EVAC|DUSTOFF)/.test(cs)) return 'transport';
if (/^(C17|C5|C130|C40)/.test(cs)) return 'transport';
// Reconnaissance
if (/^(HOMER|OLIVE|JAKE|PSEUDO|GORDO)/.test(cs)) return 'reconnaissance';
if (/^(RC|U2|SR)/.test(cs)) return 'reconnaissance';
// Drones/UAVs
if (/^(RQ|MQ|REAPER|PREDATOR|GLOBAL)/.test(cs)) return 'drone';
// Bombers
if (/^(DEATH|BONE|DOOM)/.test(cs)) return 'bomber';
if (/^(B52|B1|B2)/.test(cs)) return 'bomber';
// Default to unknown for unrecognized military aircraft
return 'unknown';
}
// Check if callsign is military
function isMilitaryCallsign(callsign) {
if (!callsign) return false;
const cs = callsign.toUpperCase().trim();
// Check prefixes
for (const prefix of MILITARY_PREFIXES) {
if (cs.startsWith(prefix)) return true;
}
// Check patterns - tactical callsigns (word + small number)
// DUKE01, VIPER12, RAGE1 but NOT airline codes like PGT5873, IAW9011
if (/^[A-Z]{4,}\d{1,3}$/.test(cs)) return true;
// Short tactical: 3 letters + 1-2 digits (but exclude common airlines)
// This catches OPS4, CTF01, TF12 but blocks SVA12, QTR76, etc.
if (/^[A-Z]{3}\d{1,2}$/.test(cs)) {
const prefix = cs.slice(0, 3);
if (!AIRLINE_CODES.has(prefix)) return true;
}
return false;
}
// Fetch military flights from OpenSky
async function fetchMilitaryFlights() {
const isSidecar = (process.env.LOCAL_API_MODE || '').includes('sidecar');
// Desktop sidecar: fetch directly from OpenSky (single user, no rate limit concern)
// Cloud: use Railway relay to avoid OpenSky rate limits across many users
const baseUrl = isSidecar
? 'https://opensky-network.org/api/states/all'
: (process.env.WS_RELAY_URL ? process.env.WS_RELAY_URL + '/opensky' : null);
if (!baseUrl) return [];
// Fetch global data with 20s timeout (Edge has 25s limit)
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 20000);
try {
console.log('[TheaterPosture] Fetching from:', baseUrl);
const response = await fetch(baseUrl, {
headers: {
'Accept': 'application/json',
'User-Agent': 'Mozilla/5.0 WorldMonitor/1.0',
},
signal: controller.signal,
});
if (!response.ok) {
throw new Error(`OpenSky API error: ${response.status}`);
}
const data = await response.json();
if (!data.states) return [];
// Filter and transform to military flights
const flights = [];
for (const state of data.states) {
const [icao24, callsign, , , , lon, lat, altitude, onGround, velocity, heading] = state;
// Skip if no position
if (lat == null || lon == null) continue;
// Skip if on ground
if (onGround) continue;
// Check if military (by callsign OR hex range)
const isMilitary = isMilitaryCallsign(callsign) || isMilitaryHex(icao24);
if (!isMilitary) continue;
flights.push({
id: icao24,
callsign: callsign?.trim() || '',
lat,
lon,
altitude: altitude || 0,
heading: heading || 0,
speed: velocity || 0,
aircraftType: detectAircraftType(callsign),
operator: 'unknown',
militaryHex: isMilitaryHex(icao24),
});
}
return flights;
} catch (err) {
if (err.name === 'AbortError') {
throw new Error('OpenSky API timeout - try again');
}
throw err;
} finally {
clearTimeout(timeoutId);
}
}
// Fetch military flights from Wingbits (fallback when OpenSky fails)
async function fetchMilitaryFlightsFromWingbits() {
const apiKey = process.env.WINGBITS_API_KEY;
if (!apiKey) {
console.log('[TheaterPosture] Wingbits not configured, skipping fallback');
return null;
}
console.log('[TheaterPosture] Trying Wingbits fallback...');
// Build batch request for all theaters
const areas = POSTURE_THEATERS.map(theater => ({
alias: theater.id,
by: 'box',
la: (theater.bounds.north + theater.bounds.south) / 2,
lo: (theater.bounds.east + theater.bounds.west) / 2,
w: Math.abs(theater.bounds.east - theater.bounds.west) * 60, // degrees to nm
h: Math.abs(theater.bounds.north - theater.bounds.south) * 60,
unit: 'nm',
}));
const controller = new AbortController();
const timeoutId = setTimeout(() => controller.abort(), 15000);
try {
const response = await fetch('https://customer-api.wingbits.com/v1/flights', {
method: 'POST',
headers: {
'x-api-key': apiKey,
'Accept': 'application/json',
'Content-Type': 'application/json',
},
body: JSON.stringify(areas),
signal: controller.signal,
});
if (!response.ok) {
console.warn('[TheaterPosture] Wingbits API error:', response.status);
return null;
}
const data = await response.json();
console.log('[TheaterPosture] Wingbits returned', data.length, 'theater results');
// Transform Wingbits data to our format
// Wingbits uses short field names: h=icao24, f=flight, la=lat, lo=lon, ab=alt, th=heading, gs=speed
const flights = [];
const seenIds = new Set();
for (const areaResult of data) {
// Batch response: each area result has flights in various possible formats
const areaFlights = areaResult.flights || areaResult.data || areaResult || [];
const flightList = Array.isArray(areaFlights) ? areaFlights : [];
for (const f of flightList) {
// Get icao24 - Wingbits uses 'h' for hex ID
const icao24 = f.h || f.icao24 || f.id;
if (!icao24) continue;
// Skip duplicates (aircraft may appear in multiple theaters)
if (seenIds.has(icao24)) continue;
seenIds.add(icao24);
// Get callsign - Wingbits uses 'f' for flight
const callsign = f.f || f.callsign || f.flight || '';
// Skip if not military (by callsign OR hex range)
const isMilitary = isMilitaryCallsign(callsign) || isMilitaryHex(icao24);
if (!isMilitary) continue;
flights.push({
id: icao24,
callsign: callsign.trim(),
lat: f.la || f.latitude || f.lat,
lon: f.lo || f.longitude || f.lon || f.lng,
altitude: f.ab || f.altitude || f.alt || 0,
heading: f.th || f.heading || f.track || 0,
speed: f.gs || f.groundSpeed || f.speed || f.velocity || 0,
aircraftType: detectAircraftType(callsign),
operator: f.operator || 'unknown',
source: 'wingbits',
militaryHex: isMilitaryHex(icao24),
});
}
}
console.log('[TheaterPosture] Wingbits: found', flights.length, 'military flights');
return flights;
} catch (err) {
console.error('[TheaterPosture] Wingbits fetch error:', err.message);
return null;
} finally {
clearTimeout(timeoutId);
}
}
// Calculate theater postures
function calculatePostures(flights) {
const summaries = [];
for (const theater of POSTURE_THEATERS) {
// Filter flights within theater bounds
const theaterFlights = flights.filter(f =>
f.lat >= theater.bounds.south &&
f.lat <= theater.bounds.north &&
f.lon >= theater.bounds.west &&
f.lon <= theater.bounds.east
);
// Count by type
const byType = {
fighters: theaterFlights.filter(f => f.aircraftType === 'fighter').length,
tankers: theaterFlights.filter(f => f.aircraftType === 'tanker').length,
awacs: theaterFlights.filter(f => f.aircraftType === 'awacs').length,
reconnaissance: theaterFlights.filter(f => f.aircraftType === 'reconnaissance').length,
transport: theaterFlights.filter(f => f.aircraftType === 'transport').length,
bombers: theaterFlights.filter(f => f.aircraftType === 'bomber').length,
drones: theaterFlights.filter(f => f.aircraftType === 'drone').length,
unknown: theaterFlights.filter(f => f.aircraftType === 'unknown').length,
};
const total = Object.values(byType).reduce((a, b) => a + b, 0);
// Determine posture level
const postureLevel = total >= theater.thresholds.critical ? 'critical' :
total >= theater.thresholds.elevated ? 'elevated' : 'normal';
// Check strike capability
const strikeCapable =
byType.tankers >= theater.strikeIndicators.minTankers &&
byType.awacs >= theater.strikeIndicators.minAwacs &&
byType.fighters >= theater.strikeIndicators.minFighters;
// Build summary string
const parts = [];
if (byType.fighters > 0) parts.push(`${byType.fighters} fighters`);
if (byType.tankers > 0) parts.push(`${byType.tankers} tankers`);
if (byType.awacs > 0) parts.push(`${byType.awacs} AWACS`);
if (byType.reconnaissance > 0) parts.push(`${byType.reconnaissance} recon`);
if (byType.bombers > 0) parts.push(`${byType.bombers} bombers`);
if (byType.transport > 0) parts.push(`${byType.transport} transport`);
if (byType.drones > 0) parts.push(`${byType.drones} drones`);
if (byType.unknown > 0) parts.push(`${byType.unknown} other`);
const summary = parts.join(', ') || 'No military aircraft';
// Build headline
const headline = postureLevel === 'critical'
? `Critical military buildup - ${theater.name}`
: postureLevel === 'elevated'
? `Elevated military activity - ${theater.name}`
: `Normal activity - ${theater.name}`;
// Build byOperator map for aircraft
const byOperator = {};
for (const f of theaterFlights) {
const op = f.operator || 'unknown';
byOperator[op] = (byOperator[op] || 0) + 1;
}
summaries.push({
theaterId: theater.id,
theaterName: theater.name,
shortName: theater.shortName,
targetNation: theater.targetNation,
// Aircraft
fighters: byType.fighters,
tankers: byType.tankers,
awacs: byType.awacs,
reconnaissance: byType.reconnaissance,
transport: byType.transport,
bombers: byType.bombers,
drones: byType.drones,
unknown: byType.unknown,
totalAircraft: total,
// Vessels (populated client-side)
destroyers: 0,
frigates: 0,
carriers: 0,
submarines: 0,
patrol: 0,
auxiliaryVessels: 0,
totalVessels: 0,
// By operator (aircraft + vessels added client-side)
byOperator,
// Metadata
postureLevel,
strikeCapable,
trend: 'stable',
changePercent: 0,
summary,
headline,
centerLat: (theater.bounds.north + theater.bounds.south) / 2,
centerLon: (theater.bounds.east + theater.bounds.west) / 2,
bounds: theater.bounds,
});
}
return summaries;
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: corsHeaders });
}
if (req.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: corsHeaders });
}
try {
// Try to get from cache first
const cached = await getCachedJson(CACHE_KEY);
if (cached) {
console.log('[TheaterPosture] Cache hit');
return Response.json({
...cached,
cached: true,
}, {
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
},
});
}
// Fetch and calculate - try OpenSky first, then Wingbits fallback
console.log('[TheaterPosture] Fetching fresh data...');
let flights;
let source = 'opensky';
try {
flights = await fetchMilitaryFlights();
} catch (openskyError) {
console.warn('[TheaterPosture] OpenSky failed:', openskyError.message);
console.log('[TheaterPosture] Trying Wingbits fallback...');
flights = await fetchMilitaryFlightsFromWingbits();
if (flights && flights.length > 0) {
source = 'wingbits';
console.log('[TheaterPosture] Wingbits fallback succeeded:', flights.length, 'flights');
} else {
// Both failed, re-throw OpenSky error to trigger cache fallback
throw openskyError;
}
}
const postures = calculatePostures(flights);
const result = {
postures,
totalFlights: flights.length,
timestamp: new Date().toISOString(),
cached: false,
source, // 'opensky' or 'wingbits'
};
// Cache the result (regular, stale, and long-term backup)
await Promise.all([
setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS),
setCachedJson(STALE_CACHE_KEY, result, STALE_CACHE_TTL_SECONDS),
setCachedJson(BACKUP_CACHE_KEY, result, BACKUP_CACHE_TTL_SECONDS),
]);
return Response.json(result, {
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
},
});
} catch (error) {
console.warn('[TheaterPosture] Error:', error.message);
// Try to return cached data when API fails (stale first, then backup)
const stale = await getCachedJson(STALE_CACHE_KEY);
if (stale) {
console.log('[TheaterPosture] Returning stale cached data (24h) due to API error');
return Response.json({
...stale,
cached: true,
stale: true,
error: 'Using cached data - live feed temporarily unavailable',
}, {
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
},
});
}
const backup = await getCachedJson(BACKUP_CACHE_KEY);
if (backup) {
console.log('[TheaterPosture] Returning backup cached data (7d) due to API error');
return Response.json({
...backup,
cached: true,
stale: true,
error: 'Using backup data - live feed temporarily unavailable',
}, {
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
},
});
}
// No cached data available - return error
return Response.json({
error: error.message,
postures: [],
timestamp: new Date().toISOString(),
}, {
status: 500,
headers: corsHeaders,
});
}
}
-237
View File
@@ -1,237 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { createIpRateLimiter } from './_ip-rate-limit.js';
export const config = { runtime: 'edge' };
const CACHE_KEY = 'ucdp:gedevents:v2';
const CACHE_TTL_SECONDS = 6 * 60 * 60;
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
const UCDP_PAGE_SIZE = 1000;
const MAX_PAGES = 12;
const TRAILING_WINDOW_MS = 365 * 24 * 60 * 60 * 1000;
let fallbackCache = { data: null, timestamp: 0 };
const rateLimiter = createIpRateLimiter({
limit: 15,
windowMs: 60 * 1000,
maxEntries: 5000,
});
function getClientIp(req) {
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
req.headers.get('x-real-ip') ||
'unknown';
}
function toErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'unknown error');
}
function isValidResult(data) {
return Boolean(data && typeof data === 'object' && Array.isArray(data.data));
}
const VIOLENCE_TYPE_MAP = {
1: 'state-based',
2: 'non-state',
3: 'one-sided',
};
function parseDateMs(value) {
if (!value) return NaN;
return Date.parse(String(value));
}
function getMaxDateMs(events) {
let maxMs = NaN;
for (const event of events) {
const ms = parseDateMs(event?.date_start);
if (!Number.isFinite(ms)) continue;
if (!Number.isFinite(maxMs) || ms > maxMs) {
maxMs = ms;
}
}
return maxMs;
}
function buildVersionCandidates() {
const year = new Date().getFullYear() - 2000;
return Array.from(new Set([
`${year}.1`,
`${year - 1}.1`,
'25.1',
'24.1',
]));
}
async function fetchGedPage(version, page) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 8000);
try {
const response = await fetch(
`https://ucdpapi.pcr.uu.se/api/gedevents/${version}?pagesize=${UCDP_PAGE_SIZE}&page=${page}`,
{ headers: { Accept: 'application/json' }, signal: controller.signal }
);
if (!response.ok) {
throw new Error(`UCDP GED API error (${version}, page ${page}): ${response.status}`);
}
return response.json();
} finally {
clearTimeout(timeout);
}
}
async function discoverGedVersion() {
const candidates = buildVersionCandidates();
for (const version of candidates) {
try {
const page0 = await fetchGedPage(version, 0);
if (Array.isArray(page0?.Result)) {
return { version, page0 };
}
} catch {
// Try the next version candidate.
}
}
throw new Error('Unable to fetch UCDP GED metadata from known API versions');
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return Response.json({ error: 'Method not allowed', data: [] }, {
status: 405, headers: corsHeaders,
});
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed', data: [] }, {
status: 403, headers: corsHeaders,
});
}
const ip = getClientIp(req);
if (!rateLimiter.check(ip)) {
return Response.json({ error: 'Rate limited', data: [] }, {
status: 429,
headers: { ...corsHeaders, 'Retry-After': '60' },
});
}
const now = Date.now();
const cached = await getCachedJson(CACHE_KEY);
if (isValidResult(cached)) {
recordCacheTelemetry('/api/ucdp-events', 'REDIS-HIT');
return Response.json(cached, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
});
}
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
recordCacheTelemetry('/api/ucdp-events', 'MEMORY-HIT');
return Response.json(fallbackCache.data, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
});
}
try {
const { version, page0 } = await discoverGedVersion();
const totalPages = Math.max(1, Number(page0?.TotalPages) || 1);
const newestPage = totalPages - 1;
let allEvents = [];
let latestDatasetMs = NaN;
for (let offset = 0; offset < MAX_PAGES && (newestPage - offset) >= 0; offset++) {
const page = newestPage - offset;
const rawData = page === 0 ? page0 : await fetchGedPage(version, page);
const events = Array.isArray(rawData?.Result) ? rawData.Result : [];
allEvents = allEvents.concat(events);
const pageMaxMs = getMaxDateMs(events);
if (!Number.isFinite(latestDatasetMs) && Number.isFinite(pageMaxMs)) {
latestDatasetMs = pageMaxMs;
}
// Pages are ordered oldest->newest; once we are fully outside trailing window, stop.
if (Number.isFinite(latestDatasetMs) && Number.isFinite(pageMaxMs)) {
const cutoffMs = latestDatasetMs - TRAILING_WINDOW_MS;
if (pageMaxMs < cutoffMs) {
break;
}
}
}
const sanitized = allEvents
.filter((event) => {
if (!Number.isFinite(latestDatasetMs)) return true;
const eventMs = parseDateMs(event?.date_start);
if (!Number.isFinite(eventMs)) return false;
return eventMs >= (latestDatasetMs - TRAILING_WINDOW_MS);
})
.map(e => ({
id: String(e.id || ''),
date_start: e.date_start || '',
date_end: e.date_end || '',
latitude: Number(e.latitude) || 0,
longitude: Number(e.longitude) || 0,
country: e.country || '',
side_a: (e.side_a || '').substring(0, 200),
side_b: (e.side_b || '').substring(0, 200),
deaths_best: Number(e.best) || 0,
deaths_low: Number(e.low) || 0,
deaths_high: Number(e.high) || 0,
type_of_violence: VIOLENCE_TYPE_MAP[e.type_of_violence] || 'state-based',
source_original: (e.source_original || '').substring(0, 300),
}))
.sort((a, b) => {
const bMs = parseDateMs(b.date_start);
const aMs = parseDateMs(a.date_start);
return (Number.isFinite(bMs) ? bMs : 0) - (Number.isFinite(aMs) ? aMs : 0);
});
const result = {
success: true,
count: sanitized.length,
data: sanitized,
version,
cached_at: new Date().toISOString(),
};
fallbackCache = { data: result, timestamp: now };
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/ucdp-events', 'MISS');
return Response.json(result, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
});
} catch (error) {
if (isValidResult(fallbackCache.data)) {
recordCacheTelemetry('/api/ucdp-events', 'STALE');
return Response.json(fallbackCache.data, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
});
}
recordCacheTelemetry('/api/ucdp-events', 'ERROR');
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, data: [] }, {
status: 500, headers: corsHeaders,
});
}
}
-150
View File
@@ -1,150 +0,0 @@
// UCDP (Uppsala Conflict Data Program) proxy
// Returns conflict classification per country with intensity levels
// No auth required - public API
export const config = { runtime: 'edge' };
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
const CACHE_KEY = 'ucdp:country-conflicts:v2';
const CACHE_TTL_SECONDS = 24 * 60 * 60; // 24 hours (annual data)
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
const RESPONSE_CACHE_CONTROL = 'public, max-age=3600';
// In-memory fallback when Redis is unavailable.
let fallbackCache = { data: null, timestamp: 0 };
function isValidResult(data) {
return Boolean(
data &&
typeof data === 'object' &&
Array.isArray(data.conflicts)
);
}
function toErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'unknown error');
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const now = Date.now();
const cached = await getCachedJson(CACHE_KEY);
if (isValidResult(cached)) {
recordCacheTelemetry('/api/ucdp', 'REDIS-HIT');
return Response.json(cached, {
status: 200,
headers: {
...cors,
'Cache-Control': RESPONSE_CACHE_CONTROL,
'X-Cache': 'REDIS-HIT',
},
});
}
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
recordCacheTelemetry('/api/ucdp', 'MEMORY-HIT');
return Response.json(fallbackCache.data, {
status: 200,
headers: {
...cors,
'Cache-Control': RESPONSE_CACHE_CONTROL,
'X-Cache': 'MEMORY-HIT',
},
});
}
try {
// Fetch all pages of conflicts
let allConflicts = [];
let page = 0;
let totalPages = 1;
while (page < totalPages) {
const response = await fetch(`https://ucdpapi.pcr.uu.se/api/ucdpprioconflict/24.1?pagesize=100&page=${page}`, {
headers: { 'Accept': 'application/json' },
});
if (!response.ok) {
throw new Error(`UCDP API error: ${response.status}`);
}
const rawData = await response.json();
totalPages = rawData.TotalPages || 1;
const conflicts = rawData.Result || [];
allConflicts = allConflicts.concat(conflicts);
page++;
}
// Fields are snake_case: conflict_id, location, side_a, side_b, year, intensity_level, type_of_conflict
const countryConflicts = {};
for (const c of allConflicts) {
const name = c.location || '';
const year = parseInt(c.year, 10) || 0;
const intensity = parseInt(c.intensity_level, 10) || 0;
const entry = {
conflictId: parseInt(c.conflict_id, 10) || 0,
conflictName: c.side_b || '',
location: name,
year,
intensityLevel: intensity,
typeOfConflict: parseInt(c.type_of_conflict, 10) || 0,
startDate: c.start_date,
startDate2: c.start_date2,
sideA: c.side_a,
sideB: c.side_b,
region: c.region,
};
// Keep most recent / highest intensity per location
if (!countryConflicts[name] || year > countryConflicts[name].year ||
(year === countryConflicts[name].year && intensity > countryConflicts[name].intensityLevel)) {
countryConflicts[name] = entry;
}
}
const result = {
success: true,
count: Object.keys(countryConflicts).length,
conflicts: Object.values(countryConflicts),
cached_at: new Date().toISOString(),
};
fallbackCache = { data: result, timestamp: now };
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/ucdp', 'MISS');
return Response.json(result, {
status: 200,
headers: {
...cors,
'Cache-Control': RESPONSE_CACHE_CONTROL,
'X-Cache': 'MISS',
},
});
} catch (error) {
if (isValidResult(fallbackCache.data)) {
recordCacheTelemetry('/api/ucdp', 'STALE');
return Response.json(fallbackCache.data, {
status: 200,
headers: {
...cors,
'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120',
'X-Cache': 'STALE',
},
});
}
recordCacheTelemetry('/api/ucdp', 'ERROR');
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, conflicts: [] }, {
status: 500,
headers: { ...cors },
});
}
}
-270
View File
@@ -1,270 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { createIpRateLimiter } from './_ip-rate-limit.js';
export const config = { runtime: 'edge' };
const CACHE_KEY = 'unhcr:population:v2';
const CACHE_TTL_SECONDS = 24 * 60 * 60;
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
let fallbackCache = { data: null, timestamp: 0 };
const rateLimiter = createIpRateLimiter({
limit: 20,
windowMs: 60 * 1000,
maxEntries: 5000,
});
function getClientIp(req) {
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
req.headers.get('x-real-ip') ||
'unknown';
}
function toErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'unknown error');
}
function isValidResult(data) {
return Boolean(data && typeof data === 'object' && Array.isArray(data.countries));
}
const COUNTRY_CENTROIDS = {
AFG: [33.9, 67.7], SYR: [35.0, 38.0], UKR: [48.4, 31.2], SDN: [15.5, 32.5],
SSD: [6.9, 31.3], SOM: [5.2, 46.2], COD: [-4.0, 21.8], MMR: [19.8, 96.7],
YEM: [15.6, 48.5], ETH: [9.1, 40.5], VEN: [6.4, -66.6], IRQ: [33.2, 43.7],
COL: [4.6, -74.1], NGA: [9.1, 7.5], PSE: [31.9, 35.2], TUR: [39.9, 32.9],
DEU: [51.2, 10.4], PAK: [30.4, 69.3], UGA: [1.4, 32.3], BGD: [23.7, 90.4],
KEN: [0.0, 38.0], TCD: [15.5, 19.0], JOR: [31.0, 36.0], LBN: [33.9, 35.5],
EGY: [26.8, 30.8], IRN: [32.4, 53.7], TZA: [-6.4, 34.9], RWA: [-1.9, 29.9],
CMR: [7.4, 12.4], MLI: [17.6, -4.0], BFA: [12.3, -1.6], NER: [17.6, 8.1],
CAF: [6.6, 20.9], MOZ: [-18.7, 35.5], USA: [37.1, -95.7], FRA: [46.2, 2.2],
GBR: [55.4, -3.4], IND: [20.6, 79.0], CHN: [35.9, 104.2], RUS: [61.5, 105.3],
};
async function fetchUnhcrYearItems(year) {
const limit = 10000;
const maxPageGuard = 25;
const items = [];
for (let page = 1; page <= maxPageGuard; page++) {
const response = await fetch(
`https://api.unhcr.org/population/v1/population/?year=${year}&limit=${limit}&page=${page}`,
{ headers: { Accept: 'application/json' } }
);
if (!response.ok) return null;
const data = await response.json();
const pageItems = Array.isArray(data.items) ? data.items : [];
if (pageItems.length === 0) break;
items.push(...pageItems);
const maxPages = Number(data.maxPages);
if (Number.isFinite(maxPages) && maxPages > 0) {
if (page >= maxPages) break;
continue;
}
if (pageItems.length < limit) break;
}
return items;
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
}
const ip = getClientIp(req);
if (!rateLimiter.check(ip)) {
return Response.json({ error: 'Rate limited' }, {
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
});
}
const now = Date.now();
const cached = await getCachedJson(CACHE_KEY);
if (isValidResult(cached)) {
recordCacheTelemetry('/api/unhcr-population', 'REDIS-HIT');
return Response.json(cached, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
});
}
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
recordCacheTelemetry('/api/unhcr-population', 'MEMORY-HIT');
return Response.json(fallbackCache.data, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
});
}
try {
const currentYear = new Date().getFullYear();
let rawItems = [];
let dataYearUsed = null;
for (let year = currentYear; year >= currentYear - 2; year--) {
const yearItems = await fetchUnhcrYearItems(year);
if (!yearItems) {
continue;
}
rawItems = yearItems;
if (rawItems.length > 0) {
dataYearUsed = year;
break;
}
}
const byOrigin = {};
const byAsylum = {};
const flowMap = {};
let totalRefugees = 0, totalAsylumSeekers = 0, totalIdps = 0, totalStateless = 0;
for (const item of rawItems) {
const originCode = item.coo_iso || '';
const asylumCode = item.coa_iso || '';
const refugees = Number(item.refugees) || 0;
const asylumSeekers = Number(item.asylum_seekers) || 0;
const idps = Number(item.idps) || 0;
const stateless = Number(item.stateless) || 0;
totalRefugees += refugees;
totalAsylumSeekers += asylumSeekers;
totalIdps += idps;
totalStateless += stateless;
if (originCode) {
if (!byOrigin[originCode]) byOrigin[originCode] = { refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, name: item.coo_name || originCode };
byOrigin[originCode].refugees += refugees;
byOrigin[originCode].asylumSeekers += asylumSeekers;
byOrigin[originCode].idps += idps;
byOrigin[originCode].stateless += stateless;
}
if (asylumCode) {
if (!byAsylum[asylumCode]) byAsylum[asylumCode] = { refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, name: item.coa_name || asylumCode };
byAsylum[asylumCode].refugees += refugees;
byAsylum[asylumCode].asylumSeekers += asylumSeekers;
}
if (originCode && asylumCode && refugees > 0) {
const flowKey = `${originCode}->${asylumCode}`;
if (!flowMap[flowKey]) {
flowMap[flowKey] = {
originCode, originName: item.coo_name || originCode,
asylumCode, asylumName: item.coa_name || asylumCode,
refugees: 0,
};
}
flowMap[flowKey].refugees += refugees;
}
}
const countries = {};
for (const [code, data] of Object.entries(byOrigin)) {
const centroid = COUNTRY_CENTROIDS[code];
countries[code] = {
code, name: data.name,
refugees: data.refugees, asylumSeekers: data.asylumSeekers,
idps: data.idps, stateless: data.stateless,
totalDisplaced: data.refugees + data.asylumSeekers + data.idps + data.stateless,
hostRefugees: 0,
hostAsylumSeekers: 0,
hostTotal: 0,
lat: centroid?.[0], lon: centroid?.[1],
};
}
for (const [code, data] of Object.entries(byAsylum)) {
const hostRefugees = data.refugees;
const hostAsylumSeekers = data.asylumSeekers;
const hostTotal = hostRefugees + hostAsylumSeekers;
if (!countries[code]) {
const centroid = COUNTRY_CENTROIDS[code];
countries[code] = {
code, name: data.name,
refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, totalDisplaced: 0,
hostRefugees,
hostAsylumSeekers,
hostTotal,
lat: centroid?.[0], lon: centroid?.[1],
};
} else {
countries[code].hostRefugees = hostRefugees;
countries[code].hostAsylumSeekers = hostAsylumSeekers;
countries[code].hostTotal = hostTotal;
}
}
const topFlows = Object.values(flowMap)
.sort((a, b) => b.refugees - a.refugees)
.slice(0, 50)
.map(f => {
const oC = COUNTRY_CENTROIDS[f.originCode];
const aC = COUNTRY_CENTROIDS[f.asylumCode];
return {
...f,
originLat: oC?.[0], originLon: oC?.[1],
asylumLat: aC?.[0], asylumLon: aC?.[1],
};
});
const result = {
success: true,
year: dataYearUsed ?? currentYear,
globalTotals: {
refugees: totalRefugees,
asylumSeekers: totalAsylumSeekers,
idps: totalIdps,
stateless: totalStateless,
total: totalRefugees + totalAsylumSeekers + totalIdps + totalStateless,
},
countries: Object.values(countries).sort((a, b) => {
const aSize = Math.max(a.totalDisplaced || 0, a.hostTotal || 0);
const bSize = Math.max(b.totalDisplaced || 0, b.hostTotal || 0);
return bSize - aSize;
}),
topFlows,
cached_at: new Date().toISOString(),
};
fallbackCache = { data: result, timestamp: now };
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
recordCacheTelemetry('/api/unhcr-population', 'MISS');
return Response.json(result, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
});
} catch (error) {
if (isValidResult(fallbackCache.data)) {
recordCacheTelemetry('/api/unhcr-population', 'STALE');
return Response.json(fallbackCache.data, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
});
}
recordCacheTelemetry('/api/unhcr-population', 'ERROR');
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, countries: [], topFlows: [] }, {
status: 500, headers: corsHeaders,
});
}
}
+1
View File
@@ -1,3 +1,4 @@
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
export const config = { runtime: 'edge' };
const RELEASES_URL = 'https://api.github.com/repos/koala73/worldmonitor/releases/latest';
-304
View File
@@ -1,304 +0,0 @@
// Wingbits API proxy - keeps API key server-side
// Note: Edge runtime is stateless - caching happens client-side and via HTTP Cache-Control
import { getCorsHeaders, isDisallowedOrigin } from '../_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(req) {
const url = new URL(req.url);
const path = url.pathname.replace('/api/wingbits', '');
const corsHeaders = getCorsHeaders(req, 'GET, POST, OPTIONS');
// Handle CORS preflight
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, {
status: 403,
headers: corsHeaders,
});
}
return new Response(null, {
status: 204,
headers: corsHeaders,
});
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed' }, {
status: 403,
headers: corsHeaders,
});
}
// Get API key from server-side env
const apiKey = process.env.WINGBITS_API_KEY;
if (!apiKey) {
return Response.json({
error: 'Wingbits not configured',
configured: false
}, {
status: 200,
headers: corsHeaders,
});
}
// Route: GET /details/:icao24 - Aircraft details
const detailsMatch = path.match(/^\/details\/([a-fA-F0-9]+)$/);
if (detailsMatch) {
const icao24 = detailsMatch[1].toLowerCase();
try {
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
headers: {
'x-api-key': apiKey,
'Accept': 'application/json',
},
});
if (!response.ok) {
return Response.json({
error: `Wingbits API error: ${response.status}`,
icao24,
}, {
status: response.status,
headers: corsHeaders,
});
}
const data = await response.json();
return Response.json(data, {
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', // 24h - aircraft details rarely change
},
});
} catch (error) {
return Response.json({
error: `Fetch failed: ${error.message}`,
icao24,
}, {
status: 500,
headers: corsHeaders,
});
}
}
// Route: POST /details/batch - Batch lookup multiple aircraft (parallel)
if (path === '/details/batch' && req.method === 'POST') {
try {
const body = await req.json();
const icao24List = body.icao24s || [];
if (!Array.isArray(icao24List) || icao24List.length === 0) {
return Response.json({ error: 'icao24s array required' }, {
status: 400,
headers: corsHeaders,
});
}
// Limit batch size to avoid overwhelming the API
const limitedList = icao24List.slice(0, 20).map(id => id.toLowerCase());
const results = {};
// Fetch all in parallel
const fetchPromises = limitedList.map(async (icao24) => {
try {
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
headers: {
'x-api-key': apiKey,
'Accept': 'application/json',
},
});
if (response.ok) {
const data = await response.json();
return { icao24, data };
}
} catch {
// Skip failed lookups
}
return null;
});
const fetchResults = await Promise.all(fetchPromises);
for (const result of fetchResults) {
if (result) {
results[result.icao24] = result.data;
}
}
return Response.json({
results,
fetched: Object.keys(results).length,
requested: limitedList.length,
}, {
headers: {
...corsHeaders,
},
});
} catch (error) {
return Response.json({
error: `Batch lookup failed: ${error.message}`,
}, {
status: 500,
headers: corsHeaders,
});
}
}
// Route: GET /flights - Get live flight positions in a geographic area
// Query params: la (lat), lo (lon), w (width), h (height), unit (km|nm)
if (path === '/flights' && req.method === 'GET') {
try {
const params = new URLSearchParams(url.search);
const la = params.get('la') || params.get('lat');
const lo = params.get('lo') || params.get('lon');
const w = params.get('w') || params.get('width') || '500';
const h = params.get('h') || params.get('height') || '500';
const unit = params.get('unit') || 'nm';
if (!la || !lo) {
return Response.json({ error: 'lat (la) and lon (lo) required' }, {
status: 400,
headers: corsHeaders,
});
}
const wingbitsUrl = `https://customer-api.wingbits.com/v1/flights?by=box&la=${la}&lo=${lo}&w=${w}&h=${h}&unit=${unit}`;
console.log('[Wingbits] Fetching flights:', wingbitsUrl);
const response = await fetch(wingbitsUrl, {
headers: {
'x-api-key': apiKey,
'Accept': 'application/json',
},
});
if (!response.ok) {
const errorText = await response.text();
console.error('[Wingbits] API error:', response.status, errorText);
return Response.json({
error: `Wingbits API error: ${response.status}`,
details: errorText,
}, {
status: response.status,
headers: corsHeaders,
});
}
const data = await response.json();
console.log('[Wingbits] Got', Array.isArray(data) ? data.length : 0, 'flights');
return Response.json(data, {
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15', // 30 seconds - live data
},
});
} catch (error) {
console.error('[Wingbits] Flights fetch error:', error);
return Response.json({
error: `Fetch failed: ${error.message}`,
}, {
status: 500,
headers: corsHeaders,
});
}
}
// Route: POST /flights/batch - Get flights for multiple areas (for theater posture)
if (path === '/flights/batch' && req.method === 'POST') {
try {
const body = await req.json();
const areas = body.areas || [];
if (!Array.isArray(areas) || areas.length === 0) {
return Response.json({ error: 'areas array required' }, {
status: 400,
headers: corsHeaders,
});
}
// Wingbits batch endpoint format
const wingbitsAreas = areas.map(area => ({
alias: area.id || area.alias,
by: 'box',
la: (area.north + area.south) / 2,
lo: (area.east + area.west) / 2,
w: Math.abs(area.east - area.west) * 60, // degrees to nautical miles (approx)
h: Math.abs(area.north - area.south) * 60,
unit: 'nm',
}));
const response = await fetch('https://customer-api.wingbits.com/v1/flights', {
method: 'POST',
headers: {
'x-api-key': apiKey,
'Accept': 'application/json',
'Content-Type': 'application/json',
},
body: JSON.stringify(wingbitsAreas),
});
if (!response.ok) {
const errorText = await response.text();
return Response.json({
error: `Wingbits API error: ${response.status}`,
details: errorText,
}, {
status: response.status,
headers: corsHeaders,
});
}
const data = await response.json();
console.log('[Wingbits] Batch got', data.length, 'area results');
return Response.json(data, {
headers: {
...corsHeaders,
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
},
});
} catch (error) {
console.error('[Wingbits] Batch flights error:', error);
return Response.json({
error: `Fetch failed: ${error.message}`,
}, {
status: 500,
headers: corsHeaders,
});
}
}
// Route: GET /health - Check Wingbits status
if (path === '/health' || path === '') {
try {
const response = await fetch('https://customer-api.wingbits.com/health', {
headers: { 'x-api-key': apiKey },
});
const data = await response.json();
return Response.json({
...data,
configured: true,
}, {
headers: corsHeaders,
});
} catch (error) {
return Response.json({
error: error.message,
configured: true,
}, {
status: 500,
headers: corsHeaders,
});
}
}
return Response.json({ error: 'Not found' }, {
status: 404,
headers: corsHeaders,
});
}
-57
View File
@@ -1,57 +0,0 @@
// Wingbits single aircraft details
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(req, { params }) {
const icao24 = params.icao24?.toLowerCase();
const apiKey = process.env.WINGBITS_API_KEY;
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
}
if (!apiKey) {
return Response.json({ error: 'Wingbits not configured', configured: false }, {
headers: corsHeaders,
});
}
if (!icao24 || !/^[a-f0-9]+$/i.test(icao24)) {
return Response.json({ error: 'Invalid icao24' }, { status: 400, headers: corsHeaders });
}
try {
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
headers: {
'x-api-key': apiKey,
'Accept': 'application/json',
},
});
if (!response.ok) {
return Response.json({ error: `Wingbits API error: ${response.status}`, icao24 }, {
status: response.status,
headers: corsHeaders,
});
}
const data = await response.json();
return Response.json(data, {
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600' },
});
} catch (error) {
return Response.json({ error: error.message, icao24 }, { status: 500, headers: corsHeaders });
}
}
-74
View File
@@ -1,74 +0,0 @@
// Wingbits batch aircraft details
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
export const config = { runtime: 'edge' };
export default async function handler(req) {
const apiKey = process.env.WINGBITS_API_KEY;
const corsHeaders = getCorsHeaders(req, 'POST, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) {
return new Response(null, { status: 403, headers: corsHeaders });
}
return new Response(null, { status: 204, headers: corsHeaders });
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
}
if (req.method !== 'POST') {
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
}
if (!apiKey) {
return Response.json({ error: 'Wingbits not configured', configured: false }, {
headers: corsHeaders,
});
}
try {
const body = await req.json();
const icao24List = body.icao24s || [];
if (!Array.isArray(icao24List) || icao24List.length === 0) {
return Response.json({ error: 'icao24s array required' }, { status: 400, headers: corsHeaders });
}
// Limit batch size
const limitedList = icao24List.slice(0, 20).map(id => id.toLowerCase());
const results = {};
// Fetch all in parallel
const fetchPromises = limitedList.map(async (icao24) => {
try {
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
headers: {
'x-api-key': apiKey,
'Accept': 'application/json',
},
});
if (response.ok) {
return { icao24, data: await response.json() };
}
} catch {
// Skip failed lookups
}
return null;
});
const fetchResults = await Promise.all(fetchPromises);
for (const result of fetchResults) {
if (result) results[result.icao24] = result.data;
}
return Response.json({
results,
fetched: Object.keys(results).length,
requested: limitedList.length,
}, { headers: { 'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', ...corsHeaders } });
} catch (error) {
return Response.json({ error: error.message }, { status: 500, headers: corsHeaders });
}
}
-145
View File
@@ -1,145 +0,0 @@
// World Bank API proxy (Web API handler for Edge + sidecar compatibility)
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
export const config = {
runtime: 'edge',
};
const TECH_INDICATORS = {
'IT.NET.USER.ZS': 'Internet Users (% of population)',
'IT.CEL.SETS.P2': 'Mobile Subscriptions (per 100 people)',
'IT.NET.BBND.P2': 'Fixed Broadband Subscriptions (per 100 people)',
'IT.NET.SECR.P6': 'Secure Internet Servers (per million people)',
'GB.XPD.RSDV.GD.ZS': 'R&D Expenditure (% of GDP)',
'IP.PAT.RESD': 'Patent Applications (residents)',
'IP.PAT.NRES': 'Patent Applications (non-residents)',
'IP.TMK.TOTL': 'Trademark Applications',
'TX.VAL.TECH.MF.ZS': 'High-Tech Exports (% of manufactured exports)',
'BX.GSR.CCIS.ZS': 'ICT Service Exports (% of service exports)',
'TM.VAL.ICTG.ZS.UN': 'ICT Goods Imports (% of total goods imports)',
'SE.TER.ENRR': 'Tertiary Education Enrollment (%)',
'SE.XPD.TOTL.GD.ZS': 'Education Expenditure (% of GDP)',
'NY.GDP.MKTP.KD.ZG': 'GDP Growth (annual %)',
'NY.GDP.PCAP.CD': 'GDP per Capita (current US$)',
'NE.EXP.GNFS.ZS': 'Exports of Goods & Services (% of GDP)',
};
const TECH_COUNTRIES = [
'USA', 'CHN', 'JPN', 'DEU', 'KOR', 'GBR', 'IND', 'ISR', 'SGP', 'TWN',
'FRA', 'CAN', 'SWE', 'NLD', 'CHE', 'FIN', 'IRL', 'AUS', 'BRA', 'IDN',
'ARE', 'SAU', 'QAT', 'BHR', 'EGY', 'TUR',
'MYS', 'THA', 'VNM', 'PHL',
'ESP', 'ITA', 'POL', 'CZE', 'DNK', 'NOR', 'AUT', 'BEL', 'PRT', 'EST',
'MEX', 'ARG', 'CHL', 'COL',
'ZAF', 'NGA', 'KEN',
];
export default async function handler(request) {
const CORS = getCorsHeaders(request);
if (isDisallowedOrigin(request)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: CORS });
}
function json(data, status = 200, extra = {}) {
return new Response(JSON.stringify(data), {
status,
headers: { 'Content-Type': 'application/json', ...CORS, ...extra },
});
}
if (request.method === 'OPTIONS') {
return new Response(null, { status: 204, headers: CORS });
}
const url = new URL(request.url);
const indicator = url.searchParams.get('indicator');
const country = url.searchParams.get('country');
const countries = url.searchParams.get('countries');
const years = url.searchParams.get('years') || '5';
const action = url.searchParams.get('action');
if (action === 'indicators') {
return json({ indicators: TECH_INDICATORS, defaultCountries: TECH_COUNTRIES }, 200, { 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600' });
}
if (!indicator) {
return json({ error: 'Missing indicator parameter', availableIndicators: Object.keys(TECH_INDICATORS) }, 400);
}
try {
let countryList = country || countries || TECH_COUNTRIES.join(';');
if (countries) {
countryList = countries.split(',').join(';');
}
const currentYear = new Date().getFullYear();
const startYear = currentYear - parseInt(years);
const wbUrl = `https://api.worldbank.org/v2/country/${countryList}/indicator/${indicator}?format=json&date=${startYear}:${currentYear}&per_page=1000`;
const response = await fetch(wbUrl, {
headers: {
'Accept': 'application/json',
'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0; +https://worldmonitor.app)',
},
});
if (!response.ok) {
throw new Error(`World Bank API error: ${response.status}`);
}
const data = await response.json();
if (!data || !Array.isArray(data) || data.length < 2 || !data[1]) {
return json({
indicator,
indicatorName: TECH_INDICATORS[indicator] || indicator,
metadata: { page: 1, pages: 1, total: 0 },
byCountry: {},
latestByCountry: {},
timeSeries: [],
}, 200, { 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' });
}
const [metadata, records] = data;
const transformed = {
indicator,
indicatorName: TECH_INDICATORS[indicator] || (records[0]?.indicator?.value || indicator),
metadata: { page: metadata.page, pages: metadata.pages, total: metadata.total },
byCountry: {},
latestByCountry: {},
timeSeries: [],
};
for (const record of records || []) {
const countryCode = record.countryiso3code || record.country?.id;
const countryName = record.country?.value;
const year = record.date;
const value = record.value;
if (!countryCode || value === null) continue;
if (!transformed.byCountry[countryCode]) {
transformed.byCountry[countryCode] = { code: countryCode, name: countryName, values: [] };
}
transformed.byCountry[countryCode].values.push({ year, value });
if (!transformed.latestByCountry[countryCode] || year > transformed.latestByCountry[countryCode].year) {
transformed.latestByCountry[countryCode] = { code: countryCode, name: countryName, year, value };
}
transformed.timeSeries.push({ countryCode, countryName, year, value });
}
for (const c of Object.values(transformed.byCountry)) {
c.values.sort((a, b) => a.year - b.year);
}
transformed.timeSeries.sort((a, b) => b.year - a.year || a.countryCode.localeCompare(b.countryCode));
return json(transformed, 200, { 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' });
} catch (error) {
return json({ error: error.message, indicator }, 500);
}
}
-171
View File
@@ -1,171 +0,0 @@
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
import { recordCacheTelemetry } from './_cache-telemetry.js';
import { createIpRateLimiter } from './_ip-rate-limit.js';
export const config = { runtime: 'edge' };
const COUNTRIES_CACHE_KEY = 'worldpop:countries:v1';
const COUNTRIES_TTL_SECONDS = 7 * 24 * 60 * 60;
const COUNTRIES_TTL_MS = COUNTRIES_TTL_SECONDS * 1000;
const EXPOSURE_TTL_SECONDS = 24 * 60 * 60;
let countriesFallback = { data: null, timestamp: 0 };
const rateLimiter = createIpRateLimiter({
limit: 30,
windowMs: 60 * 1000,
maxEntries: 5000,
});
function getClientIp(req) {
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
req.headers.get('x-real-ip') ||
'unknown';
}
function toErrorMessage(error) {
if (error instanceof Error) return error.message;
return String(error || 'unknown error');
}
const PRIORITY_COUNTRIES = {
UKR: { name: 'Ukraine', pop: 37000000, area: 603550 },
RUS: { name: 'Russia', pop: 144100000, area: 17098242 },
ISR: { name: 'Israel', pop: 9800000, area: 22072 },
PSE: { name: 'Palestine', pop: 5400000, area: 6020 },
SYR: { name: 'Syria', pop: 22100000, area: 185180 },
IRN: { name: 'Iran', pop: 88600000, area: 1648195 },
TWN: { name: 'Taiwan', pop: 23600000, area: 36193 },
ETH: { name: 'Ethiopia', pop: 126500000, area: 1104300 },
SDN: { name: 'Sudan', pop: 48100000, area: 1861484 },
SSD: { name: 'South Sudan', pop: 11400000, area: 619745 },
SOM: { name: 'Somalia', pop: 18100000, area: 637657 },
YEM: { name: 'Yemen', pop: 34400000, area: 527968 },
AFG: { name: 'Afghanistan', pop: 42200000, area: 652230 },
PAK: { name: 'Pakistan', pop: 240500000, area: 881913 },
IND: { name: 'India', pop: 1428600000, area: 3287263 },
MMR: { name: 'Myanmar', pop: 54200000, area: 676578 },
COD: { name: 'DR Congo', pop: 102300000, area: 2344858 },
NGA: { name: 'Nigeria', pop: 223800000, area: 923768 },
MLI: { name: 'Mali', pop: 22600000, area: 1240192 },
BFA: { name: 'Burkina Faso', pop: 22700000, area: 274200 },
};
function isValidCountries(data) {
return Boolean(data && typeof data === 'object' && Array.isArray(data.countries));
}
async function handleCountries(corsHeaders, now) {
const cached = await getCachedJson(COUNTRIES_CACHE_KEY);
if (isValidCountries(cached)) {
recordCacheTelemetry('/api/worldpop-exposure?countries', 'REDIS-HIT');
return Response.json(cached, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'REDIS-HIT' },
});
}
if (isValidCountries(countriesFallback.data) && now - countriesFallback.timestamp < COUNTRIES_TTL_MS) {
recordCacheTelemetry('/api/worldpop-exposure?countries', 'MEMORY-HIT');
return Response.json(countriesFallback.data, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'MEMORY-HIT' },
});
}
const countries = Object.entries(PRIORITY_COUNTRIES).map(([code, info]) => ({
code,
name: info.name,
population: info.pop,
densityPerKm2: Math.round(info.pop / info.area),
}));
const result = { success: true, countries, cached_at: new Date().toISOString() };
countriesFallback = { data: result, timestamp: now };
void setCachedJson(COUNTRIES_CACHE_KEY, result, COUNTRIES_TTL_SECONDS);
recordCacheTelemetry('/api/worldpop-exposure?countries', 'MISS');
return Response.json(result, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'MISS' },
});
}
function handleExposure(corsHeaders, lat, lon, radius) {
let bestMatch = null;
let bestDist = Infinity;
const CENTROIDS = {
UKR: [48.4, 31.2], RUS: [61.5, 105.3], ISR: [31.0, 34.8], PSE: [31.9, 35.2],
SYR: [35.0, 38.0], IRN: [32.4, 53.7], TWN: [23.7, 121.0], ETH: [9.1, 40.5],
SDN: [15.5, 32.5], SSD: [6.9, 31.3], SOM: [5.2, 46.2], YEM: [15.6, 48.5],
AFG: [33.9, 67.7], PAK: [30.4, 69.3], IND: [20.6, 79.0], MMR: [19.8, 96.7],
COD: [-4.0, 21.8], NGA: [9.1, 7.5], MLI: [17.6, -4.0], BFA: [12.3, -1.6],
};
for (const [code, [cLat, cLon]] of Object.entries(CENTROIDS)) {
const dist = Math.sqrt(Math.pow(lat - cLat, 2) + Math.pow(lon - cLon, 2));
if (dist < bestDist) {
bestDist = dist;
bestMatch = code;
}
}
const info = PRIORITY_COUNTRIES[bestMatch] || { pop: 50000000, area: 500000 };
const density = info.pop / info.area;
const areaKm2 = Math.PI * radius * radius;
const exposed = Math.round(density * areaKm2);
return Response.json({
success: true,
exposedPopulation: exposed,
exposureRadiusKm: radius,
nearestCountry: bestMatch,
densityPerKm2: Math.round(density),
}, {
status: 200,
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
});
}
export default async function handler(req) {
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
if (req.method === 'OPTIONS') {
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
return new Response(null, { status: 204, headers: corsHeaders });
}
if (req.method !== 'GET') {
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
}
if (isDisallowedOrigin(req)) {
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
}
const ip = getClientIp(req);
if (!rateLimiter.check(ip)) {
return Response.json({ error: 'Rate limited' }, {
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
});
}
const url = new URL(req.url);
const mode = url.searchParams.get('mode') || 'countries';
if (mode === 'exposure') {
const lat = Number(url.searchParams.get('lat'));
const lon = Number(url.searchParams.get('lon'));
const radius = Number(url.searchParams.get('radius')) || 50;
if (isNaN(lat) || isNaN(lon)) {
return Response.json({ error: 'lat and lon required' }, { status: 400, headers: corsHeaders });
}
return handleExposure(corsHeaders, lat, lon, radius);
}
return handleCountries(corsHeaders, Date.now());
}
-54
View File
@@ -1,54 +0,0 @@
export const config = { runtime: 'edge' };
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
const SYMBOL_PATTERN = /^[A-Za-z0-9.^=\-]+$/;
const MAX_SYMBOL_LENGTH = 20;
function validateSymbol(symbol) {
if (!symbol) return null;
const trimmed = symbol.trim().toUpperCase();
if (trimmed.length > MAX_SYMBOL_LENGTH) return null;
if (!SYMBOL_PATTERN.test(trimmed)) return null;
return trimmed;
}
export default async function handler(req) {
const cors = getCorsHeaders(req);
if (isDisallowedOrigin(req)) {
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
}
const url = new URL(req.url);
const symbol = validateSymbol(url.searchParams.get('symbol'));
if (!symbol) {
return new Response(JSON.stringify({ error: 'Invalid or missing symbol parameter' }), {
status: 400,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
try {
const yahooUrl = `https://query1.finance.yahoo.com/v8/finance/chart/${encodeURIComponent(symbol)}`;
const response = await fetch(yahooUrl, {
headers: {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
},
});
const data = await response.text();
return new Response(data, {
status: response.status,
headers: {
'Content-Type': 'application/json',
...cors,
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
},
});
} catch (error) {
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
status: 500,
headers: { 'Content-Type': 'application/json', ...cors },
});
}
}
+12 -14
View File
@@ -44,22 +44,20 @@ export default async function handler(request) {
const html = await response.text();
// Extract video ID from the page
const videoIdMatch = html.match(/"videoId":"([a-zA-Z0-9_-]{11})"/);
const isLiveMatch = html.match(/"isLive":\s*true/);
if (videoIdMatch && isLiveMatch) {
return new Response(JSON.stringify({ videoId: videoIdMatch[1], isLive: true }), {
status: 200,
headers: {
'Content-Type': 'application/json',
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', // Cache for 5 minutes
},
});
// Scope both fields to the same videoDetails block so we don't
// combine a videoId from one object with isLive from another.
let videoId = null;
const detailsIdx = html.indexOf('"videoDetails"');
if (detailsIdx !== -1) {
const block = html.substring(detailsIdx, detailsIdx + 5000);
const vidMatch = block.match(/"videoId":"([a-zA-Z0-9_-]{11})"/);
const liveMatch = block.match(/"isLive"\s*:\s*true/);
if (vidMatch && liveMatch) {
videoId = vidMatch[1];
}
}
// Return null if no live stream found
return new Response(JSON.stringify({ videoId: null, isLive: false }), {
return new Response(JSON.stringify({ videoId, isLive: videoId !== null }), {
status: 200,
headers: {
'Content-Type': 'application/json',
+32
View File
@@ -0,0 +1,32 @@
import { mutation } from "./_generated/server";
import { v } from "convex/values";
export const register = mutation({
args: {
email: v.string(),
source: v.optional(v.string()),
appVersion: v.optional(v.string()),
},
handler: async (ctx, args) => {
const normalizedEmail = args.email.trim().toLowerCase();
const existing = await ctx.db
.query("registrations")
.withIndex("by_normalized_email", (q) => q.eq("normalizedEmail", normalizedEmail))
.first();
if (existing) {
return { status: "already_registered" as const };
}
await ctx.db.insert("registrations", {
email: args.email.trim(),
normalizedEmail,
registeredAt: Date.now(),
source: args.source ?? "unknown",
appVersion: args.appVersion ?? "unknown",
});
return { status: "registered" as const };
},
});
+12
View File
@@ -0,0 +1,12 @@
import { defineSchema, defineTable } from "convex/server";
import { v } from "convex/values";
export default defineSchema({
registrations: defineTable({
email: v.string(),
normalizedEmail: v.string(),
registeredAt: v.number(),
source: v.optional(v.string()),
appVersion: v.optional(v.string()),
}).index("by_normalized_email", ["normalizedEmail"]),
});
+14
View File
@@ -0,0 +1,14 @@
{
"compilerOptions": {
"target": "ESNext",
"lib": ["ES2021"],
"module": "ESNext",
"moduleResolution": "Bundler",
"strict": true,
"skipLibCheck": true,
"allowJs": true,
"outDir": "./_generated"
},
"include": ["./**/*.ts"],
"exclude": ["./_generated"]
}
+34
View File
@@ -0,0 +1,34 @@
# Nginx API proxy compression baseline for WorldMonitor.
# Requires ngx_brotli (or Nginx Plus Brotli module) to be installed.
# Prefer Brotli for HTTPS clients and keep gzip as fallback.
brotli on;
brotli_comp_level 5;
brotli_min_length 1024;
brotli_types application/json application/javascript text/css text/plain application/xml text/xml;
gzip on;
gzip_comp_level 5;
gzip_min_length 1024;
gzip_vary on;
gzip_proxied any;
gzip_types application/json application/javascript text/css text/plain application/xml text/xml;
server {
listen 443 ssl;
server_name api.worldmonitor.local;
location /api/ {
proxy_pass http://127.0.0.1:8787;
proxy_http_version 1.1;
# Preserve upstream compression behavior and pass through client preferences.
proxy_set_header Accept-Encoding $http_accept_encoding;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# If upstream sends pre-compressed content, do not decompress.
gunzip off;
}
}
+461
View File
@@ -0,0 +1,461 @@
# Adding API Endpoints
All JSON API endpoints in WorldMonitor **must** use sebuf. Do not create standalone `api/*.js` files — the legacy pattern is deprecated and being removed.
This guide walks through adding a new RPC to an existing service and adding an entirely new service.
> **Important:** After modifying any `.proto` file, you **must** run `make generate` before building or pushing. The generated TypeScript files in `src/generated/` are checked into the repo and must stay in sync with the proto definitions. CI does not run generation yet — this is your responsibility until we add it to the pipeline (see [#200](https://github.com/koala73/worldmonitor/issues/200)).
## Prerequisites
You need **Go 1.21+** and **Node.js 18+** installed. Everything else is installed automatically:
```bash
make install # one-time: installs buf, sebuf plugins, npm deps, proto deps
```
This installs:
- **buf** — proto linting, dependency management, and code generation orchestrator
- **protoc-gen-ts-client** — generates TypeScript client classes (from [sebuf](https://github.com/SebastienMelki/sebuf))
- **protoc-gen-ts-server** — generates TypeScript server handler interfaces (from sebuf)
- **protoc-gen-openapiv3** — generates OpenAPI v3 specs (from sebuf)
- **npm dependencies** — all Node.js packages
Run code generation from the repo root:
```bash
make generate # regenerate all TypeScript + OpenAPI from protos
```
This produces three outputs per service:
- `src/generated/client/{domain}/v1/service_client.ts` — typed fetch client for the frontend
- `src/generated/server/{domain}/v1/service_server.ts` — handler interface + route factory for the backend
- `docs/api/{Domain}Service.openapi.yaml` + `.json` — OpenAPI v3 documentation
## Adding an RPC to an existing service
Example: adding `GetEarthquakeDetails` to `SeismologyService`.
### 1. Define the request/response messages
Create `proto/worldmonitor/seismology/v1/get_earthquake_details.proto`:
```protobuf
syntax = "proto3";
package worldmonitor.seismology.v1;
import "buf/validate/validate.proto";
import "worldmonitor/seismology/v1/earthquake.proto";
// GetEarthquakeDetailsRequest specifies which earthquake to retrieve.
message GetEarthquakeDetailsRequest {
// USGS event identifier (e.g., "us7000abcd").
string earthquake_id = 1 [
(buf.validate.field).required = true,
(buf.validate.field).string.min_len = 1,
(buf.validate.field).string.max_len = 100
];
}
// GetEarthquakeDetailsResponse contains the full earthquake record.
message GetEarthquakeDetailsResponse {
// The earthquake matching the requested ID.
Earthquake earthquake = 1;
}
```
### 2. Add the RPC to the service definition
Edit `proto/worldmonitor/seismology/v1/service.proto`:
```protobuf
import "worldmonitor/seismology/v1/get_earthquake_details.proto";
service SeismologyService {
// ... existing RPCs ...
// GetEarthquakeDetails retrieves a single earthquake by its USGS event ID.
rpc GetEarthquakeDetails(GetEarthquakeDetailsRequest) returns (GetEarthquakeDetailsResponse) {
option (sebuf.http.config) = {path: "/get-earthquake-details"};
}
}
```
### 3. Lint and generate
```bash
make check # lint + generate in one step
```
At this point, `npx tsc --noEmit` will **fail** because the handler doesn't implement the new method yet. This is by design — the compiler enforces the contract.
### 4. Implement the handler
Create `server/worldmonitor/seismology/v1/get-earthquake-details.ts`:
```typescript
import type {
SeismologyServiceHandler,
ServerContext,
GetEarthquakeDetailsRequest,
GetEarthquakeDetailsResponse,
} from '../../../../src/generated/server/worldmonitor/seismology/v1/service_server';
export const getEarthquakeDetails: SeismologyServiceHandler['getEarthquakeDetails'] = async (
_ctx: ServerContext,
req: GetEarthquakeDetailsRequest,
): Promise<GetEarthquakeDetailsResponse> => {
const response = await fetch(
`https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/${req.earthquakeId}.geojson`,
);
if (!response.ok) {
throw new Error(`USGS API error: ${response.status}`);
}
const f: any = await response.json();
return {
earthquake: {
id: f.id,
place: f.properties.place || '',
magnitude: f.properties.mag ?? 0,
depthKm: f.geometry.coordinates[2] ?? 0,
location: {
latitude: f.geometry.coordinates[1],
longitude: f.geometry.coordinates[0],
},
occurredAt: f.properties.time,
sourceUrl: f.properties.url || '',
},
};
};
```
### 5. Wire it into the handler re-export
Edit `server/worldmonitor/seismology/v1/handler.ts`:
```typescript
import type { SeismologyServiceHandler } from '../../../../src/generated/server/worldmonitor/seismology/v1/service_server';
import { listEarthquakes } from './list-earthquakes';
import { getEarthquakeDetails } from './get-earthquake-details';
export const seismologyHandler: SeismologyServiceHandler = {
listEarthquakes,
getEarthquakeDetails,
};
```
### 6. Verify
```bash
npx tsc --noEmit # should pass with zero errors
```
The route is already live. `createSeismologyServiceRoutes()` picks up the new RPC automatically — no changes needed to `api/[[...path]].ts` or `vite.config.ts`.
### 7. Check the generated docs
Open `docs/api/SeismologyService.openapi.yaml` — the new endpoint should appear with all validation constraints from your proto annotations.
## Adding a new service
Example: adding a `SanctionsService`.
### 1. Create the proto directory
```
proto/worldmonitor/sanctions/v1/
```
### 2. Define entity messages
Create `proto/worldmonitor/sanctions/v1/sanctions_entry.proto`:
```protobuf
syntax = "proto3";
package worldmonitor.sanctions.v1;
import "buf/validate/validate.proto";
import "sebuf/http/annotations.proto";
// SanctionsEntry represents a single entity on a sanctions list.
message SanctionsEntry {
// Unique identifier.
string id = 1 [
(buf.validate.field).required = true,
(buf.validate.field).string.min_len = 1
];
// Name of the sanctioned entity or individual.
string name = 2;
// Issuing authority (e.g., "OFAC", "EU", "UN").
string authority = 3;
// ISO 3166-1 alpha-2 country code of the target.
string country_code = 4;
// Date the sanction was imposed, as Unix epoch milliseconds.
int64 imposed_at = 5 [(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER];
}
```
### 3. Define request/response messages
Create `proto/worldmonitor/sanctions/v1/list_sanctions.proto`:
```protobuf
syntax = "proto3";
package worldmonitor.sanctions.v1;
import "buf/validate/validate.proto";
import "worldmonitor/core/v1/pagination.proto";
import "worldmonitor/sanctions/v1/sanctions_entry.proto";
// ListSanctionsRequest specifies filters for sanctions data.
message ListSanctionsRequest {
// Filter by issuing authority (e.g., "OFAC"). Empty returns all.
string authority = 1;
// Filter by country code.
string country_code = 2 [(buf.validate.field).string.max_len = 2];
// Pagination parameters.
worldmonitor.core.v1.PaginationRequest pagination = 3;
}
// ListSanctionsResponse contains the matching sanctions entries.
message ListSanctionsResponse {
// The list of sanctions entries.
repeated SanctionsEntry entries = 1;
// Pagination metadata.
worldmonitor.core.v1.PaginationResponse pagination = 2;
}
```
### 4. Define the service
Create `proto/worldmonitor/sanctions/v1/service.proto`:
```protobuf
syntax = "proto3";
package worldmonitor.sanctions.v1;
import "sebuf/http/annotations.proto";
import "worldmonitor/sanctions/v1/list_sanctions.proto";
// SanctionsService provides APIs for international sanctions monitoring.
service SanctionsService {
option (sebuf.http.service_config) = {base_path: "/api/sanctions/v1"};
// ListSanctions retrieves sanctions entries matching the given filters.
rpc ListSanctions(ListSanctionsRequest) returns (ListSanctionsResponse) {
option (sebuf.http.config) = {path: "/list-sanctions"};
}
}
```
### 5. Generate
```bash
make check # lint + generate in one step
```
### 6. Implement the handler
Create the handler directory and files:
```
server/worldmonitor/sanctions/v1/
├── handler.ts # thin re-export
└── list-sanctions.ts # RPC implementation
```
`server/worldmonitor/sanctions/v1/list-sanctions.ts`:
```typescript
import type {
SanctionsServiceHandler,
ServerContext,
ListSanctionsRequest,
ListSanctionsResponse,
} from '../../../../src/generated/server/worldmonitor/sanctions/v1/service_server';
export const listSanctions: SanctionsServiceHandler['listSanctions'] = async (
_ctx: ServerContext,
req: ListSanctionsRequest,
): Promise<ListSanctionsResponse> => {
// Your implementation here — fetch from upstream API, transform to proto shape
return { entries: [], pagination: undefined };
};
```
`server/worldmonitor/sanctions/v1/handler.ts`:
```typescript
import type { SanctionsServiceHandler } from '../../../../src/generated/server/worldmonitor/sanctions/v1/service_server';
import { listSanctions } from './list-sanctions';
export const sanctionsHandler: SanctionsServiceHandler = {
listSanctions,
};
```
### 7. Register the service in the gateway
Edit `api/[[...path]].js` — add the import and mount the routes:
```typescript
import { createSanctionsServiceRoutes } from '../src/generated/server/worldmonitor/sanctions/v1/service_server';
import { sanctionsHandler } from './server/worldmonitor/sanctions/v1/handler';
const allRoutes = [
// ... existing routes ...
...createSanctionsServiceRoutes(sanctionsHandler, serverOptions),
];
```
### 8. Register in the Vite dev server
Edit `vite.config.ts` — add the lazy import and route mount inside the `sebufApiPlugin()` function. Follow the existing pattern (search for any other service to see the exact locations).
### 9. Create the frontend service wrapper
Create `src/services/sanctions.ts`:
```typescript
import {
SanctionsServiceClient,
type SanctionsEntry,
type ListSanctionsResponse,
} from '@/generated/client/worldmonitor/sanctions/v1/service_client';
import { createCircuitBreaker } from '@/utils';
export type { SanctionsEntry };
const client = new SanctionsServiceClient('', { fetch: fetch.bind(globalThis) });
const breaker = createCircuitBreaker<ListSanctionsResponse>({ name: 'Sanctions' });
const emptyFallback: ListSanctionsResponse = { entries: [] };
export async function fetchSanctions(authority?: string): Promise<SanctionsEntry[]> {
const response = await breaker.execute(async () => {
return client.listSanctions({ authority: authority ?? '', countryCode: '', pagination: undefined });
}, emptyFallback);
return response.entries;
}
```
### 10. Verify
```bash
npx tsc --noEmit # zero errors
```
## Proto conventions
These conventions are enforced across the codebase. Follow them for consistency.
### File naming
- One file per message type: `earthquake.proto`, `sanctions_entry.proto`
- One file per RPC pair: `list_earthquakes.proto`, `get_earthquake_details.proto`
- Service definition: `service.proto`
- Use `snake_case` for file names and field names
### Time fields
Always use `int64` with Unix epoch milliseconds. Never use `google.protobuf.Timestamp`.
Always add the `INT64_ENCODING_NUMBER` annotation so TypeScript gets `number` instead of `string`:
```protobuf
int64 occurred_at = 6 [(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER];
```
### Validation annotations
Import `buf/validate/validate.proto` and annotate fields at the proto level. These constraints flow through to the generated OpenAPI spec automatically.
Common patterns:
```protobuf
// Required string with length bounds
string id = 1 [
(buf.validate.field).required = true,
(buf.validate.field).string.min_len = 1,
(buf.validate.field).string.max_len = 100
];
// Numeric range (e.g., score 0-100)
double risk_score = 2 [
(buf.validate.field).double.gte = 0,
(buf.validate.field).double.lte = 100
];
// Non-negative value
double min_magnitude = 3 [(buf.validate.field).double.gte = 0];
// Coordinate bounds (prefer using core.v1.GeoCoordinates instead)
double latitude = 1 [
(buf.validate.field).double.gte = -90,
(buf.validate.field).double.lte = 90
];
```
### Shared core types
Reuse these instead of redefining:
| Type | Import | Use for |
|------|--------|---------|
| `GeoCoordinates` | `worldmonitor/core/v1/geo.proto` | Any lat/lon location (has built-in -90/90 and -180/180 bounds) |
| `BoundingBox` | `worldmonitor/core/v1/geo.proto` | Spatial filtering |
| `TimeRange` | `worldmonitor/core/v1/time.proto` | Time-based filtering (has `INT64_ENCODING_NUMBER`) |
| `PaginationRequest` | `worldmonitor/core/v1/pagination.proto` | Request pagination (has page_size 1-100 constraint) |
| `PaginationResponse` | `worldmonitor/core/v1/pagination.proto` | Response pagination metadata |
### Comments
buf lint enforces comments on all messages, fields, services, RPCs, and enum values. Every proto element must have a `//` comment. This is not optional — `buf lint` will fail without them.
### Route paths
- Service base path: `/api/{domain}/v1`
- RPC path: `/{verb}-{noun}` in kebab-case (e.g., `/list-earthquakes`, `/get-vessel-snapshot`)
### Handler typing
Always type the handler function against the generated interface using indexed access:
```typescript
export const listSanctions: SanctionsServiceHandler['listSanctions'] = async (
_ctx: ServerContext,
req: ListSanctionsRequest,
): Promise<ListSanctionsResponse> => {
// ...
};
```
This ensures the compiler catches any mismatch between your implementation and the proto contract.
### Client construction
Always pass `{ fetch: fetch.bind(globalThis) }` when creating clients:
```typescript
const client = new SanctionsServiceClient('', { fetch: fetch.bind(globalThis) });
```
The empty string base URL works because both Vite dev server and Vercel serve the API on the same origin. The `fetch.bind(globalThis)` is required for Tauri compatibility.
## Generated documentation
Every time you run `make generate`, OpenAPI v3 specs are generated for each service:
- `docs/api/{Domain}Service.openapi.yaml` — human-readable YAML
- `docs/api/{Domain}Service.openapi.json` — machine-readable JSON
These specs include:
- All endpoints with request/response schemas
- Validation constraints from `buf.validate` annotations (min/max, required fields, ranges)
- Field descriptions from proto comments
- Error response schemas (400 validation errors, 500 server errors)
You do not need to write or maintain OpenAPI specs by hand. They are generated artifacts. If you need to change the API documentation, change the proto and regenerate.
+140
View File
@@ -0,0 +1,140 @@
# API Key Gating & Registration — Deployment Guide
## Overview
Desktop cloud fallback is gated on a `WORLDMONITOR_API_KEY`. Without a valid key, the desktop app operates local-only (sidecar). A registration form collects emails via Convex DB for future key distribution.
## Architecture
```
Desktop App Cloud (Vercel)
┌──────────────────┐ ┌──────────────────────┐
│ fetch('/api/...')│ │ api/[domain]/v1/[rpc]│
│ │ │ │ │ │
│ ┌──────▼───────┐ │ │ ┌──────▼───────┐ │
│ │ sidecar try │ │ │ │ validateApiKey│ │
│ │ (local-first)│ │ │ │ (origin-aware)│ │
│ └──────┬───────┘ │ │ └──────┬───────┘ │
│ fail │ │ │ 401 if invalid │
│ ┌──────▼───────┐ │ fallback │ │
│ │ WM key check │─┼──────────────►│ ┌──────────────┐ │
│ │ (gate) │ │ +header │ │ route handler │ │
│ └──────────────┘ │ │ └──────────────┘ │
└──────────────────┘ └──────────────────────┘
```
## Required Environment Variables
### Vercel
| Variable | Description | Example |
|----------|-------------|---------|
| `WORLDMONITOR_VALID_KEYS` | Comma-separated list of valid API keys | `wm_abc123def456,wm_xyz789` |
| `CONVEX_URL` | Convex deployment URL (from `npx convex deploy`) | `https://xyz-123.convex.cloud` |
### Generating API keys
Keys must be at least 16 characters (validated client-side). Recommended format:
```bash
# Generate a key
openssl rand -hex 24 | sed 's/^/wm_/'
# Example output: wm_a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6
```
Add to `WORLDMONITOR_VALID_KEYS` in Vercel dashboard (comma-separated, no spaces).
## Convex Setup
### First-time deployment
```bash
# 1. Install (already in package.json)
npm install
# 2. Login to Convex
npx convex login
# 3. Initialize project (creates .env.local with CONVEX_URL)
npx convex init
# 4. Deploy schema and functions
npx convex deploy
# 5. Copy the deployment URL to Vercel env vars
# The URL is printed by `npx convex deploy` and saved in .env.local
```
### Verify Convex deployment
```bash
# Typecheck Convex functions
npx convex dev --typecheck
# Open Convex dashboard to see registrations
npx convex dashboard
```
### Schema
The `registrations` table stores:
| Field | Type | Description |
|-------|------|-------------|
| `email` | string | Original email (for display) |
| `normalizedEmail` | string | Lowercased email (for dedup) |
| `registeredAt` | number | Unix timestamp |
| `source` | string? | Where the registration came from |
| `appVersion` | string? | Desktop app version |
Indexed by `normalizedEmail` for duplicate detection.
## Security Model
### Client-side (desktop app)
- `installRuntimeFetchPatch()` checks `WORLDMONITOR_API_KEY` before allowing cloud fallback
- Key must be present AND valid (min 16 chars)
- `secretsReady` promise ensures secrets are loaded before first fetch (2s timeout)
- Fail-closed: any error in key check blocks cloud fallback
### Server-side (Vercel edge)
- `api/_api-key.js` validates `X-WorldMonitor-Key` header on sebuf routes
- **Origin-aware**: desktop origins (`tauri.localhost`, `tauri://`, `asset://`) require a key
- Web origins (`worldmonitor.app`) pass through without a key
- Non-desktop origin with key header: key is still validated
- Invalid key returns `401 { error: "Invalid API key" }`
### CORS
`X-WorldMonitor-Key` is allowed in both `server/cors.ts` and `api/_cors.js`.
## Verification Checklist
After deployment:
- [ ] Set `WORLDMONITOR_VALID_KEYS` in Vercel
- [ ] Set `CONVEX_URL` in Vercel
- [ ] Run `npx convex deploy` to push schema
- [ ] Desktop without key: cloud fallback blocked (console shows `cloud fallback blocked`)
- [ ] Desktop with invalid key: sebuf requests get `401`
- [ ] Desktop with valid key: cloud fallback works as before
- [ ] Web access: no key required, works normally
- [ ] Registration form: submit email, check Convex dashboard
- [ ] Duplicate email: shows "already registered"
- [ ] Existing settings tabs (LLMs, API Keys, Debug) unchanged
## Files Reference
| File | Role |
|------|------|
| `src/services/runtime.ts` | Client-side key gate + header attachment |
| `src/services/runtime-config.ts` | `WORLDMONITOR_API_KEY` type, validation, `secretsReady` |
| `api/_api-key.js` | Server-side key validation (origin-aware) |
| `api/[domain]/v1/[rpc].ts` | Sebuf gateway — calls `validateApiKey` |
| `api/register-interest.js` | Registration endpoint → Convex |
| `server/cors.ts` / `api/_cors.js` | CORS headers with `X-WorldMonitor-Key` |
| `src/components/WorldMonitorTab.ts` | Settings UI for key + registration |
| `convex/schema.ts` | Convex DB schema |
| `convex/registerInterest.ts` | Convex mutation |
+184
View File
@@ -0,0 +1,184 @@
# World Monitor — Community Promotion Guide
Thank you for helping spread the word about World Monitor! This guide provides talking points, must-see features, and visual suggestions to help you create compelling content for your audience.
---
## What is World Monitor?
**One-line pitch**: A free, open-source, real-time global intelligence dashboard — like Bloomberg Terminal meets OSINT, for everyone.
**Longer description**: World Monitor aggregates 150+ news feeds, military tracking, financial markets, conflict data, protest monitoring, satellite imagery, and AI-powered analysis into a single unified dashboard with an interactive globe. Available as a web app, desktop app (macOS/Windows/Linux), and installable PWA.
---
## Key URLs
| Link | Description |
|------|-------------|
| [worldmonitor.app](https://worldmonitor.app) | Main dashboard — geopolitics, military, conflicts |
| [tech.worldmonitor.app](https://tech.worldmonitor.app) | Tech variant — startups, AI/ML, cybersecurity |
| [finance.worldmonitor.app](https://finance.worldmonitor.app) | Finance variant — markets, exchanges, central banks |
| [GitHub](https://github.com/koala73/worldmonitor) | Source code (AGPL-3.0) |
---
## Must-See Features (Top 10)
### 1. Interactive Globe with 35+ Data Layers
The centerpiece. A WebGL-accelerated globe (deck.gl) with toggleable layers for conflicts, military bases, nuclear facilities, undersea cables, pipelines, satellite fires, protests, cyber threats, and more. Zoom in and the detail layers progressively reveal.
**Show**: Toggle different layers on/off. Zoom into a conflict region. Show the layer panel.
### 2. AI-Powered World Brief
One-click AI summary of the top global developments. Three-tier LLM provider chain: local Ollama/LM Studio (fully private, offline), Groq (fast cloud), or OpenRouter (fallback). Redis caching for instant responses on repeat queries.
**Show**: The summary card at the top of the news panel.
### 3. Country Intelligence Dossiers
Click any country on the map for a full-page intelligence brief: instability score ring, AI-generated analysis, top headlines, prediction markets, 7-day event timeline, active signal chips, infrastructure exposure, and stock market data.
**Show**: Click a country (e.g., Japan, Ukraine, or Iran) → full dossier page.
### 4. 14 Languages Support
Full UI in 14 languages including Japanese. Regional news feeds auto-adapt — Japanese users see NHK World, Nikkei Asia, and Japan-relevant sources. Language bundles are lazy-loaded for fast performance.
**Show**: Switch language to Japanese in the settings. Note how feeds change.
### 5. Live Military Tracking
Real-time ADS-B military flight tracking and AIS naval vessel monitoring. Strategic Posture panel shows theater-level risk assessment across 9 global regions (Baltic, Black Sea, South China Sea, Eastern Mediterranean, etc.).
**Show**: Enable the Military layer. Show the Strategic Posture panel.
### 6. Three Variant Dashboards
One codebase, three specialized views — switch between World (geopolitics), Tech (startups/AI), and Finance (markets/exchanges) with one click in the header bar.
**Show**: Click the variant switcher (🌍 WORLD | 💻 TECH | 📈 FINANCE).
### 7. Market & Crypto Intelligence
7-signal macro radar with composite BUY/CASH verdict, BTC spot ETF flow tracker, stablecoin peg monitor, Fear & Greed Index, and Bitcoin technical indicators. Sparkline charts and donut gauges for visual trends.
**Show**: Scroll to the crypto/market panels. Point out the sparklines.
### 8. Live Video & Webcam Feeds
8 live news streams (Bloomberg, Al Jazeera, Sky News, etc.) + 19 live webcams from geopolitical hotspots across 4 regions. Idle-aware — auto-pauses after 5 minutes of inactivity.
**Show**: Open the video panel or webcam panel.
### 9. Desktop Application (Free)
Native app for macOS, Windows, and Linux via Tauri. API keys stored in OS keychain (not plaintext). Local Node.js sidecar runs all 60+ API handlers offline-capable. Run local LLMs for fully private, offline AI summaries.
**Show**: The download buttons on the site, or the desktop app running natively.
### 10. Story Sharing & Social Export
Generate intelligence briefs for any country and share to Twitter/X, LinkedIn, WhatsApp, Telegram, Reddit. Includes canvas-rendered PNG images with QR codes linking back to the live dashboard.
**Show**: Generate a story for a country → share dialog with platform options.
### 11. Local LLM Support (Ollama / LM Studio)
Run AI summarization entirely on your own hardware — no API keys, no cloud, no data leaving your machine. The desktop app auto-discovers models from Ollama or LM Studio, with a three-tier fallback chain: local → Groq → OpenRouter. Settings are split into dedicated LLMs and API Keys tabs for easy configuration.
**Show**: Open Settings → LLMs tab → Ollama model dropdown auto-populated → generate a summary with the local model.
---
## Visual Content Suggestions
### Screenshots Worth Taking
1. **Full dashboard overview** — globe in center, panels on sides, news feed visible
2. **Country dossier page** — click Japan or a hotspot country, show the full brief
3. **Layer toggle demo** — before/after with conflicts + military bases enabled
4. **Finance variant** — stock exchanges, financial centers, market panels
5. **Japanese UI** — show the language switcher and Japanese interface
6. **Webcam grid** — 4 live feeds from different regions
7. **Strategic Posture** — theater risk levels panel
8. **Settings LLMs tab** — Ollama model dropdown with local models discovered
### Video/GIF Ideas
1. **30-second tour**: Open site → rotate globe → toggle layers → click country → show brief
2. **Language switch**: English → Japanese, show how feeds adapt
3. **Layer stacking**: Start empty → add conflicts → military → cyber → fires → wow
4. **Variant switching**: World → Tech → Finance in quick succession
---
## Talking Points for Posts
### For General Audience
- "An open-source Bloomberg Terminal for everyone — free, no login required"
- "150+ news sources, military tracking, AI analysis — all in one dashboard"
- "Run AI summaries locally with Ollama — your data never leaves your machine"
- "Available in Japanese with NHK and Nikkei feeds built in"
- "Native desktop app for macOS/Windows/Linux, completely free"
### For Tech Audience
- "Built with TypeScript, Vite, deck.gl, MapLibre GL, Tauri"
- "35+ WebGL data layers running at 60fps"
- "ONNX Runtime Web for browser-based ML inference (sentiment, NER, summarization)"
- "Local LLM support — plug in Ollama or LM Studio, zero cloud dependency"
- "Open source under AGPL-3.0 — contribute on GitHub"
### For Finance/OSINT Audience
- "7-signal crypto macro radar with BUY/CASH composite verdict"
- "92 global stock exchanges mapped with market caps and trading hours"
- "Country Instability Index tracking 22 nations in real-time"
- "Prediction market integration for geopolitical forecasting"
- "Air-gapped AI analysis — run Ollama locally for sensitive intelligence work"
### For Japanese Audience Specifically
- 日本語完全対応 — UI、ニュースフィード、AI要約すべて日本語で利用可能
- NHK World、日経アジアなど日本向けニュースソース内蔵
- 無料・オープンソース — アカウント登録不要
- macOS/Windows/Linux対応のデスクトップアプリあり
---
## Recent Major Features (Changelog Highlights)
| Version | Feature |
|---------|---------|
| v2.5.1 | Batch FRED fetching, parallel UCDP, partial cache TTL, bot middleware |
| v2.5.0 | Ollama/LM Studio local LLM support, settings split into LLMs + API Keys tabs, keychain vault consolidation |
| v2.4.1 | Ultra-wide layout (panels wrap around map on 2000px+ screens) |
| v2.4.0 | Live webcams from 19 geopolitical hotspots, 4 regions |
| v2.3.9 | Full i18n: 14 languages including Japanese, Arabic (RTL), Chinese |
| v2.3.8 | Finance variant with 92 exchanges, Gulf FDI investments |
| v2.3.7 | Light/dark theme system, UCDP/UNHCR/Climate panels |
| v2.3.6 | Desktop app with Tauri, OS keychain, auto-updates |
| v2.3.0 | Country Intelligence dossiers, story sharing |
---
## Branding Notes
- **Name**: "World Monitor" (two words, capitalized)
- **Tagline**: "Real-time global intelligence dashboard"
- **License**: AGPL-3.0 (free and open source)
- **Creator**: Credit "World Monitor by Elie Habib" or link to the GitHub repo
- **Variants**: You can mention all three (World/Tech/Finance) or focus on the main one
- **No login required**: Anyone can use the web app immediately — no signup, no paywall
---
## Thank You
We genuinely appreciate community members helping grow World Monitor's reach. Feel free to interpret these guidelines creatively — there's no strict template. The most compelling content comes from showing what YOU find most interesting or useful about the tool.
If you have questions or want specific screenshots/assets, open a Discussion on the GitHub repo or reach out directly.
+9 -2
View File
@@ -4,7 +4,7 @@ World Monitor desktop now uses a runtime configuration schema with per-feature t
## Secret keys
The desktop vault schema supports the following 17 keys used by services and relays:
The desktop vault schema (Rust `SUPPORTED_SECRET_KEYS`) supports the following 21 keys:
- `GROQ_API_KEY`
- `OPENROUTER_API_KEY`
@@ -18,11 +18,17 @@ The desktop vault schema supports the following 17 keys used by services and rel
- `ABUSEIPDB_API_KEY`
- `NASA_FIRMS_API_KEY`
- `WINGBITS_API_KEY`
- `WS_RELAY_URL`
- `VITE_WS_RELAY_URL`
- `VITE_OPENSKY_RELAY_URL`
- `OPENSKY_CLIENT_ID`
- `OPENSKY_CLIENT_SECRET`
- `AISSTREAM_API_KEY`
- `VITE_WS_RELAY_URL`
- `OLLAMA_API_URL`
- `OLLAMA_MODEL`
- `WORLDMONITOR_API_KEY` — gates cloud fallback access (min 16 chars)
Note: `UC_DP_KEY` exists in the TypeScript `RuntimeSecretKey` union but is not in the desktop Rust keychain or sidecar.
## Feature schema
@@ -51,3 +57,4 @@ If required secrets are missing/disabled:
- NASA FIRMS: satellite fire detection returns empty state.
- Wingbits: flight enrichment disabled, heuristic-only flight classification remains.
- AIS / OpenSky relay: live tracking features are disabled cleanly.
- WorldMonitor API key: cloud fallback is blocked; desktop operates local-only.
+31 -17
View File
@@ -7,7 +7,7 @@ AI-powered real-time global intelligence dashboard aggregating news, markets, ge
![TypeScript](https://img.shields.io/badge/TypeScript-007ACC?style=flat&logo=typescript&logoColor=white)
![Vite](https://img.shields.io/badge/Vite-646CFF?style=flat&logo=vite&logoColor=white)
![D3.js](https://img.shields.io/badge/D3.js-F9A03C?style=flat&logo=d3.js&logoColor=white)
![Version](https://img.shields.io/badge/version-2.1.4-blue)
![Version](https://img.shields.io/badge/version-2.5.1-blue)
![World Monitor Dashboard](../new-world-monitor.png)
@@ -3298,13 +3298,15 @@ const header = `World Monitor v${__APP_VERSION__}`;
## Installation
**Requirements:** Go 1.21+ and Node.js 18+.
```bash
# Clone the repository
git clone https://github.com/koala73/worldmonitor.git
cd worldmonitor
# Install dependencies
npm install
# Install everything (buf, sebuf plugins, npm deps, proto deps)
make install
# Start development server
npm run dev
@@ -3313,6 +3315,14 @@ npm run dev
npm run build
```
If you modify any `.proto` files, regenerate before building or pushing:
```bash
make generate # regenerate TypeScript clients, servers, and OpenAPI docs
```
See [ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md) for the full proto workflow.
## API Dependencies
The dashboard fetches data from various public APIs and data sources:
@@ -3642,7 +3652,7 @@ The system degrades gracefully—blocked sources are skipped while others contin
## Roadmap
See [ROADMAP.md](ROADMAP.md) for detailed planning. Recent intelligence enhancements:
See [ROADMAP.md](../.planning/ROADMAP.md) for detailed planning. Recent intelligence enhancements:
### Completed
@@ -3698,7 +3708,7 @@ See [ROADMAP.md](ROADMAP.md) for detailed planning. Recent intelligence enhancem
- **Additional Data Sources** - World Bank, IMF, OFAC sanctions, UNHCR refugee data, FAO food security
- **Think Tank Feeds** - RUSI, Chatham House, ECFR, CFR, Wilson Center, CNAS, Arms Control Association
The full [ROADMAP.md](ROADMAP.md) documents implementation details, API endpoints, and 30+ free data sources for future integration.
The full [ROADMAP.md](../.planning/ROADMAP.md) documents implementation details, API endpoints, and 30+ free data sources for future integration.
---
@@ -3994,20 +4004,24 @@ PRs that don't follow the code style or introduce security issues will be asked
### Development Tips
**Adding or Modifying API Endpoints**
All JSON API endpoints **must** use sebuf. Do not create standalone `api/*.js` files — the legacy pattern is deprecated.
See **[docs/ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md)** for the complete guide covering:
- Adding an RPC to an existing service
- Adding an entirely new service
- Proto conventions (validation, time fields, shared types)
- Generated OpenAPI documentation
**Adding a New Data Layer**
1. Create service in `src/services/` for data fetching
2. Add layer toggle in `src/components/Map.ts`
3. Add rendering logic for map markers/overlays
4. Add to help panel documentation
5. Update README with layer description
**Adding a New API Proxy**
1. Create handler in `api/` directory
2. Implement input validation (see existing proxies)
3. Add appropriate cache headers
4. Document any required environment variables
1. Define the proto contract and generate code (see [ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md))
2. Implement the handler in `server/worldmonitor/{domain}/v1/`
3. Create the frontend service wrapper in `src/services/`
4. Add layer toggle in `src/components/Map.ts`
5. Add rendering logic for map markers/overlays
**Debugging**
-47
View File
@@ -1,47 +0,0 @@
# News Translation Analysis
## Current Architecture
The application fetches news via `src/services/rss.ts`.
- **Mechanism**: Direct HTTP requests (via proxy) to RSS/Atom XML feeds.
- **Processing**: `DOMParser` parses XML client-side.
- **Storage**: Items are stored in-memory in `App.ts` (`allNews`, `newsByCategory`).
## The Challenge
Legacy RSS feeds are static XML files in their original language. There is no built-in "negotiation" for language. To display French news, we must either:
1. Fetch French feeds.
2. Translate English feeds on the fly.
## Proposed Solutions
### Option 1: Localized Feed Discovery (Recommended for "Major" Support)
Instead of forcing translation, we switch the *source* based on the selected language.
- **Implementation**:
- In `src/config/feeds.ts`, change the simple URL string to an object: `url: { en: '...', fr: '...' }` or separate constant lists `FEEDS_EN`, `FEEDS_FR`.
- **Pros**: Zero latency, native content quality, no API costs.
- **Cons**: Hard to find equivalent feeds for niche topics (e.g., specific mil-tech blogs) in all languages.
- **Strategy**: Creating a curated list of international feeds for major categories (World, Politics, Finance) is the most robust & scalable approach.
### Option 2: On-Demand Client-Side Translation
Add a "Translate" button to each news card.
- **Implementation**:
- Click triggers a call to a translation API (Google/DeepL/LLM).
- Store result in a local cache (Map).
- **Pros**: Low cost (only used when needed), preserves original context.
- **Cons**: User friction (click to read).
### Option 3: Automatic Auto-Translation (Not Recommended)
Translating 500+ headlines on every load.
- **Cons**:
- **Cost**: Prohibitive for free/low-cost APIs.
- **Latency**: Massive slowdown on startup.
- **Quality**: Short headlines often translate poorly without context.
## Recommendation
**Hybrid Approach**:
1. **Primary**: Source localized feeds where possible (e.g., Le Monde for FR, Spiegel for DE). This requires a community effort to curate `feeds.json` for each locale.
2. **Fallback**: Keep English feeds for niche tech/intel sources where no alternative exists.
3. **Feature**: Add a "Summarize & Translate" button using the existing LLM worker. The prompt to the LLM (currently used for summaries) can be adjusted to "Summarize this in [Current Language]".
## Next Steps
1. Audit `src/config/feeds.ts` to structure it for multi-language support.
2. Update `rss.ts` to select the correct URL based on `i18n.language`.
File diff suppressed because one or more lines are too long
+240
View File
@@ -0,0 +1,240 @@
openapi: 3.1.0
info:
title: AviationService API
version: 1.0.0
paths:
/api/aviation/v1/list-airport-delays:
post:
tags:
- AviationService
summary: ListAirportDelays
description: ListAirportDelays retrieves current airport delay alerts.
operationId: ListAirportDelays
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ListAirportDelaysRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ListAirportDelaysResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
components:
schemas:
Error:
type: object
properties:
message:
type: string
description: Error message (e.g., 'user not found', 'database connection failed')
description: Error is returned when a handler encounters an error. It contains a simple error message that the developer can customize.
FieldViolation:
type: object
properties:
field:
type: string
description: The field path that failed validation (e.g., 'user.email' for nested fields). For header validation, this will be the header name (e.g., 'X-API-Key')
description:
type: string
description: Human-readable description of the validation violation (e.g., 'must be a valid email address', 'required field missing')
required:
- field
- description
description: FieldViolation describes a single validation error for a specific field.
ValidationError:
type: object
properties:
violations:
type: array
items:
$ref: '#/components/schemas/FieldViolation'
description: List of validation violations
required:
- violations
description: ValidationError is returned when request validation fails. It contains a list of field violations describing what went wrong.
ListAirportDelaysRequest:
type: object
properties:
pagination:
$ref: '#/components/schemas/PaginationRequest'
region:
type: string
enum:
- AIRPORT_REGION_UNSPECIFIED
- AIRPORT_REGION_AMERICAS
- AIRPORT_REGION_EUROPE
- AIRPORT_REGION_APAC
- AIRPORT_REGION_MENA
- AIRPORT_REGION_AFRICA
description: AirportRegion represents the geographic region of an airport.
minSeverity:
type: string
enum:
- FLIGHT_DELAY_SEVERITY_UNSPECIFIED
- FLIGHT_DELAY_SEVERITY_NORMAL
- FLIGHT_DELAY_SEVERITY_MINOR
- FLIGHT_DELAY_SEVERITY_MODERATE
- FLIGHT_DELAY_SEVERITY_MAJOR
- FLIGHT_DELAY_SEVERITY_SEVERE
description: |-
FlightDelaySeverity represents the severity of flight delays at an airport.
Maps to TS union: 'normal' | 'minor' | 'moderate' | 'major' | 'severe'.
description: ListAirportDelaysRequest specifies filters for retrieving airport delay alerts.
PaginationRequest:
type: object
properties:
pageSize:
type: integer
maximum: 100
minimum: 1
format: int32
description: Maximum number of items to return per page (1 to 100).
cursor:
type: string
description: Opaque cursor for fetching the next page. Empty string for the first page.
description: PaginationRequest specifies cursor-based pagination parameters for list endpoints.
ListAirportDelaysResponse:
type: object
properties:
alerts:
type: array
items:
$ref: '#/components/schemas/AirportDelayAlert'
pagination:
$ref: '#/components/schemas/PaginationResponse'
description: ListAirportDelaysResponse contains airport delay alerts matching the request.
AirportDelayAlert:
type: object
properties:
id:
type: string
minLength: 1
description: Unique alert identifier.
iata:
type: string
description: IATA airport code (e.g., "JFK").
icao:
type: string
description: ICAO airport code (e.g., "KJFK").
name:
type: string
description: Airport name.
city:
type: string
description: City where the airport is located.
country:
type: string
description: Country code (ISO 3166-1 alpha-2).
location:
$ref: '#/components/schemas/GeoCoordinates'
region:
type: string
enum:
- AIRPORT_REGION_UNSPECIFIED
- AIRPORT_REGION_AMERICAS
- AIRPORT_REGION_EUROPE
- AIRPORT_REGION_APAC
- AIRPORT_REGION_MENA
- AIRPORT_REGION_AFRICA
description: AirportRegion represents the geographic region of an airport.
delayType:
type: string
enum:
- FLIGHT_DELAY_TYPE_UNSPECIFIED
- FLIGHT_DELAY_TYPE_GROUND_STOP
- FLIGHT_DELAY_TYPE_GROUND_DELAY
- FLIGHT_DELAY_TYPE_DEPARTURE_DELAY
- FLIGHT_DELAY_TYPE_ARRIVAL_DELAY
- FLIGHT_DELAY_TYPE_GENERAL
description: FlightDelayType represents the type of flight delay.
severity:
type: string
enum:
- FLIGHT_DELAY_SEVERITY_UNSPECIFIED
- FLIGHT_DELAY_SEVERITY_NORMAL
- FLIGHT_DELAY_SEVERITY_MINOR
- FLIGHT_DELAY_SEVERITY_MODERATE
- FLIGHT_DELAY_SEVERITY_MAJOR
- FLIGHT_DELAY_SEVERITY_SEVERE
description: |-
FlightDelaySeverity represents the severity of flight delays at an airport.
Maps to TS union: 'normal' | 'minor' | 'moderate' | 'major' | 'severe'.
avgDelayMinutes:
type: integer
format: int32
description: Average delay in minutes.
delayedFlightsPct:
type: number
format: double
description: Percentage of delayed flights.
cancelledFlights:
type: integer
format: int32
description: Number of cancelled flights.
totalFlights:
type: integer
format: int32
description: Total flights scheduled.
reason:
type: string
description: Human-readable reason for delays.
source:
type: string
enum:
- FLIGHT_DELAY_SOURCE_UNSPECIFIED
- FLIGHT_DELAY_SOURCE_FAA
- FLIGHT_DELAY_SOURCE_EUROCONTROL
- FLIGHT_DELAY_SOURCE_COMPUTED
description: FlightDelaySource represents the source of delay data.
updatedAt:
type: integer
format: int64
description: 'Last data update time, as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
required:
- id
description: |-
AirportDelayAlert represents a flight delay advisory at an airport.
Sourced from FAA and Eurocontrol.
GeoCoordinates:
type: object
properties:
latitude:
type: number
maximum: 90
minimum: -90
format: double
description: Latitude in decimal degrees (-90 to 90).
longitude:
type: number
maximum: 180
minimum: -180
format: double
description: Longitude in decimal degrees (-180 to 180).
description: GeoCoordinates represents a geographic location using WGS84 coordinates.
PaginationResponse:
type: object
properties:
nextCursor:
type: string
description: Cursor for fetching the next page. Empty string indicates no more pages.
totalCount:
type: integer
format: int32
description: Total count of items matching the query, if known. Zero if the total is unknown.
description: PaginationResponse contains pagination metadata returned alongside list results.
File diff suppressed because one or more lines are too long
+185
View File
@@ -0,0 +1,185 @@
openapi: 3.1.0
info:
title: ClimateService API
version: 1.0.0
paths:
/api/climate/v1/list-climate-anomalies:
post:
tags:
- ClimateService
summary: ListClimateAnomalies
description: ListClimateAnomalies retrieves temperature and precipitation anomalies from ERA5 data.
operationId: ListClimateAnomalies
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ListClimateAnomaliesRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ListClimateAnomaliesResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
components:
schemas:
Error:
type: object
properties:
message:
type: string
description: Error message (e.g., 'user not found', 'database connection failed')
description: Error is returned when a handler encounters an error. It contains a simple error message that the developer can customize.
FieldViolation:
type: object
properties:
field:
type: string
description: The field path that failed validation (e.g., 'user.email' for nested fields). For header validation, this will be the header name (e.g., 'X-API-Key')
description:
type: string
description: Human-readable description of the validation violation (e.g., 'must be a valid email address', 'required field missing')
required:
- field
- description
description: FieldViolation describes a single validation error for a specific field.
ValidationError:
type: object
properties:
violations:
type: array
items:
$ref: '#/components/schemas/FieldViolation'
description: List of validation violations
required:
- violations
description: ValidationError is returned when request validation fails. It contains a list of field violations describing what went wrong.
ListClimateAnomaliesRequest:
type: object
properties:
pagination:
$ref: '#/components/schemas/PaginationRequest'
minSeverity:
type: string
enum:
- ANOMALY_SEVERITY_UNSPECIFIED
- ANOMALY_SEVERITY_NORMAL
- ANOMALY_SEVERITY_MODERATE
- ANOMALY_SEVERITY_EXTREME
description: |-
AnomalySeverity represents the severity of a climate anomaly.
Maps to existing TS union: 'normal' | 'moderate' | 'extreme'.
description: ListClimateAnomaliesRequest specifies filters for retrieving climate anomaly data.
PaginationRequest:
type: object
properties:
pageSize:
type: integer
maximum: 100
minimum: 1
format: int32
description: Maximum number of items to return per page (1 to 100).
cursor:
type: string
description: Opaque cursor for fetching the next page. Empty string for the first page.
description: PaginationRequest specifies cursor-based pagination parameters for list endpoints.
ListClimateAnomaliesResponse:
type: object
properties:
anomalies:
type: array
items:
$ref: '#/components/schemas/ClimateAnomaly'
pagination:
$ref: '#/components/schemas/PaginationResponse'
description: ListClimateAnomaliesResponse contains the list of climate anomalies.
ClimateAnomaly:
type: object
properties:
zone:
type: string
minLength: 1
description: Climate zone name (e.g., "Northern Europe", "Sahel").
location:
$ref: '#/components/schemas/GeoCoordinates'
tempDelta:
type: number
format: double
description: Temperature deviation from normal in degrees Celsius.
precipDelta:
type: number
format: double
description: Precipitation deviation from normal as a percentage.
severity:
type: string
enum:
- ANOMALY_SEVERITY_UNSPECIFIED
- ANOMALY_SEVERITY_NORMAL
- ANOMALY_SEVERITY_MODERATE
- ANOMALY_SEVERITY_EXTREME
description: |-
AnomalySeverity represents the severity of a climate anomaly.
Maps to existing TS union: 'normal' | 'moderate' | 'extreme'.
type:
type: string
enum:
- ANOMALY_TYPE_UNSPECIFIED
- ANOMALY_TYPE_WARM
- ANOMALY_TYPE_COLD
- ANOMALY_TYPE_WET
- ANOMALY_TYPE_DRY
- ANOMALY_TYPE_MIXED
description: |-
AnomalyType represents the type of climate anomaly.
Maps to existing TS union: 'warm' | 'cold' | 'wet' | 'dry' | 'mixed'.
period:
type: string
minLength: 1
description: Time period covered (e.g., "2024-W03", "2024-01").
required:
- zone
- period
description: |-
ClimateAnomaly represents a temperature or precipitation deviation from historical norms.
Sourced from Open-Meteo / ERA5 reanalysis data.
GeoCoordinates:
type: object
properties:
latitude:
type: number
maximum: 90
minimum: -90
format: double
description: Latitude in decimal degrees (-90 to 90).
longitude:
type: number
maximum: 180
minimum: -180
format: double
description: Longitude in decimal degrees (-180 to 180).
description: GeoCoordinates represents a geographic location using WGS84 coordinates.
PaginationResponse:
type: object
properties:
nextCursor:
type: string
description: Cursor for fetching the next page. Empty string indicates no more pages.
totalCount:
type: integer
format: int32
description: Total count of items matching the query, if known. Zero if the total is unknown.
description: PaginationResponse contains pagination metadata returned alongside list results.
File diff suppressed because one or more lines are too long
+370
View File
@@ -0,0 +1,370 @@
openapi: 3.1.0
info:
title: ConflictService API
version: 1.0.0
paths:
/api/conflict/v1/list-acled-events:
post:
tags:
- ConflictService
summary: ListAcledEvents
description: ListAcledEvents retrieves armed conflict events from the ACLED dataset.
operationId: ListAcledEvents
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ListAcledEventsRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ListAcledEventsResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
/api/conflict/v1/list-ucdp-events:
post:
tags:
- ConflictService
summary: ListUcdpEvents
description: ListUcdpEvents retrieves georeferenced violence events from the UCDP dataset.
operationId: ListUcdpEvents
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ListUcdpEventsRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ListUcdpEventsResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
/api/conflict/v1/get-humanitarian-summary:
post:
tags:
- ConflictService
summary: GetHumanitarianSummary
description: GetHumanitarianSummary retrieves a humanitarian overview for a country from HAPI/HDX.
operationId: GetHumanitarianSummary
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GetHumanitarianSummaryRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/GetHumanitarianSummaryResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
components:
schemas:
Error:
type: object
properties:
message:
type: string
description: Error message (e.g., 'user not found', 'database connection failed')
description: Error is returned when a handler encounters an error. It contains a simple error message that the developer can customize.
FieldViolation:
type: object
properties:
field:
type: string
description: The field path that failed validation (e.g., 'user.email' for nested fields). For header validation, this will be the header name (e.g., 'X-API-Key')
description:
type: string
description: Human-readable description of the validation violation (e.g., 'must be a valid email address', 'required field missing')
required:
- field
- description
description: FieldViolation describes a single validation error for a specific field.
ValidationError:
type: object
properties:
violations:
type: array
items:
$ref: '#/components/schemas/FieldViolation'
description: List of validation violations
required:
- violations
description: ValidationError is returned when request validation fails. It contains a list of field violations describing what went wrong.
ListAcledEventsRequest:
type: object
properties:
timeRange:
$ref: '#/components/schemas/TimeRange'
pagination:
$ref: '#/components/schemas/PaginationRequest'
country:
type: string
description: Optional country filter (ISO 3166-1 alpha-2).
description: ListAcledEventsRequest specifies filters for retrieving ACLED conflict events.
TimeRange:
type: object
properties:
start:
type: integer
format: int64
description: 'Start of the time range (inclusive), as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
end:
type: integer
format: int64
description: 'End of the time range (inclusive), as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
description: |-
TimeRange represents a time interval defined by a start and end timestamp.
Used for filtering data within a specific time period.
PaginationRequest:
type: object
properties:
pageSize:
type: integer
maximum: 100
minimum: 1
format: int32
description: Maximum number of items to return per page (1 to 100).
cursor:
type: string
description: Opaque cursor for fetching the next page. Empty string for the first page.
description: PaginationRequest specifies cursor-based pagination parameters for list endpoints.
ListAcledEventsResponse:
type: object
properties:
events:
type: array
items:
$ref: '#/components/schemas/AcledConflictEvent'
pagination:
$ref: '#/components/schemas/PaginationResponse'
description: ListAcledEventsResponse contains ACLED conflict events matching the request.
AcledConflictEvent:
type: object
properties:
id:
type: string
minLength: 1
description: Unique ACLED event identifier.
eventType:
type: string
description: ACLED event type classification (e.g., "Battles", "Explosions/Remote violence").
country:
type: string
description: Country where the event occurred.
location:
$ref: '#/components/schemas/GeoCoordinates'
occurredAt:
type: integer
format: int64
description: 'Time the event occurred, as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
fatalities:
type: integer
format: int32
description: Reported fatalities from this event.
actors:
type: array
items:
type: string
description: Named actors involved in the event.
source:
type: string
description: Source article or report.
admin1:
type: string
description: Administrative region within the country.
required:
- id
description: AcledConflictEvent represents an armed conflict event from the ACLED dataset.
GeoCoordinates:
type: object
properties:
latitude:
type: number
maximum: 90
minimum: -90
format: double
description: Latitude in decimal degrees (-90 to 90).
longitude:
type: number
maximum: 180
minimum: -180
format: double
description: Longitude in decimal degrees (-180 to 180).
description: GeoCoordinates represents a geographic location using WGS84 coordinates.
PaginationResponse:
type: object
properties:
nextCursor:
type: string
description: Cursor for fetching the next page. Empty string indicates no more pages.
totalCount:
type: integer
format: int32
description: Total count of items matching the query, if known. Zero if the total is unknown.
description: PaginationResponse contains pagination metadata returned alongside list results.
ListUcdpEventsRequest:
type: object
properties:
timeRange:
$ref: '#/components/schemas/TimeRange'
pagination:
$ref: '#/components/schemas/PaginationRequest'
country:
type: string
description: Optional country filter (ISO 3166-1 alpha-2).
description: ListUcdpEventsRequest specifies filters for retrieving UCDP violence events.
ListUcdpEventsResponse:
type: object
properties:
events:
type: array
items:
$ref: '#/components/schemas/UcdpViolenceEvent'
pagination:
$ref: '#/components/schemas/PaginationResponse'
description: ListUcdpEventsResponse contains UCDP violence events matching the request.
UcdpViolenceEvent:
type: object
properties:
id:
type: string
minLength: 1
description: Unique UCDP event identifier.
dateStart:
type: integer
format: int64
description: 'Start date of the event, as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
dateEnd:
type: integer
format: int64
description: 'End date of the event, as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
location:
$ref: '#/components/schemas/GeoCoordinates'
country:
type: string
description: Country where the event occurred.
sideA:
type: string
description: Primary party in the conflict (Side A).
sideB:
type: string
description: Secondary party in the conflict (Side B).
deathsBest:
type: integer
format: int32
description: Best estimate of deaths.
deathsLow:
type: integer
format: int32
description: Low estimate of deaths.
deathsHigh:
type: integer
format: int32
description: High estimate of deaths.
violenceType:
type: string
enum:
- UCDP_VIOLENCE_TYPE_UNSPECIFIED
- UCDP_VIOLENCE_TYPE_STATE_BASED
- UCDP_VIOLENCE_TYPE_NON_STATE
- UCDP_VIOLENCE_TYPE_ONE_SIDED
description: |-
UcdpViolenceType represents the UCDP violence classification.
Maps to existing TS union: 'state-based' | 'non-state' | 'one-sided'.
sourceOriginal:
type: string
description: Original source of the event report.
required:
- id
description: UcdpViolenceEvent represents a georeferenced violence event from the UCDP dataset.
GetHumanitarianSummaryRequest:
type: object
properties:
countryCode:
type: string
pattern: ^[A-Z]{2}$
description: ISO 3166-1 alpha-2 country code (e.g., "YE", "SD", "SO").
required:
- countryCode
description: GetHumanitarianSummaryRequest specifies which country to retrieve the humanitarian summary for.
GetHumanitarianSummaryResponse:
type: object
properties:
summary:
$ref: '#/components/schemas/HumanitarianCountrySummary'
description: GetHumanitarianSummaryResponse contains the humanitarian summary for the requested country.
HumanitarianCountrySummary:
type: object
properties:
countryCode:
type: string
description: ISO 3166-1 alpha-2 country code.
countryName:
type: string
description: Country name.
conflictEventsTotal:
type: integer
format: int32
description: Total conflict events in the reference period.
conflictPoliticalViolenceEvents:
type: integer
format: int32
description: Political violence + civilian targeting event count.
conflictFatalities:
type: integer
format: int32
description: Total fatalities from political violence and civilian targeting.
referencePeriod:
type: string
description: Reference period start date (YYYY-MM-DD).
conflictDemonstrations:
type: integer
format: int32
description: Number of demonstration events.
updatedAt:
type: integer
format: int64
description: 'Last data update time, as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
description: HumanitarianCountrySummary represents HAPI conflict event counts for a country.
File diff suppressed because one or more lines are too long
+256
View File
@@ -0,0 +1,256 @@
openapi: 3.1.0
info:
title: CyberService API
version: 1.0.0
paths:
/api/cyber/v1/list-cyber-threats:
post:
tags:
- CyberService
summary: ListCyberThreats
description: ListCyberThreats retrieves threat indicators from multiple intelligence sources.
operationId: ListCyberThreats
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ListCyberThreatsRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ListCyberThreatsResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
components:
schemas:
Error:
type: object
properties:
message:
type: string
description: Error message (e.g., 'user not found', 'database connection failed')
description: Error is returned when a handler encounters an error. It contains a simple error message that the developer can customize.
FieldViolation:
type: object
properties:
field:
type: string
description: The field path that failed validation (e.g., 'user.email' for nested fields). For header validation, this will be the header name (e.g., 'X-API-Key')
description:
type: string
description: Human-readable description of the validation violation (e.g., 'must be a valid email address', 'required field missing')
required:
- field
- description
description: FieldViolation describes a single validation error for a specific field.
ValidationError:
type: object
properties:
violations:
type: array
items:
$ref: '#/components/schemas/FieldViolation'
description: List of validation violations
required:
- violations
description: ValidationError is returned when request validation fails. It contains a list of field violations describing what went wrong.
ListCyberThreatsRequest:
type: object
properties:
timeRange:
$ref: '#/components/schemas/TimeRange'
pagination:
$ref: '#/components/schemas/PaginationRequest'
type:
type: string
enum:
- CYBER_THREAT_TYPE_UNSPECIFIED
- CYBER_THREAT_TYPE_C2_SERVER
- CYBER_THREAT_TYPE_MALWARE_HOST
- CYBER_THREAT_TYPE_PHISHING
- CYBER_THREAT_TYPE_MALICIOUS_URL
description: |-
CyberThreatType represents the classification of a cyber threat.
Maps to TS union: 'c2_server' | 'malware_host' | 'phishing' | 'malicious_url'.
source:
type: string
enum:
- CYBER_THREAT_SOURCE_UNSPECIFIED
- CYBER_THREAT_SOURCE_FEODO
- CYBER_THREAT_SOURCE_URLHAUS
- CYBER_THREAT_SOURCE_C2INTEL
- CYBER_THREAT_SOURCE_OTX
- CYBER_THREAT_SOURCE_ABUSEIPDB
description: |-
CyberThreatSource represents the intelligence source of a cyber threat.
Maps to TS union: 'feodo' | 'urlhaus' | 'c2intel' | 'otx' | 'abuseipdb'.
minSeverity:
type: string
enum:
- CRITICALITY_LEVEL_UNSPECIFIED
- CRITICALITY_LEVEL_LOW
- CRITICALITY_LEVEL_MEDIUM
- CRITICALITY_LEVEL_HIGH
- CRITICALITY_LEVEL_CRITICAL
description: |-
CriticalityLevel represents a four-tier criticality classification for cyber and risk domains.
Maps to existing TS union: 'low' | 'medium' | 'high' | 'critical'.
description: ListCyberThreatsRequest specifies filters for retrieving cyber threat indicators.
TimeRange:
type: object
properties:
start:
type: integer
format: int64
description: 'Start of the time range (inclusive), as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
end:
type: integer
format: int64
description: 'End of the time range (inclusive), as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
description: |-
TimeRange represents a time interval defined by a start and end timestamp.
Used for filtering data within a specific time period.
PaginationRequest:
type: object
properties:
pageSize:
type: integer
maximum: 100
minimum: 1
format: int32
description: Maximum number of items to return per page (1 to 100).
cursor:
type: string
description: Opaque cursor for fetching the next page. Empty string for the first page.
description: PaginationRequest specifies cursor-based pagination parameters for list endpoints.
ListCyberThreatsResponse:
type: object
properties:
threats:
type: array
items:
$ref: '#/components/schemas/CyberThreat'
pagination:
$ref: '#/components/schemas/PaginationResponse'
description: ListCyberThreatsResponse contains cyber threats matching the request.
CyberThreat:
type: object
properties:
id:
type: string
minLength: 1
description: Unique threat identifier.
type:
type: string
enum:
- CYBER_THREAT_TYPE_UNSPECIFIED
- CYBER_THREAT_TYPE_C2_SERVER
- CYBER_THREAT_TYPE_MALWARE_HOST
- CYBER_THREAT_TYPE_PHISHING
- CYBER_THREAT_TYPE_MALICIOUS_URL
description: |-
CyberThreatType represents the classification of a cyber threat.
Maps to TS union: 'c2_server' | 'malware_host' | 'phishing' | 'malicious_url'.
source:
type: string
enum:
- CYBER_THREAT_SOURCE_UNSPECIFIED
- CYBER_THREAT_SOURCE_FEODO
- CYBER_THREAT_SOURCE_URLHAUS
- CYBER_THREAT_SOURCE_C2INTEL
- CYBER_THREAT_SOURCE_OTX
- CYBER_THREAT_SOURCE_ABUSEIPDB
description: |-
CyberThreatSource represents the intelligence source of a cyber threat.
Maps to TS union: 'feodo' | 'urlhaus' | 'c2intel' | 'otx' | 'abuseipdb'.
indicator:
type: string
description: Threat indicator value (IP, domain, or URL).
indicatorType:
type: string
enum:
- CYBER_THREAT_INDICATOR_TYPE_UNSPECIFIED
- CYBER_THREAT_INDICATOR_TYPE_IP
- CYBER_THREAT_INDICATOR_TYPE_DOMAIN
- CYBER_THREAT_INDICATOR_TYPE_URL
description: |-
CyberThreatIndicatorType represents the type of threat indicator.
Maps to TS union: 'ip' | 'domain' | 'url'.
location:
$ref: '#/components/schemas/GeoCoordinates'
country:
type: string
description: Country of origin (ISO 3166-1 alpha-2).
severity:
type: string
enum:
- CRITICALITY_LEVEL_UNSPECIFIED
- CRITICALITY_LEVEL_LOW
- CRITICALITY_LEVEL_MEDIUM
- CRITICALITY_LEVEL_HIGH
- CRITICALITY_LEVEL_CRITICAL
description: |-
CriticalityLevel represents a four-tier criticality classification for cyber and risk domains.
Maps to existing TS union: 'low' | 'medium' | 'high' | 'critical'.
malwareFamily:
type: string
description: Associated malware family, if known.
tags:
type: array
items:
type: string
description: Descriptive tags.
firstSeenAt:
type: integer
format: int64
description: 'First seen time, as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
lastSeenAt:
type: integer
format: int64
description: 'Last seen time, as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
required:
- id
description: |-
CyberThreat represents a cyber threat indicator aggregated from multiple sources.
Sources include Feodo Tracker, URLhaus, OTX, AbuseIPDB, and C2Intel.
GeoCoordinates:
type: object
properties:
latitude:
type: number
maximum: 90
minimum: -90
format: double
description: Latitude in decimal degrees (-90 to 90).
longitude:
type: number
maximum: 180
minimum: -180
format: double
description: Longitude in decimal degrees (-180 to 180).
description: GeoCoordinates represents a geographic location using WGS84 coordinates.
PaginationResponse:
type: object
properties:
nextCursor:
type: string
description: Cursor for fetching the next page. Empty string indicates no more pages.
totalCount:
type: integer
format: int32
description: Total count of items matching the query, if known. Zero if the total is unknown.
description: PaginationResponse contains pagination metadata returned alongside list results.
File diff suppressed because one or more lines are too long
+343
View File
@@ -0,0 +1,343 @@
openapi: 3.1.0
info:
title: DisplacementService API
version: 1.0.0
paths:
/api/displacement/v1/get-displacement-summary:
post:
tags:
- DisplacementService
summary: GetDisplacementSummary
description: GetDisplacementSummary retrieves global refugee and IDP statistics from UNHCR.
operationId: GetDisplacementSummary
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GetDisplacementSummaryRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/GetDisplacementSummaryResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
/api/displacement/v1/get-population-exposure:
post:
tags:
- DisplacementService
summary: GetPopulationExposure
description: GetPopulationExposure returns country population data or estimates population within a radius.
operationId: GetPopulationExposure
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GetPopulationExposureRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/GetPopulationExposureResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
components:
schemas:
Error:
type: object
properties:
message:
type: string
description: Error message (e.g., 'user not found', 'database connection failed')
description: Error is returned when a handler encounters an error. It contains a simple error message that the developer can customize.
FieldViolation:
type: object
properties:
field:
type: string
description: The field path that failed validation (e.g., 'user.email' for nested fields). For header validation, this will be the header name (e.g., 'X-API-Key')
description:
type: string
description: Human-readable description of the validation violation (e.g., 'must be a valid email address', 'required field missing')
required:
- field
- description
description: FieldViolation describes a single validation error for a specific field.
ValidationError:
type: object
properties:
violations:
type: array
items:
$ref: '#/components/schemas/FieldViolation'
description: List of validation violations
required:
- violations
description: ValidationError is returned when request validation fails. It contains a list of field violations describing what went wrong.
GetDisplacementSummaryRequest:
type: object
properties:
year:
type: integer
minimum: 0
format: int32
description: Data year to retrieve (e.g., 2023). Uses latest available if zero.
countryLimit:
type: integer
minimum: 0
format: int32
description: Maximum number of country entries to return.
flowLimit:
type: integer
minimum: 0
format: int32
description: Maximum number of displacement flows to return.
description: GetDisplacementSummaryRequest specifies parameters for retrieving displacement data.
GetDisplacementSummaryResponse:
type: object
properties:
summary:
$ref: '#/components/schemas/DisplacementSummary'
description: GetDisplacementSummaryResponse contains the global displacement summary.
DisplacementSummary:
type: object
properties:
year:
type: integer
format: int32
description: Data year.
globalTotals:
$ref: '#/components/schemas/GlobalDisplacementTotals'
countries:
type: array
items:
$ref: '#/components/schemas/CountryDisplacement'
topFlows:
type: array
items:
$ref: '#/components/schemas/DisplacementFlow'
description: DisplacementSummary represents a global overview of displacement data from UNHCR.
GlobalDisplacementTotals:
type: object
properties:
refugees:
type: integer
minimum: 0
format: int64
description: 'Total recognized refugees worldwide.. Warning: Values > 2^53 may lose precision in JavaScript'
asylumSeekers:
type: integer
minimum: 0
format: int64
description: 'Total asylum seekers worldwide.. Warning: Values > 2^53 may lose precision in JavaScript'
idps:
type: integer
minimum: 0
format: int64
description: 'Total internally displaced persons worldwide.. Warning: Values > 2^53 may lose precision in JavaScript'
stateless:
type: integer
minimum: 0
format: int64
description: 'Total stateless persons worldwide.. Warning: Values > 2^53 may lose precision in JavaScript'
total:
type: integer
minimum: 0
format: int64
description: 'Grand total of displaced persons.. Warning: Values > 2^53 may lose precision in JavaScript'
description: GlobalDisplacementTotals represents worldwide displacement figures.
CountryDisplacement:
type: object
properties:
code:
type: string
minLength: 1
description: ISO 3166-1 alpha-2 country code.
name:
type: string
description: Country name.
refugees:
type: integer
format: int64
description: 'Refugees originating from this country.. Warning: Values > 2^53 may lose precision in JavaScript'
asylumSeekers:
type: integer
format: int64
description: 'Asylum seekers from this country.. Warning: Values > 2^53 may lose precision in JavaScript'
idps:
type: integer
format: int64
description: 'Internally displaced persons within this country.. Warning: Values > 2^53 may lose precision in JavaScript'
stateless:
type: integer
format: int64
description: 'Stateless persons associated with this country.. Warning: Values > 2^53 may lose precision in JavaScript'
totalDisplaced:
type: integer
format: int64
description: 'Total displaced from this country.. Warning: Values > 2^53 may lose precision in JavaScript'
hostRefugees:
type: integer
format: int64
description: 'Refugees hosted by this country.. Warning: Values > 2^53 may lose precision in JavaScript'
hostAsylumSeekers:
type: integer
format: int64
description: 'Asylum seekers hosted by this country.. Warning: Values > 2^53 may lose precision in JavaScript'
hostTotal:
type: integer
format: int64
description: 'Total persons hosted by this country.. Warning: Values > 2^53 may lose precision in JavaScript'
location:
$ref: '#/components/schemas/GeoCoordinates'
required:
- code
description: CountryDisplacement represents displacement metrics for a single country.
GeoCoordinates:
type: object
properties:
latitude:
type: number
maximum: 90
minimum: -90
format: double
description: Latitude in decimal degrees (-90 to 90).
longitude:
type: number
maximum: 180
minimum: -180
format: double
description: Longitude in decimal degrees (-180 to 180).
description: GeoCoordinates represents a geographic location using WGS84 coordinates.
DisplacementFlow:
type: object
properties:
originCode:
type: string
minLength: 1
description: ISO 3166-1 alpha-2 code of the origin country.
originName:
type: string
description: Origin country name.
asylumCode:
type: string
minLength: 1
description: ISO 3166-1 alpha-2 code of the asylum country.
asylumName:
type: string
description: Asylum country name.
refugees:
type: integer
format: int64
description: 'Number of refugees in this flow.. Warning: Values > 2^53 may lose precision in JavaScript'
originLocation:
$ref: '#/components/schemas/GeoCoordinates'
asylumLocation:
$ref: '#/components/schemas/GeoCoordinates'
required:
- originCode
- asylumCode
description: DisplacementFlow represents a refugee movement corridor between two countries.
GetPopulationExposureRequest:
type: object
properties:
mode:
type: string
description: 'Mode: "countries" (default) or "exposure".'
lat:
type: number
maximum: 90
minimum: -90
format: double
description: Latitude (required for exposure mode).
lon:
type: number
maximum: 180
minimum: -180
format: double
description: Longitude (required for exposure mode).
radius:
type: number
minimum: 0
format: double
description: Radius in km (required for exposure mode, defaults to 50).
description: |-
GetPopulationExposureRequest supports two modes:
- countries mode (default): returns the priority countries list
- exposure mode: estimates population within a radius of a point
GetPopulationExposureResponse:
type: object
properties:
success:
type: boolean
description: True if the request succeeded.
countries:
type: array
items:
$ref: '#/components/schemas/CountryPopulationEntry'
exposure:
$ref: '#/components/schemas/ExposureResult'
description: GetPopulationExposureResponse returns either a countries list or an exposure estimate.
CountryPopulationEntry:
type: object
properties:
code:
type: string
description: ISO 3166-1 alpha-3 country code.
name:
type: string
description: Country name.
population:
type: integer
format: int64
description: 'Total population.. Warning: Values > 2^53 may lose precision in JavaScript'
densityPerKm2:
type: integer
format: int32
description: Population density per square kilometer.
description: CountryPopulationEntry represents a country with population data.
ExposureResult:
type: object
properties:
exposedPopulation:
type: integer
format: int64
description: 'Estimated exposed population.. Warning: Values > 2^53 may lose precision in JavaScript'
exposureRadiusKm:
type: number
format: double
description: Radius used for the estimate in km.
nearestCountry:
type: string
description: ISO3 code of nearest priority country.
densityPerKm2:
type: integer
format: int32
description: Population density used for the estimate.
description: ExposureResult contains the population exposure estimate.
File diff suppressed because one or more lines are too long
+529
View File
@@ -0,0 +1,529 @@
openapi: 3.1.0
info:
title: EconomicService API
version: 1.0.0
paths:
/api/economic/v1/get-fred-series:
post:
tags:
- EconomicService
summary: GetFredSeries
description: GetFredSeries retrieves time series data from the Federal Reserve Economic Data.
operationId: GetFredSeries
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GetFredSeriesRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/GetFredSeriesResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
/api/economic/v1/list-world-bank-indicators:
post:
tags:
- EconomicService
summary: ListWorldBankIndicators
description: ListWorldBankIndicators retrieves development indicator data from the World Bank.
operationId: ListWorldBankIndicators
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/ListWorldBankIndicatorsRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/ListWorldBankIndicatorsResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
/api/economic/v1/get-energy-prices:
post:
tags:
- EconomicService
summary: GetEnergyPrices
description: GetEnergyPrices retrieves current energy commodity prices from EIA.
operationId: GetEnergyPrices
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GetEnergyPricesRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/GetEnergyPricesResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
/api/economic/v1/get-macro-signals:
post:
tags:
- EconomicService
summary: GetMacroSignals
description: GetMacroSignals computes 7 macro signals from 6 upstream sources with BUY/CASH verdict.
operationId: GetMacroSignals
requestBody:
content:
application/json:
schema:
$ref: '#/components/schemas/GetMacroSignalsRequest'
required: true
responses:
"200":
description: Successful response
content:
application/json:
schema:
$ref: '#/components/schemas/GetMacroSignalsResponse'
"400":
description: Validation error
content:
application/json:
schema:
$ref: '#/components/schemas/ValidationError'
default:
description: Error response
content:
application/json:
schema:
$ref: '#/components/schemas/Error'
components:
schemas:
Error:
type: object
properties:
message:
type: string
description: Error message (e.g., 'user not found', 'database connection failed')
description: Error is returned when a handler encounters an error. It contains a simple error message that the developer can customize.
FieldViolation:
type: object
properties:
field:
type: string
description: The field path that failed validation (e.g., 'user.email' for nested fields). For header validation, this will be the header name (e.g., 'X-API-Key')
description:
type: string
description: Human-readable description of the validation violation (e.g., 'must be a valid email address', 'required field missing')
required:
- field
- description
description: FieldViolation describes a single validation error for a specific field.
ValidationError:
type: object
properties:
violations:
type: array
items:
$ref: '#/components/schemas/FieldViolation'
description: List of validation violations
required:
- violations
description: ValidationError is returned when request validation fails. It contains a list of field violations describing what went wrong.
GetFredSeriesRequest:
type: object
properties:
seriesId:
type: string
minLength: 1
description: FRED series ID (e.g., "GDP", "UNRATE", "CPIAUCSL").
limit:
type: integer
format: int32
description: Maximum number of observations to return. Defaults to 120.
required:
- seriesId
description: GetFredSeriesRequest specifies which FRED series to retrieve.
GetFredSeriesResponse:
type: object
properties:
series:
$ref: '#/components/schemas/FredSeries'
description: GetFredSeriesResponse contains the requested FRED series data.
FredSeries:
type: object
properties:
seriesId:
type: string
minLength: 1
description: Series identifier (e.g., "GDP", "UNRATE", "CPIAUCSL").
title:
type: string
description: Series title.
units:
type: string
description: Unit of measurement.
frequency:
type: string
description: Data frequency (e.g., "Monthly", "Quarterly").
observations:
type: array
items:
$ref: '#/components/schemas/FredObservation'
required:
- seriesId
description: FredSeries represents a FRED time series with metadata.
FredObservation:
type: object
properties:
date:
type: string
description: Observation date as YYYY-MM-DD string.
value:
type: number
format: double
description: Observation value.
description: FredObservation represents a single data point from a FRED economic series.
ListWorldBankIndicatorsRequest:
type: object
properties:
indicatorCode:
type: string
minLength: 1
description: World Bank indicator code (e.g., "NY.GDP.MKTP.CD").
countryCode:
type: string
description: Optional country filter (ISO 3166-1 alpha-2).
year:
type: integer
format: int32
description: Optional year filter. Defaults to latest available.
pagination:
$ref: '#/components/schemas/PaginationRequest'
required:
- indicatorCode
description: ListWorldBankIndicatorsRequest specifies filters for retrieving World Bank data.
PaginationRequest:
type: object
properties:
pageSize:
type: integer
maximum: 100
minimum: 1
format: int32
description: Maximum number of items to return per page (1 to 100).
cursor:
type: string
description: Opaque cursor for fetching the next page. Empty string for the first page.
description: PaginationRequest specifies cursor-based pagination parameters for list endpoints.
ListWorldBankIndicatorsResponse:
type: object
properties:
data:
type: array
items:
$ref: '#/components/schemas/WorldBankCountryData'
pagination:
$ref: '#/components/schemas/PaginationResponse'
description: ListWorldBankIndicatorsResponse contains World Bank indicator data.
WorldBankCountryData:
type: object
properties:
countryCode:
type: string
minLength: 1
description: ISO 3166-1 alpha-2 country code.
countryName:
type: string
description: Country name.
indicatorCode:
type: string
minLength: 1
description: World Bank indicator code (e.g., "NY.GDP.MKTP.CD").
indicatorName:
type: string
description: Indicator name.
year:
type: integer
format: int32
description: Data year.
value:
type: number
format: double
description: Indicator value.
required:
- countryCode
- indicatorCode
description: WorldBankCountryData represents a World Bank indicator value for a country.
PaginationResponse:
type: object
properties:
nextCursor:
type: string
description: Cursor for fetching the next page. Empty string indicates no more pages.
totalCount:
type: integer
format: int32
description: Total count of items matching the query, if known. Zero if the total is unknown.
description: PaginationResponse contains pagination metadata returned alongside list results.
GetEnergyPricesRequest:
type: object
properties:
commodities:
type: array
items:
type: string
description: Optional commodity filter. Empty returns all tracked commodities.
description: GetEnergyPricesRequest specifies which energy commodities to retrieve.
GetEnergyPricesResponse:
type: object
properties:
prices:
type: array
items:
$ref: '#/components/schemas/EnergyPrice'
description: GetEnergyPricesResponse contains energy price data.
EnergyPrice:
type: object
properties:
commodity:
type: string
minLength: 1
description: Energy commodity identifier.
name:
type: string
description: Human-readable name (e.g., "WTI Crude Oil", "Henry Hub Natural Gas").
price:
type: number
format: double
description: Current price in USD.
unit:
type: string
description: Unit of measurement (e.g., "$/barrel", "$/MMBtu").
change:
type: number
format: double
description: Percentage change from previous period.
priceAt:
type: integer
format: int64
description: 'Price date, as Unix epoch milliseconds.. Warning: Values > 2^53 may lose precision in JavaScript'
required:
- commodity
description: EnergyPrice represents a current energy commodity price from EIA.
GetMacroSignalsRequest:
type: object
description: GetMacroSignalsRequest requests the current macro signal dashboard.
GetMacroSignalsResponse:
type: object
properties:
timestamp:
type: string
description: ISO 8601 timestamp of computation.
verdict:
type: string
description: 'Overall verdict: "BUY", "CASH", or "UNKNOWN".'
bullishCount:
type: integer
format: int32
description: Number of bullish signals.
totalCount:
type: integer
format: int32
description: Total number of evaluated signals (excluding UNKNOWN).
signals:
$ref: '#/components/schemas/MacroSignals'
meta:
$ref: '#/components/schemas/MacroMeta'
unavailable:
type: boolean
description: True when upstream data is unavailable (fallback result).
description: GetMacroSignalsResponse contains the full macro signal dashboard with 7 signals and verdict.
MacroSignals:
type: object
properties:
liquidity:
$ref: '#/components/schemas/LiquiditySignal'
flowStructure:
$ref: '#/components/schemas/FlowStructureSignal'
macroRegime:
$ref: '#/components/schemas/MacroRegimeSignal'
technicalTrend:
$ref: '#/components/schemas/TechnicalTrendSignal'
hashRate:
$ref: '#/components/schemas/HashRateSignal'
miningCost:
$ref: '#/components/schemas/MiningCostSignal'
fearGreed:
$ref: '#/components/schemas/FearGreedSignal'
description: MacroSignals contains all 7 individual signal computations.
LiquiditySignal:
type: object
properties:
status:
type: string
description: '"SQUEEZE", "NORMAL", or "UNKNOWN".'
value:
type: number
format: double
description: JPY 30d ROC percentage, absent if unavailable.
sparkline:
type: array
items:
type: number
format: double
description: Last 30 JPY close prices.
description: LiquiditySignal tracks JPY 30d rate of change as a liquidity proxy.
FlowStructureSignal:
type: object
properties:
status:
type: string
description: '"PASSIVE GAP", "ALIGNED", or "UNKNOWN".'
btcReturn5:
type: number
format: double
description: BTC 5-day return percentage.
qqqReturn5:
type: number
format: double
description: QQQ 5-day return percentage.
description: FlowStructureSignal compares BTC vs QQQ 5-day returns.
MacroRegimeSignal:
type: object
properties:
status:
type: string
description: '"RISK-ON", "DEFENSIVE", or "UNKNOWN".'
qqqRoc20:
type: number
format: double
description: QQQ 20d ROC percentage.
xlpRoc20:
type: number
format: double
description: XLP 20d ROC percentage.
description: MacroRegimeSignal compares QQQ vs XLP 20-day rate of change.
TechnicalTrendSignal:
type: object
properties:
status:
type: string
description: '"BULLISH", "BEARISH", "NEUTRAL", or "UNKNOWN".'
btcPrice:
type: number
format: double
description: Current BTC price.
sma50:
type: number
format: double
description: 50-day simple moving average.
sma200:
type: number
format: double
description: 200-day simple moving average.
vwap30d:
type: number
format: double
description: 30-day volume-weighted average price.
mayerMultiple:
type: number
format: double
description: Mayer multiple (BTC price / SMA200).
sparkline:
type: array
items:
type: number
format: double
description: Last 30 BTC close prices.
description: TechnicalTrendSignal evaluates BTC price vs moving averages and VWAP.
HashRateSignal:
type: object
properties:
status:
type: string
description: '"GROWING", "DECLINING", "STABLE", or "UNKNOWN".'
change30d:
type: number
format: double
description: Hash rate change over 30 days as percentage.
description: HashRateSignal tracks Bitcoin hash rate momentum.
MiningCostSignal:
type: object
properties:
status:
type: string
description: '"PROFITABLE", "TIGHT", "SQUEEZE", or "UNKNOWN".'
description: MiningCostSignal estimates mining profitability from BTC price thresholds.
FearGreedSignal:
type: object
properties:
status:
type: string
description: Classification label (e.g., "Extreme Fear", "Greed").
value:
type: integer
format: int32
description: Current index value (0-100).
history:
type: array
items:
$ref: '#/components/schemas/FearGreedHistoryEntry'
description: FearGreedSignal tracks the Crypto Fear & Greed index.
FearGreedHistoryEntry:
type: object
properties:
value:
type: integer
maximum: 100
minimum: 0
format: int32
description: Index value (0-100).
date:
type: string
description: Date string (YYYY-MM-DD).
description: FearGreedHistoryEntry is a single day's Fear & Greed index reading.
MacroMeta:
type: object
properties:
qqqSparkline:
type: array
items:
type: number
format: double
description: Last 30 QQQ close prices for sparkline.
description: MacroMeta contains supplementary chart data.
File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show More