Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
012907ae84 | ||
|
|
2db217211e | ||
|
|
1cc961b491 | ||
|
|
fc7b91fa9d | ||
|
|
e5fc09d1fc | ||
|
|
812c6d0562 | ||
|
|
a560efff49 | ||
|
|
2839d884db | ||
|
|
0060a71a66 | ||
|
|
f912c5f15a | ||
|
|
384cc78daa | ||
|
|
5221486c88 | ||
|
|
e5b30ba28c | ||
|
|
8ac824ebb4 | ||
|
|
be82576037 | ||
|
|
66e9f818d4 | ||
|
|
75dc6418ea | ||
|
|
e0d0cf9790 | ||
|
|
3983278f53 | ||
|
|
dc30693135 | ||
|
|
b0071d1a14 | ||
|
|
1844276017 | ||
|
|
24d42f70f8 | ||
|
|
95228647b4 | ||
|
|
bf4e216efd | ||
|
|
6ac39d5166 | ||
|
|
5c4b1ffc6b | ||
|
|
07d0803014 | ||
|
|
b238618e13 | ||
|
|
5206027573 | ||
|
|
e7260be2f2 | ||
|
|
cc8f5f3f01 | ||
|
|
e3bdde0d92 | ||
|
|
cb0235ca6b | ||
|
|
797d127ce1 | ||
|
|
2f92dbee8b | ||
|
|
1d069229c0 | ||
|
|
1d72ac8ab5 | ||
|
|
61dcfb9ca8 | ||
|
|
37096e8954 | ||
|
|
7e0ccfe30e | ||
|
|
57af053d92 | ||
|
|
408d5d3374 | ||
|
|
b1d835b69f | ||
|
|
fb6c61d4d9 | ||
|
|
9ef3cd8402 | ||
|
|
e35f0f70e9 | ||
|
|
88434cb26b | ||
|
|
d4c768dc59 | ||
|
|
f24b04fbd7 | ||
|
|
9011c91801 | ||
|
|
2881088272 | ||
|
|
7942e0e8c7 | ||
|
|
49b73c56ec | ||
|
|
a2cff87b43 | ||
|
|
4a029420b9 | ||
|
|
59646df7d8 | ||
|
|
58da7b789c | ||
|
|
1d417af59f | ||
|
|
57a5bf44f6 | ||
|
|
09011ec479 | ||
|
|
6014f268a3 | ||
|
|
9b810e1836 | ||
|
|
b8aabdc373 | ||
|
|
eebf9a9c7b | ||
|
|
557706e2d7 | ||
|
|
3849f5135f | ||
|
|
57481e8f97 | ||
|
|
81d4777964 | ||
|
|
9b932dcfac | ||
|
|
49373e5054 | ||
|
|
03598a6cff | ||
|
|
17f5dd7267 | ||
|
|
7cec4b2113 | ||
|
|
b667b189ff | ||
|
|
da680d7397 | ||
|
|
d593fbf413 | ||
|
|
adee459791 | ||
|
|
4708e1d434 | ||
|
|
10e50e080d | ||
|
|
709545857b | ||
|
|
98132b92a5 | ||
|
|
4721fb0873 | ||
|
|
9e2432f098 | ||
|
|
d68221d7f8 | ||
|
|
dc7a1ae61a | ||
|
|
6271fafd40 | ||
|
|
a37dced84e | ||
|
|
cbfc06f315 | ||
|
|
eafc4cb955 | ||
|
|
a6f5284b09 | ||
|
|
a8068ff579 | ||
|
|
bfb7cec0d1 | ||
|
|
f06acd3c88 | ||
|
|
184603b53a | ||
|
|
080f49b0b2 | ||
|
|
353a46460b | ||
|
|
eee0eece50 | ||
|
|
f790f3f63c | ||
|
|
b8be2f6890 | ||
|
|
b06c8a17b3 | ||
|
|
8504d5649a | ||
|
|
d24d61f333 | ||
|
|
0f4420d9cc | ||
|
|
63a4c9ab9c | ||
|
|
8f8b0c843b | ||
|
|
fcd590a9e6 | ||
|
|
3e935e3541 | ||
|
|
5d7e77f8b0 | ||
|
|
b23cac04b9 | ||
|
|
df2cdafadb | ||
|
|
84f17e6c4c | ||
|
|
43cd5b3826 | ||
|
|
a2bcfeede4 | ||
|
|
584159f35c | ||
|
|
440e45a032 | ||
|
|
8cc82def37 | ||
|
|
21d7c8b7f3 | ||
|
|
1a9dc02ecc | ||
|
|
e6285e3de0 | ||
|
|
4365626df3 | ||
|
|
e1b9e9e8a9 | ||
|
|
86eff555f0 | ||
|
|
740e3513ef | ||
|
|
1922a781cd | ||
|
|
8699dae5e5 | ||
|
|
1fc6dc2dbb | ||
|
|
68e6a367d6 | ||
|
|
a388afe400 | ||
|
|
417b7d275d | ||
|
|
8243147886 | ||
|
|
5786a8f279 | ||
|
|
7e86e26a9e | ||
|
|
81577cbcfa | ||
|
|
d3da54bcb2 | ||
|
|
519da36ef6 | ||
|
|
67259da470 | ||
|
|
e1f392b45b | ||
|
|
c1b8ffb890 | ||
|
|
d2490bb7b7 | ||
|
|
f082e09d55 | ||
|
|
3bf4244cde | ||
|
|
4822a839eb | ||
|
|
9273facad1 | ||
|
|
48f8e24353 | ||
|
|
d07f14fef4 | ||
|
|
c939cc6296 | ||
|
|
f8d0c81f60 | ||
|
|
7440281d71 | ||
|
|
92892c5b7c | ||
|
|
7681202c6e | ||
|
|
b714e7b13e | ||
|
|
3ffe76b208 | ||
|
|
84336a9f94 | ||
|
|
919e7c996e | ||
|
|
fd51b57911 | ||
|
|
91c9982a41 | ||
|
|
ac7f3fac5e | ||
|
|
61c1dad6c9 | ||
|
|
b49329cdef | ||
|
|
5cfd827173 | ||
|
|
ab69f09c60 | ||
|
|
7aa60f954c | ||
|
|
7a57d39408 | ||
|
|
6e9615f428 | ||
|
|
4e10bbc283 | ||
|
|
13172cec4c | ||
|
|
c62fad7fce | ||
|
|
7ab3a8e76d | ||
|
|
f521377453 | ||
|
|
dd34c1e4ff | ||
|
|
49f9fc860f | ||
|
|
b7ca969228 | ||
|
|
383da59ffc | ||
|
|
ffb5ae0413 | ||
|
|
e549b6e5a6 | ||
|
|
db4b401b8f | ||
|
|
2b7b35efd8 | ||
|
|
4b0e50e0fe | ||
|
|
cbd7c29f03 | ||
|
|
86b83bae78 | ||
|
|
814c01fea2 | ||
|
|
7972248e1c | ||
|
|
380f1b7235 | ||
|
|
316df2997d | ||
|
|
fb1efd2ebc | ||
|
|
ef13263c7b | ||
|
|
2df58717e4 | ||
|
|
282eba9838 | ||
|
|
cf46ff8940 | ||
|
|
0735ce5c78 | ||
|
|
a1973d4681 | ||
|
|
aae90df9c9 | ||
|
|
a78410b8d2 | ||
|
|
44d2254b5b | ||
|
|
e10ed6c981 | ||
|
|
07e14deace | ||
|
|
f2a1a2ccb5 | ||
|
|
7b7fe3cbdd | ||
|
|
2e267cf307 | ||
|
|
8697c543ea | ||
|
|
81ccace81b | ||
|
|
c170ada79c | ||
|
|
01a65fb201 | ||
|
|
fe2181b92b | ||
|
|
2e85a64712 | ||
|
|
4121113547 | ||
|
|
2473c4bfd4 | ||
|
|
6c02bd2eb6 | ||
|
|
816015357a | ||
|
|
9fd1bf293d | ||
|
|
bf2c0b1598 | ||
|
|
a39aed1d64 | ||
|
|
58389ba440 | ||
|
|
fce6c52970 | ||
|
|
e040994e9f | ||
|
|
7dc53c0f4c | ||
|
|
bd3b71f4d8 | ||
|
|
20480feae4 | ||
|
|
8d20069830 | ||
|
|
6a10ca3cc0 | ||
|
|
bab0974407 | ||
|
|
6c3d2770f7 | ||
|
|
eedf43e94a | ||
|
|
1d1b1b209f | ||
|
|
bb14f0e9a8 | ||
|
|
5cdc41712c | ||
|
|
ba329e2a2a | ||
|
|
3f5fa51f40 | ||
|
|
f92de9f9da | ||
|
|
a1214c9b1e | ||
|
|
471d76223f | ||
|
|
c494f5b995 | ||
|
|
987a7b5e08 | ||
|
|
98797c9b02 | ||
|
|
bc96da9106 | ||
|
|
9cff125078 | ||
|
|
aa823bbb60 | ||
|
|
62a69acc11 | ||
|
|
4be7065f07 | ||
|
|
d7a6e95611 | ||
|
|
a851d5e8a1 | ||
|
|
172e7018a3 | ||
|
|
18f4bb98a4 | ||
|
|
22869e8062 | ||
|
|
ad32dd899d | ||
|
|
f6e7bbbfbc | ||
|
|
85e2384f0f | ||
|
|
be485ad1c2 | ||
|
|
434cd120fe |
@@ -1,32 +0,0 @@
|
||||
# Sentry Triage — 2026-02-19
|
||||
|
||||
Commit: `09174fd` on `main`
|
||||
|
||||
## Issues Triaged (5)
|
||||
|
||||
### ACTIONABLE — Fixed in Code
|
||||
|
||||
| ID | Title | Events | Users | Fix |
|
||||
|---|---|---|---|---|
|
||||
| WORLDMONITOR-1G | `Error: ML request unload-model timed out after 120000ms` | 30 | 27 | Wrapped `unloadModel()` in try/catch; timeout no longer leaks as unhandled rejection. Cleans up `loadedModels` set on failure. |
|
||||
| WORLDMONITOR-1F | `Error: ML request unload-model timed out after 120000ms` | 9 | 9 | Same root cause as 1G (different release build hash). |
|
||||
| WORLDMONITOR-1K | `TypeError: this.player.playVideo is not a function` | 1 | 1 | Added optional chaining (`playVideo?.()`, `pauseVideo?.()`) in `LiveNewsPanel.ts`. YT IFrame API player object may not have methods ready during initialization race. |
|
||||
|
||||
### NOISE — Filtered
|
||||
|
||||
| ID | Title | Events | Users | Filter |
|
||||
|---|---|---|---|---|
|
||||
| WORLDMONITOR-1J | `InternalError: too much recursion` | 1 | 1 | i18next internal `translate -> extractFromKey` cycle on Firefox 147. Added `/too much recursion/` to `ignoreErrors`. |
|
||||
| WORLDMONITOR-1H | `TypeError: Cannot read properties of null (reading 'id')` | 1 | 1 | maplibre-gl internal render crash (`_drawLayers -> renderLayers`). Extended `beforeSend` regex to suppress null `id`/`type` when stack is in map chunk. |
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `src/services/ml-worker.ts` | `unloadModel()`: try/catch around `this.request()`, clean `loadedModels` on failure |
|
||||
| `src/components/LiveNewsPanel.ts` | Optional chaining on `playVideo?.()` and `pauseVideo?.()` |
|
||||
| `src/main.ts` | Added `/too much recursion/` to `ignoreErrors`; extended maplibre `beforeSend` filter for null `id`/`type` |
|
||||
|
||||
## Sentry Status
|
||||
|
||||
All 5 issues marked **resolved (in next release)** via API. They will auto-reopen if errors recur after deployment.
|
||||
+50
-4
@@ -78,7 +78,8 @@ NASA_FIRMS_API_KEY=
|
||||
|
||||
|
||||
# ------ Railway Relay (scripts/ais-relay.cjs) ------
|
||||
# The relay server handles AIS vessel tracking and OpenSky aircraft data.
|
||||
# The relay server handles AIS vessel tracking + OpenSky aircraft data + RSS proxy.
|
||||
# It can also run the Telegram OSINT poller (stateful MTProto) when configured.
|
||||
# Deploy on Railway with: node scripts/ais-relay.cjs
|
||||
|
||||
# AISStream API key for live vessel positions
|
||||
@@ -91,22 +92,46 @@ OPENSKY_CLIENT_ID=
|
||||
OPENSKY_CLIENT_SECRET=
|
||||
|
||||
|
||||
# ------ Telegram OSINT (Railway relay) ------
|
||||
# Telegram MTProto keys (free): https://my.telegram.org/apps
|
||||
TELEGRAM_API_ID=
|
||||
TELEGRAM_API_HASH=
|
||||
|
||||
# GramJS StringSession generated locally (see: scripts/telegram/session-auth.mjs)
|
||||
TELEGRAM_SESSION=
|
||||
|
||||
# Which curated list bucket to ingest: full | tech | finance
|
||||
TELEGRAM_CHANNEL_SET=full
|
||||
|
||||
# ------ Railway Relay Connection (Vercel → Railway) ------
|
||||
|
||||
# Server-side URL (https://) — used by Vercel edge functions to reach the relay
|
||||
WS_RELAY_URL=
|
||||
|
||||
# Client-side URL (wss://) — used by the browser to connect via WebSocket
|
||||
# Optional client-side URL (wss://) — local/dev fallback only
|
||||
VITE_WS_RELAY_URL=
|
||||
|
||||
# Shared secret between Vercel and Railway relay.
|
||||
# Must be set to the SAME value on both platforms in production.
|
||||
RELAY_SHARED_SECRET=
|
||||
|
||||
# Header name used to send the relay secret (must match on both platforms)
|
||||
RELAY_AUTH_HEADER=x-relay-key
|
||||
|
||||
# Emergency production override to allow unauthenticated relay traffic.
|
||||
# Leave unset/false in production.
|
||||
ALLOW_UNAUTHENTICATED_RELAY=false
|
||||
|
||||
# Rolling window size (seconds) used by relay /metrics endpoint.
|
||||
RELAY_METRICS_WINDOW_SECONDS=60
|
||||
|
||||
|
||||
# ------ Public Data Sources (no keys required) ------
|
||||
|
||||
# UCDP (Uppsala Conflict Data Program) — public API, no auth
|
||||
# UNHCR (UN Refugee Agency) — public API, no auth (CC BY 4.0)
|
||||
# Open-Meteo — public API, no auth (processes Copernicus ERA5)
|
||||
# WorldPop — public API, optional key for higher rate limits
|
||||
# WORLDPOP_API_KEY=
|
||||
# WorldPop — public API, no auth needed
|
||||
|
||||
|
||||
# ------ Site Configuration ------
|
||||
@@ -114,7 +139,28 @@ VITE_WS_RELAY_URL=
|
||||
# Site variant: "full" (worldmonitor.app) or "tech" (tech.worldmonitor.app)
|
||||
VITE_VARIANT=full
|
||||
|
||||
# Client-side Sentry DSN (optional). Leave empty to disable error reporting.
|
||||
VITE_SENTRY_DSN=
|
||||
|
||||
# PostHog product analytics (optional). Leave empty to disable analytics.
|
||||
VITE_POSTHOG_KEY=
|
||||
VITE_POSTHOG_HOST=
|
||||
|
||||
# Map interaction mode:
|
||||
# - "flat" keeps pitch/rotation disabled (2D interaction)
|
||||
# - "3d" enables pitch/rotation interactions (default)
|
||||
VITE_MAP_INTERACTION_MODE=3d
|
||||
|
||||
|
||||
# ------ Desktop Cloud Fallback (Vercel) ------
|
||||
|
||||
# Comma-separated list of valid API keys for desktop cloud fallback.
|
||||
# Generate with: openssl rand -hex 24 | sed 's/^/wm_/'
|
||||
WORLDMONITOR_VALID_KEYS=
|
||||
|
||||
|
||||
# ------ Registration DB (Convex) ------
|
||||
|
||||
# Convex deployment URL for email registration storage.
|
||||
# Set up at: https://dashboard.convex.dev/
|
||||
CONVEX_URL=
|
||||
|
||||
@@ -18,6 +18,7 @@ body:
|
||||
- finance.worldmonitor.app (Finance)
|
||||
- Desktop app (Windows)
|
||||
- Desktop app (macOS)
|
||||
- Desktop app (Linux)
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -37,6 +38,8 @@ body:
|
||||
- Live video streams
|
||||
- Desktop app (Tauri)
|
||||
- Settings / API keys
|
||||
- Settings / LLMs (Ollama, Groq, OpenRouter)
|
||||
- Live webcams
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -50,7 +50,7 @@ jobs:
|
||||
node_target: 'x86_64-pc-windows-msvc'
|
||||
label: 'Windows-x64'
|
||||
timeout: 120
|
||||
- platform: 'ubuntu-22.04'
|
||||
- platform: 'ubuntu-24.04'
|
||||
args: ''
|
||||
node_target: 'x86_64-unknown-linux-gnu'
|
||||
label: 'Linux-x64'
|
||||
@@ -94,6 +94,9 @@ jobs:
|
||||
- name: Install frontend dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Check version consistency
|
||||
run: npm run version:check
|
||||
|
||||
- name: Bundle Node.js runtime
|
||||
shell: bash
|
||||
env:
|
||||
@@ -189,6 +192,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: full
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
|
||||
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
|
||||
APPLE_SIGNING_IDENTITY: ${{ env.APPLE_SIGNING_IDENTITY }}
|
||||
@@ -212,6 +216,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: full
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
with:
|
||||
tagName: v__VERSION__
|
||||
releaseName: 'World Monitor v__VERSION__'
|
||||
@@ -229,6 +234,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: tech
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
|
||||
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
|
||||
APPLE_SIGNING_IDENTITY: ${{ env.APPLE_SIGNING_IDENTITY }}
|
||||
@@ -253,6 +259,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: tech
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
with:
|
||||
tagName: v__VERSION__-tech
|
||||
releaseName: 'Tech Monitor v__VERSION__'
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
name: Typecheck
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths-ignore:
|
||||
- '*.md'
|
||||
- '.planning/**'
|
||||
- 'docs/**'
|
||||
- 'e2e/**'
|
||||
- 'scripts/**'
|
||||
|
||||
jobs:
|
||||
typecheck:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
cache: 'npm'
|
||||
- run: npm ci
|
||||
- run: npm run typecheck
|
||||
+10
-1
@@ -1,6 +1,5 @@
|
||||
node_modules/
|
||||
.idea/
|
||||
.planning/
|
||||
dist/
|
||||
.DS_Store
|
||||
*.log
|
||||
@@ -8,6 +7,8 @@ dist/
|
||||
.env.local
|
||||
.playwright-mcp/
|
||||
.vercel
|
||||
api/\[domain\]/v1/\[rpc\].js
|
||||
api/\[\[...path\]\].js
|
||||
.claude/
|
||||
.cursor/
|
||||
CLAUDE.md
|
||||
@@ -18,6 +19,14 @@ CLAUDE.md
|
||||
.windsurf/
|
||||
skills/
|
||||
ideas/
|
||||
docs/internal/
|
||||
test-results/
|
||||
src-tauri/sidecar/node/*
|
||||
!src-tauri/sidecar/node/.gitkeep
|
||||
|
||||
# AI planning session state
|
||||
.planning/
|
||||
|
||||
# Compiled sebuf gateway bundle (built by scripts/build-sidecar-sebuf.mjs)
|
||||
api/[[][[].*.js
|
||||
.claudedocs/
|
||||
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
echo "Running type check..."
|
||||
npm run typecheck || exit 1
|
||||
|
||||
echo "Running Vite build (catches esbuild errors in server/)..."
|
||||
npm run build:full || exit 1
|
||||
|
||||
echo "Running version sync check..."
|
||||
npm run version:check || exit 1
|
||||
@@ -6,5 +6,5 @@
|
||||
"MD022": true,
|
||||
"MD032": true
|
||||
},
|
||||
"ignores": ["node_modules/**", "dist/**", "src-tauri/target/**"]
|
||||
"ignores": ["node_modules/**", "dist/**", "src-tauri/target/**", ".planning/**"]
|
||||
}
|
||||
|
||||
+127
@@ -2,6 +2,133 @@
|
||||
|
||||
All notable changes to World Monitor are documented here.
|
||||
|
||||
## [2.5.10] - 2026-02-26
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Yahoo Finance rate-limit UX**: Show "rate limited — retrying shortly" instead of generic "Failed to load" on Markets, ETF, Commodities, and Sector panels when Yahoo returns 429 (#407)
|
||||
- **Sequential Yahoo calls**: Replace `Promise.all` with staggered batching in commodity quotes, ETF flows, and macro signals to prevent 429 rate limiting (#406)
|
||||
- **Sector heatmap Yahoo fallback**: Sector data now loads via Yahoo Finance when `FINNHUB_API_KEY` is missing (#406)
|
||||
- **Finnhub-to-Yahoo fallback**: Market quotes route Finnhub symbols through Yahoo when API key is not configured (#407)
|
||||
- **ETF early-exit on rate limit**: Skip retry loop and show rate-limit message immediately instead of waiting 60s (#407)
|
||||
- **Sidecar auth resilience**: 401-retry with token refresh for stale sidecar tokens after restart; `diagFetch` auth helper for settings window diagnostics (#407)
|
||||
- **Verbose toggle persistence**: Write verbose state to writable data directory instead of read-only app bundle on macOS (#407)
|
||||
- **AI summary verbosity**: Tighten prompts to 2 sentences / 60 words max with `max_tokens` reduced from 150 to 100 (#404)
|
||||
- **Settings modal title**: Rename from "PANELS" to "SETTINGS" across all 17 locales (#403)
|
||||
- **Sentry noise filters**: CSS.escape() for news ID selectors, player.destroy guard, 11 new ignoreErrors patterns, blob: URL extension frame filter (#402)
|
||||
|
||||
---
|
||||
|
||||
## [2.5.6] - 2026-02-23
|
||||
|
||||
### Added
|
||||
|
||||
- **Greek (Ελληνικά) locale** — full translation of all 1,397 i18n keys (#256)
|
||||
- **Nigeria RSS feeds** — 5 new sources: Premium Times, Vanguard, Channels TV, Daily Trust, ThisDay Live
|
||||
- **Greek locale feeds** — Naftemporiki, in.gr, iefimerida.gr for Greek-language news coverage
|
||||
- **Brasil Paralelo source** — Brazilian news with RSS feed and source tier (#260)
|
||||
|
||||
### Performance
|
||||
|
||||
- **AIS relay optimization** — backpressure queue with configurable watermarks, spatial indexing for chokepoint detection (O(chokepoints) vs O(chokepoints × vessels)), pre-serialized + pre-gzipped snapshot cache eliminating per-request JSON.stringify + gzip CPU (#266)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Vietnam flag country code** — corrected flag emoji in language selector (#245)
|
||||
- **Sentry noise filters** — added patterns for SW FetchEvent, PostHog ingest; enabled SW POST method for PostHog analytics (#246)
|
||||
- **Service Worker same-origin routing** — restricted SW route patterns to same-origin only, preventing cross-origin fetch interception (#247, #251)
|
||||
- **Social preview bot allowlisting** — whitelisted Twitterbot, facebookexternalhit, and other crawlers on OG image assets (#251)
|
||||
- **Windows CORS for Tauri** — allow `http://` origin from `tauri.localhost` for Windows desktop builds (#262)
|
||||
- **Linux AppImage GLib crash** — fix GLib symbol mismatch on newer distros by bundling compatible libraries (#263)
|
||||
|
||||
---
|
||||
|
||||
## [2.5.2] - 2026-02-21
|
||||
|
||||
### Fixed
|
||||
|
||||
- **QuotaExceededError handling** — detect storage quota exhaustion and stop further writes to localStorage/IndexedDB instead of silently failing; shared `markStorageQuotaExceeded()` flag across persistent-cache and utility storage
|
||||
- **deck.gl null.getProjection crash** — wrap `setProps()` calls in try/catch to survive map mid-teardown races in debounced/RAF callbacks
|
||||
- **MapLibre "Style is not done loading"** — guard `setFilter()` in mousemove/mouseout handlers during theme switches
|
||||
- **YouTube invalid video ID** — validate video ID format (`/^[\w-]{10,12}$/`) before passing to IFrame Player constructor
|
||||
- **Vercel build skip on empty SHA** — guard `ignoreCommand` against unset `VERCEL_GIT_PREVIOUS_SHA` (first deploy, force deploy) which caused `git diff` to fail and cancel builds
|
||||
- **Sentry noise filters** — added 7 patterns: iOS readonly property, SW FetchEvent, toLowerCase/trim/indexOf injections, QuotaExceededError
|
||||
|
||||
---
|
||||
|
||||
## [2.5.1] - 2026-02-20
|
||||
|
||||
### Performance
|
||||
|
||||
- **Batch FRED API requests** — frontend now sends a single request with comma-separated series IDs instead of 7 parallel edge function invocations, eliminating Vercel 25s timeouts
|
||||
- **Parallel UCDP page fetches** — replaced sequential loop with Promise.all for up to 12 pages, cutting fetch time from ~96s worst-case to ~8s
|
||||
- **Bot protection middleware** — blocks known social-media crawlers from hitting API routes, reducing unnecessary edge function invocations
|
||||
- **Extended API cache TTLs** — country-intel 12h→24h, GDELT 2h→4h, nuclear 12h→24h; Vercel ignoreCommand skips non-code deploys
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Partial UCDP cache poisoning** — failed page fetches no longer silently produce incomplete results cached for 6h; partial results get 10-min TTL in both Redis and memory, with `partial: true` flag propagated to CDN cache headers
|
||||
- **FRED upstream error masking** — single-series failures now return 502 instead of empty 200; batch mode surfaces per-series errors and returns 502 when all fail
|
||||
- **Sentry `Load failed` filter** — widened regex from `^TypeError: Load failed$` to `^TypeError: Load failed( \(.*\))?$` to catch host-suffixed variants (e.g., gamma-api.polymarket.com)
|
||||
- **Tooltip XSS hardening** — replaced `rawHtml()` with `safeHtml()` allowlist sanitizer for panel info tooltips
|
||||
- **UCDP country endpoint** — added missing HTTP method guards (OPTIONS/GET)
|
||||
- **Middleware exact path matching** — social preview bot allowlist uses `Set.has()` instead of `startsWith()` prefix matching
|
||||
|
||||
### Changed
|
||||
|
||||
- FRED batch API supports up to 15 comma-separated series IDs with deduplication
|
||||
- Missing FRED API key returns 200 with `X-Data-Status: skipped-no-api-key` header instead of silent empty response
|
||||
- LAYER_TO_SOURCE config extracted from duplicate inline mappings into shared constant
|
||||
|
||||
---
|
||||
|
||||
## [2.5.0] - 2026-02-20
|
||||
|
||||
### Highlights
|
||||
|
||||
**Local LLM Support (Ollama / LM Studio)** — Run AI summarization entirely on your own hardware with zero cloud dependency. The desktop app auto-discovers models from any OpenAI-compatible local inference server (Ollama, LM Studio, llama.cpp, vLLM) and populates a selection dropdown. A 4-tier fallback chain ensures summaries always generate: Local LLM → Groq → OpenRouter → browser-side T5. Combined with the Tauri desktop app, this enables fully air-gapped intelligence analysis where no data leaves your machine.
|
||||
|
||||
### Added
|
||||
|
||||
- **Ollama / LM Studio integration** — local AI summarization via OpenAI-compatible `/v1/chat/completions` endpoint with automatic model discovery, embedding model filtering, and fallback to manual text input
|
||||
- **4-tier summarization fallback chain** — Ollama (local) → Groq (cloud) → OpenRouter (cloud) → Transformers.js T5 (browser), each with 5-second timeout before silently advancing to the next
|
||||
- **Shared summarization handler factory** — all three API tiers use identical logic for headline deduplication (Jaccard >0.6), variant-aware prompting, language-aware output, and Redis caching (`summary:v3:{mode}:{variant}:{lang}:{hash}`)
|
||||
- **Settings window with 3 tabs** — dedicated **LLMs** tab (Ollama endpoint/model, Groq, OpenRouter), **API Keys** tab (12+ data source credentials), and **Debug & Logs** tab (traffic log, verbose mode, log file access). Each tab runs an independent verification pipeline
|
||||
- **Consolidated keychain vault** — all desktop secrets stored as a single JSON blob in one OS keychain entry (`secrets-vault`), reducing macOS Keychain authorization prompts from 20+ to exactly 1 on app startup. One-time auto-migration from individual entries with cleanup
|
||||
- **Cross-window secret synchronization** — saving credentials in the Settings window immediately syncs to the main dashboard via `localStorage` broadcast, with no app restart needed
|
||||
- **API key verification pipeline** — each credential is validated against its provider's actual API endpoint. Network errors (timeouts, DNS failures) soft-pass to prevent transient failures from blocking key storage; only explicit 401/403 marks a key invalid
|
||||
- **Plaintext URL inputs** — endpoint URLs (Ollama API, relay URLs, model names) display as readable text instead of masked password dots in Settings
|
||||
- **5 new defense/intel RSS feeds** — Military Times, Task & Purpose, USNI News, Oryx OSINT, UK Ministry of Defence
|
||||
- **Koeberg nuclear power plant** — added to the nuclear facilities map layer (the only commercial reactor in Africa, Cape Town, South Africa)
|
||||
- **Privacy & Offline Architecture** documentation — README now details the three privacy levels: full cloud, desktop with cloud APIs, and air-gapped local with Ollama
|
||||
- **AI Summarization Chain** documentation — README includes provider fallback flow diagram and detailed explanation of headline deduplication, variant-aware prompting, and cross-user cache deduplication
|
||||
|
||||
### Changed
|
||||
|
||||
- AI fallback chain now starts with Ollama (local) before cloud providers
|
||||
- Feature toggles increased from 14 to 15 (added AI/Ollama)
|
||||
- Desktop architecture uses consolidated vault instead of per-key keychain entries
|
||||
- README expanded with ~85 lines of new content covering local LLM support, privacy architecture, summarization chain internals, and desktop readiness framework
|
||||
|
||||
### Fixed
|
||||
|
||||
- URL and model fields in Settings display as plaintext instead of masked password dots
|
||||
- OpenAI-compatible endpoint flow hardened for Ollama/LM Studio response format differences (thinking tokens, missing `choices` array edge cases)
|
||||
- Sentry null guard for `getProjection()` crash with 6 additional noise filters
|
||||
- PathLayer cache cleared on layer toggle-off to prevent stale WebGL buffer rendering
|
||||
|
||||
---
|
||||
|
||||
## [2.4.1] - 2026-02-19
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Map PathLayer cache**: Clear PathLayer on toggle-off to prevent stale WebGL buffers
|
||||
- **Sentry noise**: Null guard for `getProjection()` crash and 6 additional noise filters
|
||||
- **Markdown docs**: Resolve lint errors in documentation files
|
||||
|
||||
---
|
||||
|
||||
## [2.4.0] - 2026-02-19
|
||||
|
||||
### Added
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
# Contributor Covenant Code of Conduct
|
||||
|
||||
## Our Pledge
|
||||
|
||||
We as members, contributors, and leaders pledge to make participation in the
|
||||
World Monitor community a harassment-free experience for everyone, regardless of age, body
|
||||
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
||||
identity and expression, level of experience, education, socio-economic status,
|
||||
nationality, personal appearance, race, caste, color, religion, or sexual
|
||||
identity and orientation.
|
||||
|
||||
We pledge to act and interact in ways that contribute to an open, welcoming,
|
||||
diverse, inclusive, and healthy community.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Examples of behavior that contributes to a positive environment for our
|
||||
community include:
|
||||
|
||||
* Demonstrating empathy and kindness toward other people
|
||||
* Being respectful of differing opinions, viewpoints, and experiences
|
||||
* Giving and gracefully accepting constructive feedback
|
||||
* Accepting responsibility and apologizing to those affected by our mistakes,
|
||||
and learning from the experience
|
||||
* Focusing on what is best not just for us as individuals, but for the overall
|
||||
community
|
||||
|
||||
Examples of unacceptable behavior include:
|
||||
|
||||
* The use of sexualized language or imagery, and sexual attention or advances of
|
||||
any kind
|
||||
* Trolling, insulting or derogatory comments, and personal or political attacks
|
||||
* Public or private harassment
|
||||
* Publishing others' private information, such as a physical or email address,
|
||||
without their explicit permission
|
||||
* Other conduct which could reasonably be considered inappropriate in a
|
||||
professional setting
|
||||
|
||||
## Scope
|
||||
|
||||
This Code of Conduct applies within all community spaces (GitHub issues, pull
|
||||
requests, discussions, and any associated communication channels) and also
|
||||
applies when an individual is officially representing the community in public
|
||||
spaces.
|
||||
|
||||
## Enforcement
|
||||
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
||||
reported to the project maintainer at **[GitHub Issues](https://github.com/koala73/worldmonitor/issues)** or by contacting the
|
||||
repository owner directly through GitHub.
|
||||
|
||||
All complaints will be reviewed and investigated promptly and fairly. The project
|
||||
team is obligated to maintain confidentiality with regard to the reporter of an
|
||||
incident.
|
||||
|
||||
## Enforcement Guidelines
|
||||
|
||||
Community leaders will follow these Community Impact Guidelines in determining
|
||||
the consequences for any action they deem in violation of this Code of Conduct:
|
||||
|
||||
### 1. Correction
|
||||
|
||||
**Community Impact**: Use of inappropriate language or other behavior deemed
|
||||
unprofessional or unwelcome in the community.
|
||||
|
||||
**Consequence**: A private, written warning from community leaders, providing
|
||||
clarity around the nature of the violation and an explanation of why the
|
||||
behavior was inappropriate. A public apology may be requested.
|
||||
|
||||
### 2. Warning
|
||||
|
||||
**Community Impact**: A violation through a single incident or series of
|
||||
actions.
|
||||
|
||||
**Consequence**: A warning with consequences for continued behavior. No
|
||||
interaction with the people involved, including unsolicited interaction with
|
||||
those enforcing the Code of Conduct, for a specified period of time. This
|
||||
includes avoiding interactions in community spaces as well as external channels
|
||||
like social media. Violating these terms may lead to a temporary or permanent
|
||||
ban.
|
||||
|
||||
### 3. Temporary Ban
|
||||
|
||||
**Community Impact**: A serious violation of community standards, including
|
||||
sustained inappropriate behavior.
|
||||
|
||||
**Consequence**: A temporary ban from any sort of interaction or public
|
||||
communication with the community for a specified period of time. No public or
|
||||
private interaction with the people involved, including unsolicited interaction
|
||||
with those enforcing the Code of Conduct, is allowed during this period.
|
||||
Violating these terms may lead to a permanent ban.
|
||||
|
||||
### 4. Permanent Ban
|
||||
|
||||
**Community Impact**: Demonstrating a pattern of violation of community
|
||||
standards, including sustained inappropriate behavior, harassment of an
|
||||
individual, or aggression toward or disparagement of classes of individuals.
|
||||
|
||||
**Consequence**: A permanent ban from any sort of public interaction within the
|
||||
community.
|
||||
|
||||
## Attribution
|
||||
|
||||
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
||||
version 2.1, available at
|
||||
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
|
||||
|
||||
Community Impact Guidelines were inspired by
|
||||
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
|
||||
|
||||
For answers to common questions about this code of conduct, see the FAQ at
|
||||
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
|
||||
[https://www.contributor-covenant.org/translations][translations].
|
||||
|
||||
[homepage]: https://www.contributor-covenant.org
|
||||
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
|
||||
[Mozilla CoC]: https://github.com/mozilla/diversity
|
||||
[FAQ]: https://www.contributor-covenant.org/faq
|
||||
[translations]: https://www.contributor-covenant.org/translations
|
||||
+301
@@ -0,0 +1,301 @@
|
||||
# Contributing to World Monitor
|
||||
|
||||
Thank you for your interest in contributing to World Monitor! This project thrives on community contributions — whether it's code, data sources, documentation, or bug reports.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Architecture Overview](#architecture-overview)
|
||||
- [Getting Started](#getting-started)
|
||||
- [Development Setup](#development-setup)
|
||||
- [How to Contribute](#how-to-contribute)
|
||||
- [Pull Request Process](#pull-request-process)
|
||||
- [AI-Assisted Development](#ai-assisted-development)
|
||||
- [Coding Standards](#coding-standards)
|
||||
- [Working with Sebuf (RPC Framework)](#working-with-sebuf-rpc-framework)
|
||||
- [Adding Data Sources](#adding-data-sources)
|
||||
- [Adding RSS Feeds](#adding-rss-feeds)
|
||||
- [Reporting Bugs](#reporting-bugs)
|
||||
- [Feature Requests](#feature-requests)
|
||||
- [Code of Conduct](#code-of-conduct)
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
World Monitor is a real-time OSINT dashboard built with **Vanilla TypeScript** (no UI framework), **MapLibre GL + deck.gl** for map rendering, and a custom Proto-first RPC framework called **Sebuf** for all API communication.
|
||||
|
||||
### Key Technologies
|
||||
|
||||
| Technology | Purpose |
|
||||
|---|---|
|
||||
| **TypeScript** | All code — frontend, edge functions, and handlers |
|
||||
| **Vite** | Build tool and dev server |
|
||||
| **Sebuf** | Proto-first HTTP RPC framework for typed API contracts |
|
||||
| **Protobuf / Buf** | Service and message definitions across 17 domains |
|
||||
| **MapLibre GL** | Base map rendering (tiles, globe mode, camera) |
|
||||
| **deck.gl** | WebGL overlay layers (scatterplot, geojson, arcs, heatmaps) |
|
||||
| **d3** | Charts, sparklines, and data visualization |
|
||||
| **Vercel Edge Functions** | Serverless API gateway |
|
||||
| **Tauri v2** | Desktop app (Windows, macOS, Linux) |
|
||||
| **Convex** | Minimal backend (beta interest registration only) |
|
||||
| **Playwright** | End-to-end and visual regression testing |
|
||||
|
||||
### Variant System
|
||||
|
||||
The codebase produces three app variants from the same source, each targeting a different audience:
|
||||
|
||||
| Variant | Command | Focus |
|
||||
|---|---|---|
|
||||
| `full` | `npm run dev` | Geopolitics, military, conflicts, infrastructure |
|
||||
| `tech` | `npm run dev:tech` | Startups, AI/ML, cloud, cybersecurity |
|
||||
| `finance` | `npm run dev:finance` | Markets, trading, central banks, commodities |
|
||||
|
||||
Variants share all code but differ in default panels, map layers, and RSS feeds. Variant configs live in `src/config/variants/`.
|
||||
|
||||
### Directory Structure
|
||||
|
||||
| Directory | Purpose |
|
||||
|---|---|
|
||||
| `src/components/` | UI components — Panel subclasses, map, modals (~50 panels) |
|
||||
| `src/services/` | Data fetching modules — sebuf client wrappers, AI, signal analysis |
|
||||
| `src/config/` | Static data and variant configs (feeds, geo, military, pipelines, ports) |
|
||||
| `src/generated/` | Auto-generated sebuf client + server stubs (**do not edit by hand**) |
|
||||
| `src/types/` | TypeScript type definitions |
|
||||
| `src/locales/` | i18n JSON files (14 languages) |
|
||||
| `src/workers/` | Web Workers for analysis |
|
||||
| `server/` | Sebuf handler implementations for all 17 domain services |
|
||||
| `api/` | Vercel Edge Functions (sebuf gateway + legacy endpoints) |
|
||||
| `proto/` | Protobuf service and message definitions |
|
||||
| `data/` | Static JSON datasets |
|
||||
| `docs/` | Documentation + generated OpenAPI specs |
|
||||
| `src-tauri/` | Tauri v2 Rust app + Node.js sidecar for desktop builds |
|
||||
| `e2e/` | Playwright end-to-end tests |
|
||||
| `scripts/` | Build and packaging scripts |
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. **Fork** the repository on GitHub
|
||||
2. **Clone** your fork locally:
|
||||
```bash
|
||||
git clone https://github.com/<your-username>/worldmonitor.git
|
||||
cd worldmonitor
|
||||
```
|
||||
3. **Create a branch** for your work:
|
||||
```bash
|
||||
git checkout -b feature/your-feature-name
|
||||
```
|
||||
|
||||
## Development Setup
|
||||
|
||||
```bash
|
||||
# Install everything (buf CLI, sebuf plugins, npm deps, Playwright browsers)
|
||||
make install
|
||||
|
||||
# Start the development server (full variant, default)
|
||||
npm run dev
|
||||
|
||||
# Start other variants
|
||||
npm run dev:tech
|
||||
npm run dev:finance
|
||||
|
||||
# Run type checking
|
||||
npm run typecheck
|
||||
|
||||
# Run tests
|
||||
npm run test:data # Data integrity tests
|
||||
npm run test:e2e # Playwright end-to-end tests
|
||||
|
||||
# Production build (per variant)
|
||||
npm run build # full
|
||||
npm run build:tech
|
||||
npm run build:finance
|
||||
```
|
||||
|
||||
The dev server runs at `http://localhost:3000`. Run `make help` to see all available make targets.
|
||||
|
||||
### Environment Variables (Optional)
|
||||
|
||||
For full functionality, copy `.env.example` to `.env.local` and fill in the API keys you need. The app runs without any API keys — external data sources will simply be unavailable.
|
||||
|
||||
See [API Dependencies](docs/DOCUMENTATION.md#api-dependencies) for the full list.
|
||||
|
||||
## How to Contribute
|
||||
|
||||
### Types of Contributions We Welcome
|
||||
|
||||
- **Bug fixes** — found something broken? Fix it!
|
||||
- **New data layers** — add new geospatial data sources to the map
|
||||
- **RSS feeds** — expand our 100+ feed collection with quality sources
|
||||
- **UI/UX improvements** — make the dashboard more intuitive
|
||||
- **Performance optimizations** — faster loading, better caching
|
||||
- **Documentation** — improve docs, add examples, fix typos
|
||||
- **Accessibility** — make the dashboard usable by everyone
|
||||
- **Internationalization** — help make World Monitor available in more languages
|
||||
- **Tests** — add unit or integration tests
|
||||
|
||||
### What We're Especially Looking For
|
||||
|
||||
- New data layers (see [Adding Data Sources](#adding-data-sources))
|
||||
- Feed quality improvements and new RSS sources
|
||||
- Mobile responsiveness improvements
|
||||
- Performance optimizations for the map rendering pipeline
|
||||
- Better anomaly detection algorithms
|
||||
|
||||
## Pull Request Process
|
||||
|
||||
1. **Update documentation** if your change affects the public API or user-facing behavior
|
||||
2. **Run type checking** before submitting: `npm run typecheck`
|
||||
3. **Test your changes** locally with at least the `full` variant, and any other variant your change affects
|
||||
4. **Keep PRs focused** — one feature or fix per pull request
|
||||
5. **Write a clear description** explaining what your PR does and why
|
||||
6. **Link related issues** if applicable
|
||||
|
||||
### PR Title Convention
|
||||
|
||||
Use a descriptive title that summarizes the change:
|
||||
|
||||
- `feat: add earthquake magnitude filtering to map layer`
|
||||
- `fix: resolve RSS feed timeout for Al Jazeera`
|
||||
- `docs: update API dependencies section`
|
||||
- `perf: optimize marker clustering at low zoom levels`
|
||||
- `refactor: extract threat classifier into separate module`
|
||||
|
||||
### Review Process
|
||||
|
||||
- All PRs require review from a maintainer before merging
|
||||
- Maintainers may request changes — this is normal and collaborative
|
||||
- Once approved, a maintainer will merge your PR
|
||||
|
||||
## AI-Assisted Development
|
||||
|
||||
We fully embrace AI-assisted development. Many of our own PRs are labeled with the LLM that helped produce them (e.g., `claude`, `codex`, `cursor`), and contributors are welcome to use any AI tools they find helpful.
|
||||
|
||||
That said, **all code is held to the same quality bar regardless of how it was written**. AI-generated code will be reviewed with the same scrutiny as human-written code. Contributors are responsible for understanding and being able to explain every line they submit. Blindly pasting LLM output without review is discouraged — treat AI as a collaborator, not a replacement for your own judgement.
|
||||
|
||||
## Coding Standards
|
||||
|
||||
### TypeScript
|
||||
|
||||
- Use TypeScript for all new code
|
||||
- Avoid `any` types — use proper typing or `unknown` with type guards
|
||||
- Export interfaces/types for public APIs
|
||||
- Use meaningful variable and function names
|
||||
|
||||
### Code Style
|
||||
|
||||
- Follow the existing code style in the repository
|
||||
- Use `const` by default, `let` when reassignment is needed
|
||||
- Prefer functional patterns (map, filter, reduce) over imperative loops
|
||||
- Keep functions focused — one responsibility per function
|
||||
- Add JSDoc comments for exported functions and complex logic
|
||||
|
||||
### File Organization
|
||||
|
||||
- Static layer/geo data and variant configs go in `src/config/`
|
||||
- Sebuf handler implementations go in `server/worldmonitor/{domain}/v1/`
|
||||
- Edge function gateway and legacy endpoints go in `api/`
|
||||
- UI components (panels, map, modals) go in `src/components/`
|
||||
- Service modules (data fetching, client wrappers) go in `src/services/`
|
||||
- Proto definitions go in `proto/worldmonitor/{domain}/v1/`
|
||||
|
||||
## Working with Sebuf (RPC Framework)
|
||||
|
||||
Sebuf is the project's custom Proto-first HTTP RPC framework — a lightweight alternative to gRPC-Web. All API communication between client and server uses Sebuf.
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Proto definitions** in `proto/worldmonitor/{domain}/v1/` define services and messages
|
||||
2. **Code generation** (`make generate`) produces:
|
||||
- TypeScript clients in `src/generated/client/` (e.g., `MarketServiceClient`)
|
||||
- Server route factories in `src/generated/server/` (e.g., `createMarketServiceRoutes`)
|
||||
3. **Handlers** in `server/worldmonitor/{domain}/v1/handler.ts` implement the service interface
|
||||
4. **Gateway** in `api/[domain]/v1/[rpc].ts` registers all handlers and routes requests
|
||||
5. **Clients** in `src/services/{domain}/index.ts` wrap the generated client for app use
|
||||
|
||||
### Adding a New RPC Method
|
||||
|
||||
1. Add the method to the `.proto` service definition
|
||||
2. Run `make generate` to regenerate client/server stubs
|
||||
3. Implement the handler method in the domain's `handler.ts`
|
||||
4. The client stub is auto-generated — use it from `src/services/{domain}/`
|
||||
|
||||
Use `make lint` to lint proto files and `make breaking` to check for breaking changes against main.
|
||||
|
||||
### Proto Conventions
|
||||
|
||||
- **Time fields**: Use `int64` (Unix epoch milliseconds), not `google.protobuf.Timestamp`
|
||||
- **int64 encoding**: Apply `[(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER]` on time fields so TypeScript receives `number` instead of `string`
|
||||
- **HTTP annotations**: Every RPC method needs `option (sebuf.http.config) = { path: "...", method: POST }`
|
||||
|
||||
### Proto Codegen Requirements
|
||||
|
||||
Run `make install` to install everything automatically, or install individually:
|
||||
|
||||
```bash
|
||||
make install-buf # Install buf CLI (requires Go)
|
||||
make install-plugins # Install sebuf protoc-gen plugins (requires Go)
|
||||
```
|
||||
|
||||
## Adding Data Sources
|
||||
|
||||
To add a new data layer to the map:
|
||||
|
||||
1. **Define the data source** — identify the API or dataset you want to integrate
|
||||
2. **Add the proto service** (if the data needs a backend proxy) — define messages and RPC methods in `proto/worldmonitor/{domain}/v1/`
|
||||
3. **Generate stubs** — run `make generate`
|
||||
4. **Implement the handler** in `server/worldmonitor/{domain}/v1/`
|
||||
5. **Register the handler** in `api/[domain]/v1/[rpc].ts` and `vite.config.ts` (for local dev)
|
||||
6. **Create the service module** in `src/services/{domain}/` wrapping the generated client
|
||||
7. **Add the layer config** and implement the map renderer following existing layer patterns
|
||||
8. **Add to layer toggles** — make it toggleable in the UI
|
||||
9. **Document the source** — add it to `docs/DOCUMENTATION.md`
|
||||
|
||||
For endpoints that deal with non-JSON payloads (XML feeds, binary data, HTML embeds), you can add a standalone Edge Function in `api/` instead of Sebuf. For anything returning JSON, prefer Sebuf — the typed contracts are always worth it.
|
||||
|
||||
### Data Source Requirements
|
||||
|
||||
- Must be freely accessible (no paid-only APIs for core functionality)
|
||||
- Must have a permissive license or be public government data
|
||||
- Should update at least daily for real-time relevance
|
||||
- Must include geographic coordinates or be geo-locatable
|
||||
|
||||
## Adding RSS Feeds
|
||||
|
||||
To add new RSS feeds:
|
||||
|
||||
1. Verify the feed is reliable and actively maintained
|
||||
2. Assign a **source tier** (1-4) based on editorial reliability
|
||||
3. Flag any **state affiliation** or **propaganda risk**
|
||||
4. Categorize the feed (geopolitics, defense, energy, tech, etc.)
|
||||
5. Test that the feed parses correctly through the RSS proxy
|
||||
|
||||
## Reporting Bugs
|
||||
|
||||
When filing a bug report, please include:
|
||||
|
||||
- **Description** — clear description of the issue
|
||||
- **Steps to reproduce** — how to trigger the bug
|
||||
- **Expected behavior** — what should happen
|
||||
- **Actual behavior** — what actually happens
|
||||
- **Screenshots** — if applicable
|
||||
- **Browser/OS** — your environment details
|
||||
- **Console errors** — any relevant browser console output
|
||||
|
||||
Use the [Bug Report issue template](https://github.com/koala73/worldmonitor/issues/new/choose) when available.
|
||||
|
||||
## Feature Requests
|
||||
|
||||
We welcome feature ideas! When suggesting a feature:
|
||||
|
||||
- **Describe the problem** it solves
|
||||
- **Propose a solution** with as much detail as possible
|
||||
- **Consider alternatives** you've thought about
|
||||
- **Provide context** — who would benefit from this feature?
|
||||
|
||||
Use the [Feature Request issue template](https://github.com/koala73/worldmonitor/issues/new/choose) when available.
|
||||
|
||||
## Code of Conduct
|
||||
|
||||
This project follows the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior through GitHub issues or by contacting the repository owner.
|
||||
|
||||
---
|
||||
|
||||
Thank you for helping make World Monitor better! 🌍
|
||||
@@ -1,21 +1,669 @@
|
||||
MIT License
|
||||
World Monitor — Real-time global intelligence dashboard
|
||||
Copyright (C) 2024-2026 Elie Habib
|
||||
|
||||
Copyright (c) 2025-2026 Elie Habib
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU Affero General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
GNU AFFERO GENERAL PUBLIC LICENSE
|
||||
Version 3, 19 November 2007
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
Preamble
|
||||
|
||||
The GNU Affero General Public License is a free, copyleft license for
|
||||
software and other kinds of works, specifically designed to ensure
|
||||
cooperation with the community in the case of network server software.
|
||||
|
||||
The licenses for most software and other practical works are designed
|
||||
to take away your freedom to share and change the works. By contrast,
|
||||
our General Public Licenses are intended to guarantee your freedom to
|
||||
share and change all versions of a program--to make sure it remains free
|
||||
software for all its users.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not
|
||||
price. Our General Public Licenses are designed to make sure that you
|
||||
have the freedom to distribute copies of free software (and charge for
|
||||
them if you wish), that you receive source code or can get it if you
|
||||
want it, that you can change the software or use pieces of it in new
|
||||
free programs, and that you know you can do these things.
|
||||
|
||||
Developers that use our General Public Licenses protect your rights
|
||||
with two steps: (1) assert copyright on the software, and (2) offer
|
||||
you this License which gives you legal permission to copy, distribute
|
||||
and/or modify the software.
|
||||
|
||||
A secondary benefit of defending all users' freedom is that
|
||||
improvements made in alternate versions of the program, if they
|
||||
receive widespread use, become available for other developers to
|
||||
incorporate. Many developers of free software are heartened and
|
||||
encouraged by the resulting cooperation. However, in the case of
|
||||
software used on network servers, this result may fail to come about.
|
||||
The GNU General Public License permits making a modified version and
|
||||
letting the public access it on a server without ever releasing its
|
||||
source code to the public.
|
||||
|
||||
The GNU Affero General Public License is designed specifically to
|
||||
ensure that, in such cases, the modified source code becomes available
|
||||
to the community. It requires the operator of a network server to
|
||||
provide the source code of the modified version running there to the
|
||||
users of that server. Therefore, public use of a modified version, on
|
||||
a publicly accessible server, gives the public access to the source
|
||||
code of the modified version.
|
||||
|
||||
An older license, called the Affero General Public License and
|
||||
published by Affero, was designed to accomplish similar goals. This is
|
||||
a different license, not a version of the Affero GPL, but Affero has
|
||||
released a new version of the Affero GPL which permits relicensing under
|
||||
this license.
|
||||
|
||||
The precise terms and conditions for copying, distribution and
|
||||
modification follow.
|
||||
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
0. Definitions.
|
||||
|
||||
"This License" refers to version 3 of the GNU Affero General Public License.
|
||||
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||
works, such as semiconductor masks.
|
||||
|
||||
"The Program" refers to any copyrightable work licensed under this
|
||||
License. Each licensee is addressed as "you". "Licensees" and
|
||||
"recipients" may be individuals or organizations.
|
||||
|
||||
To "modify" a work means to copy from or adapt all or part of the work
|
||||
in a fashion requiring copyright permission, other than the making of an
|
||||
exact copy. The resulting work is called a "modified version" of the
|
||||
earlier work or a work "based on" the earlier work.
|
||||
|
||||
A "covered work" means either the unmodified Program or a work based
|
||||
on the Program.
|
||||
|
||||
To "propagate" a work means to do anything with it that, without
|
||||
permission, would make you directly or secondarily liable for
|
||||
infringement under applicable copyright law, except executing it on a
|
||||
computer or modifying a private copy. Propagation includes copying,
|
||||
distribution (with or without modification), making available to the
|
||||
public, and in some countries other activities as well.
|
||||
|
||||
To "convey" a work means any kind of propagation that enables other
|
||||
parties to make or receive copies. Mere interaction with a user through
|
||||
a computer network, with no transfer of a copy, is not conveying.
|
||||
|
||||
An interactive user interface displays "Appropriate Legal Notices"
|
||||
to the extent that it includes a convenient and prominently visible
|
||||
feature that (1) displays an appropriate copyright notice, and (2)
|
||||
tells the user that there is no warranty for the work (except to the
|
||||
extent that warranties are provided), that licensees may convey the
|
||||
work under this License, and how to view a copy of this License. If
|
||||
the interface presents a list of user commands or options, such as a
|
||||
menu, a prominent item in the list meets this criterion.
|
||||
|
||||
1. Source Code.
|
||||
|
||||
The "source code" for a work means the preferred form of the work
|
||||
for making modifications to it. "Object code" means any non-source
|
||||
form of a work.
|
||||
|
||||
A "Standard Interface" means an interface that either is an official
|
||||
standard defined by a recognized standards body, or, in the case of
|
||||
interfaces specified for a particular programming language, one that
|
||||
is widely used among developers working in that language.
|
||||
|
||||
The "System Libraries" of an executable work include anything, other
|
||||
than the work as a whole, that (a) is included in the normal form of
|
||||
packaging a Major Component, but which is not part of that Major
|
||||
Component, and (b) serves only to enable use of the work with that
|
||||
Major Component, or to implement a Standard Interface for which an
|
||||
implementation is available to the public in source code form. A
|
||||
"Major Component", in this context, means a major essential component
|
||||
(kernel, window system, and so on) of the specific operating system
|
||||
(if any) on which the executable work runs, or a compiler used to
|
||||
produce the work, or an object code interpreter used to run it.
|
||||
|
||||
The "Corresponding Source" for a work in object code form means all
|
||||
the source code needed to generate, install, and (for an executable
|
||||
work) run the object code and to modify the work, including scripts to
|
||||
control those activities. However, it does not include the work's
|
||||
System Libraries, or general-purpose tools or generally available free
|
||||
programs which are used unmodified in performing those activities but
|
||||
which are not part of the work. For example, Corresponding Source
|
||||
includes interface definition files associated with source files for
|
||||
the work, and the source code for shared libraries and dynamically
|
||||
linked subprograms that the work is specifically designed to require,
|
||||
such as by intimate data communication or control flow between those
|
||||
subprograms and other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users
|
||||
can regenerate automatically from other parts of the Corresponding
|
||||
Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that
|
||||
same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
|
||||
All rights granted under this License are granted for the term of
|
||||
copyright on the Program, and are irrevocable provided the stated
|
||||
conditions are met. This License explicitly affirms your unlimited
|
||||
permission to run the unmodified Program. The output from running a
|
||||
covered work is covered by this License only if the output, given its
|
||||
content, constitutes a covered work. This License acknowledges your
|
||||
rights of fair use or other equivalent, as provided by copyright law.
|
||||
|
||||
You may make, run and propagate covered works that you do not
|
||||
convey, without conditions so long as your license otherwise remains
|
||||
in force. You may convey covered works to others for the sole purpose
|
||||
of having them make modifications exclusively for you, or provide you
|
||||
with facilities for running those works, provided that you comply with
|
||||
the terms of this License in conveying all material for which you do
|
||||
not control copyright. Those thus making or running the covered works
|
||||
for you must do so exclusively on your behalf, under your direction
|
||||
and control, on terms that prohibit them from making any copies of
|
||||
your copyrighted material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under
|
||||
the conditions stated below. Sublicensing is not allowed; section 10
|
||||
makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
|
||||
No covered work shall be deemed part of an effective technological
|
||||
measure under any applicable law fulfilling obligations under article
|
||||
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||
similar laws prohibiting or restricting circumvention of such
|
||||
measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid
|
||||
circumvention of technological measures to the extent such circumvention
|
||||
is effected by exercising rights under this License with respect to
|
||||
the covered work, and you disclaim any intention to limit operation or
|
||||
modification of the work as a means of enforcing, against the work's
|
||||
users, your or third parties' legal rights to forbid circumvention of
|
||||
technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
|
||||
You may convey verbatim copies of the Program's source code as you
|
||||
receive it, in any medium, provided that you conspicuously and
|
||||
appropriately publish on each copy an appropriate copyright notice;
|
||||
keep intact all notices stating that this License and any
|
||||
non-permissive terms added in accord with section 7 apply to the code;
|
||||
keep intact all notices of the absence of any warranty; and give all
|
||||
recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey,
|
||||
and you may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
|
||||
You may convey a work based on the Program, or the modifications to
|
||||
produce it from the Program, in the form of source code under the
|
||||
terms of section 4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified
|
||||
it, and giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is
|
||||
released under this License and any conditions added under section
|
||||
7. This requirement modifies the requirement in section 4 to
|
||||
"keep intact all notices".
|
||||
|
||||
c) You must license the entire work, as a whole, under this
|
||||
License to anyone who comes into possession of a copy. This
|
||||
License will therefore apply, along with any applicable section 7
|
||||
additional terms, to the whole of the work, and all its parts,
|
||||
regardless of how they are packaged. This License gives no
|
||||
permission to license the work in any other way, but it does not
|
||||
invalidate such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display
|
||||
Appropriate Legal Notices; however, if the Program has interactive
|
||||
interfaces that do not display Appropriate Legal Notices, your
|
||||
work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent
|
||||
works, which are not by their nature extensions of the covered work,
|
||||
and which are not combined with it such as to form a larger program,
|
||||
in or on a volume of a storage or distribution medium, is called an
|
||||
"aggregate" if the compilation and its resulting copyright are not
|
||||
used to limit the access or legal rights of the compilation's users
|
||||
beyond what the individual works permit. Inclusion of a covered work
|
||||
in an aggregate does not cause this License to apply to the other
|
||||
parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
|
||||
You may convey a covered work in object code form under the terms
|
||||
of sections 4 and 5, provided that you also convey the
|
||||
machine-readable Corresponding Source under the terms of this License,
|
||||
in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by the
|
||||
Corresponding Source fixed on a durable physical medium
|
||||
customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by a
|
||||
written offer, valid for at least three years and valid for as
|
||||
long as you offer spare parts or customer support for that product
|
||||
model, to give anyone who possesses the object code either (1) a
|
||||
copy of the Corresponding Source for all the software in the
|
||||
product that is covered by this License, on a durable physical
|
||||
medium customarily used for software interchange, for a price no
|
||||
more than your reasonable cost of physically performing this
|
||||
conveying of source, or (2) access to copy the
|
||||
Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the
|
||||
written offer to provide the Corresponding Source. This
|
||||
alternative is allowed only occasionally and noncommercially, and
|
||||
only if you received the object code with such an offer, in accord
|
||||
with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated
|
||||
place (gratis or for a charge), and offer equivalent access to the
|
||||
Corresponding Source in the same way through the same place at no
|
||||
further charge. You need not require recipients to copy the
|
||||
Corresponding Source along with the object code. If the place to
|
||||
copy the object code is a network server, the Corresponding Source
|
||||
may be on a different server (operated by you or a third party)
|
||||
that supports equivalent copying facilities, provided you maintain
|
||||
clear directions next to the object code saying where to find the
|
||||
Corresponding Source. Regardless of what server hosts the
|
||||
Corresponding Source, you remain obligated to ensure that it is
|
||||
available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided
|
||||
you inform other peers where the object code and Corresponding
|
||||
Source of the work are being offered to the general public at no
|
||||
charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded
|
||||
from the Corresponding Source as a System Library, need not be
|
||||
included in conveying the object code work.
|
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any
|
||||
tangible personal property which is normally used for personal, family,
|
||||
or household purposes, or (2) anything designed or sold for incorporation
|
||||
into a dwelling. In determining whether a product is a consumer product,
|
||||
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||
product received by a particular user, "normally used" refers to a
|
||||
typical or common use of that class of product, regardless of the status
|
||||
of the particular user or of the way in which the particular user
|
||||
actually uses, or expects or is expected to use, the product. A product
|
||||
is a consumer product regardless of whether the product has substantial
|
||||
commercial, industrial or non-consumer uses, unless such uses represent
|
||||
the only significant mode of use of the product.
|
||||
|
||||
"Installation Information" for a User Product means any methods,
|
||||
procedures, authorization keys, or other information required to install
|
||||
and execute modified versions of a covered work in that User Product from
|
||||
a modified version of its Corresponding Source. The information must
|
||||
suffice to ensure that the continued functioning of the modified object
|
||||
code is in no case prevented or interfered with solely because
|
||||
modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or
|
||||
specifically for use in, a User Product, and the conveying occurs as
|
||||
part of a transaction in which the right of possession and use of the
|
||||
User Product is transferred to the recipient in perpetuity or for a
|
||||
fixed term (regardless of how the transaction is characterized), the
|
||||
Corresponding Source conveyed under this section must be accompanied
|
||||
by the Installation Information. But this requirement does not apply
|
||||
if neither you nor any third party retains the ability to install
|
||||
modified object code on the User Product (for example, the work has
|
||||
been installed in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a
|
||||
requirement to continue to provide support service, warranty, or updates
|
||||
for a work that has been modified or installed by the recipient, or for
|
||||
the User Product in which it has been modified or installed. Access to a
|
||||
network may be denied when the modification itself materially and
|
||||
adversely affects the operation of the network or violates the rules and
|
||||
protocols for communication across the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided,
|
||||
in accord with this section must be in a format that is publicly
|
||||
documented (and with an implementation available to the public in
|
||||
source code form), and must require no special password or key for
|
||||
unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
|
||||
"Additional permissions" are terms that supplement the terms of this
|
||||
License by making exceptions from one or more of its conditions.
|
||||
Additional permissions that are applicable to the entire Program shall
|
||||
be treated as though they were included in this License, to the extent
|
||||
that they are valid under applicable law. If additional permissions
|
||||
apply only to part of the Program, that part may be used separately
|
||||
under those permissions, but the entire Program remains governed by
|
||||
this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option
|
||||
remove any additional permissions from that copy, or from any part of
|
||||
it. (Additional permissions may be written to require their own
|
||||
removal in certain cases when you modify the work.) You may place
|
||||
additional permissions on material, added by you to a covered work,
|
||||
for which you have or can give appropriate copyright permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you
|
||||
add to a covered work, you may (if authorized by the copyright holders of
|
||||
that material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the
|
||||
terms of sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or
|
||||
author attributions in that material or in the Appropriate Legal
|
||||
Notices displayed by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or
|
||||
requiring that modified versions of such material be marked in
|
||||
reasonable ways as different from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or
|
||||
authors of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some
|
||||
trade names, trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that
|
||||
material by anyone who conveys the material (or modified versions of
|
||||
it) with contractual assumptions of liability to the recipient, for
|
||||
any liability that these contractual assumptions directly impose on
|
||||
those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered "further
|
||||
restrictions" within the meaning of section 10. If the Program as you
|
||||
received it, or any part of it, contains a notice stating that it is
|
||||
governed by this License along with a term that is a further
|
||||
restriction, you may remove that term. If a license document contains
|
||||
a further restriction but permits relicensing or conveying under this
|
||||
License, you may add to a covered work material governed by the terms
|
||||
of that license document, provided that the further restriction does
|
||||
not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you
|
||||
must place, in the relevant source files, a statement of the
|
||||
additional terms that apply to those files, or a notice indicating
|
||||
where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the
|
||||
form of a separately written license, or stated as exceptions;
|
||||
the above requirements apply either way.
|
||||
|
||||
8. Termination.
|
||||
|
||||
You may not propagate or modify a covered work except as expressly
|
||||
provided under this License. Any attempt otherwise to propagate or
|
||||
modify it is void, and will automatically terminate your rights under
|
||||
this License (including any patent licenses granted under the third
|
||||
paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your
|
||||
license from a particular copyright holder is reinstated (a)
|
||||
provisionally, unless and until the copyright holder explicitly and
|
||||
finally terminates your license, and (b) permanently, if the copyright
|
||||
holder fails to notify you of the violation by some reasonable means
|
||||
prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is
|
||||
reinstated permanently if the copyright holder notifies you of the
|
||||
violation by some reasonable means, this is the first time you have
|
||||
received notice of violation of this License (for any work) from that
|
||||
copyright holder, and you cure the violation prior to 30 days after
|
||||
your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the
|
||||
licenses of parties who have received copies or rights from you under
|
||||
this License. If your rights have been terminated and not permanently
|
||||
reinstated, you do not qualify to receive new licenses for the same
|
||||
material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
|
||||
You are not required to accept this License in order to receive or
|
||||
run a copy of the Program. Ancillary propagation of a covered work
|
||||
occurring solely as a consequence of using peer-to-peer transmission
|
||||
to receive a copy likewise does not require acceptance. However,
|
||||
nothing other than this License grants you permission to propagate or
|
||||
modify any covered work. These actions infringe copyright if you do
|
||||
not accept this License. Therefore, by modifying or propagating a
|
||||
covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
|
||||
Each time you convey a covered work, the recipient automatically
|
||||
receives a license from the original licensors, to run, modify and
|
||||
propagate that work, subject to this License. You are not responsible
|
||||
for enforcing compliance by third parties with this License.
|
||||
|
||||
An "entity transaction" is a transaction transferring control of an
|
||||
organization, or substantially all assets of one, or subdividing an
|
||||
organization, or merging organizations. If propagation of a covered
|
||||
work results from an entity transaction, each party to that
|
||||
transaction who receives a copy of the work also receives whatever
|
||||
licenses to the work the party's predecessor in interest had or could
|
||||
give under the previous paragraph, plus a right to possession of the
|
||||
Corresponding Source of the work from the predecessor in interest, if
|
||||
the predecessor has it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the
|
||||
rights granted or affirmed under this License. For example, you may
|
||||
not impose a license fee, royalty, or other charge for exercise of
|
||||
rights granted under this License, and you may not initiate litigation
|
||||
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||
any patent claim is infringed by making, using, selling, offering for
|
||||
sale, or importing the Program or any portion of it.
|
||||
|
||||
11. Patents.
|
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this
|
||||
License of the Program or a work on which the Program is based. The
|
||||
work thus licensed is called the contributor's "contributor version".
|
||||
|
||||
A contributor's "essential patent claims" are all patent claims
|
||||
owned or controlled by the contributor, whether already acquired or
|
||||
hereafter acquired, that would be infringed by some manner, permitted
|
||||
by this License, of making, using, or selling its contributor version,
|
||||
but do not include claims that would be infringed only as a
|
||||
consequence of further modification of the contributor version. For
|
||||
purposes of this definition, "control" includes the right to grant
|
||||
patent sublicenses in a manner consistent with the requirements of
|
||||
this License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||
patent license under the contributor's essential patent claims, to
|
||||
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||
propagate the contents of its contributor version.
|
||||
|
||||
In the following three paragraphs, a "patent license" is any express
|
||||
agreement or commitment, however denominated, not to enforce a patent
|
||||
(such as an express permission to practice a patent or covenant not to
|
||||
sue for patent infringement). To "grant" such a patent license to a
|
||||
party means to make such an agreement or commitment not to enforce a
|
||||
patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license,
|
||||
and the Corresponding Source of the work is not available for anyone
|
||||
to copy, free of charge and under the terms of this License, through a
|
||||
publicly available network server or other readily accessible means,
|
||||
then you must either (1) cause the Corresponding Source to be so
|
||||
available, or (2) arrange to deprive yourself of the benefit of the
|
||||
patent license for this particular work, or (3) arrange, in a manner
|
||||
consistent with the requirements of this License, to extend the patent
|
||||
license to downstream recipients. "Knowingly relying" means you have
|
||||
actual knowledge that, but for the patent license, your conveying the
|
||||
covered work in a country, or your recipient's use of the covered work
|
||||
in a country, would infringe one or more identifiable patents in that
|
||||
country that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or
|
||||
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||
covered work, and grant a patent license to some of the parties
|
||||
receiving the covered work authorizing them to use, propagate, modify
|
||||
or convey a specific copy of the covered work, then the patent license
|
||||
you grant is automatically extended to all recipients of the covered
|
||||
work and works based on it.
|
||||
|
||||
A patent license is "discriminatory" if it does not include within
|
||||
the scope of its coverage, prohibits the exercise of, or is
|
||||
conditioned on the non-exercise of one or more of the rights that are
|
||||
specifically granted under this License. You may not convey a covered
|
||||
work if you are a party to an arrangement with a third party that is
|
||||
in the business of distributing software, under which you make payment
|
||||
to the third party based on the extent of your activity of conveying
|
||||
the work, and under which the third party grants, to any of the
|
||||
parties who would receive the covered work from you, a discriminatory
|
||||
patent license (a) in connection with copies of the covered work
|
||||
conveyed by you (or copies made from those copies), or (b) primarily
|
||||
for and in connection with specific products or compilations that
|
||||
contain the covered work, unless you entered into that arrangement,
|
||||
or that patent license was granted, prior to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting
|
||||
any implied license or other defenses to infringement that may
|
||||
otherwise be available to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or
|
||||
otherwise) that contradict the conditions of this License, they do not
|
||||
excuse you from the conditions of this License. If you cannot convey a
|
||||
covered work so as to satisfy simultaneously your obligations under this
|
||||
License and any other pertinent obligations, then as a consequence you may
|
||||
not convey it at all. For example, if you agree to terms that obligate you
|
||||
to collect a royalty for further conveying from those to whom you convey
|
||||
the Program, the only way you could satisfy both those terms and this
|
||||
License would be to refrain entirely from conveying the Program.
|
||||
|
||||
13. Remote Network Interaction; Use with the GNU General Public License.
|
||||
|
||||
Notwithstanding any other provision of this License, if you modify the
|
||||
Program, your modified version must prominently offer all users
|
||||
interacting with it remotely through a computer network (if your version
|
||||
supports such interaction) an opportunity to receive the Corresponding
|
||||
Source of your version by providing access to the Corresponding Source
|
||||
from a network server at no charge, through some standard or customary
|
||||
means of facilitating copying of software. This Corresponding Source
|
||||
shall include the Corresponding Source for any work covered by version 3
|
||||
of the GNU General Public License that is incorporated pursuant to the
|
||||
following paragraph.
|
||||
|
||||
Notwithstanding any other provision of this License, you have
|
||||
permission to link or combine any covered work with a work licensed
|
||||
under version 3 of the GNU General Public License into a single
|
||||
combined work, and to convey the resulting work. The terms of this
|
||||
License will continue to apply to the part which is the covered work,
|
||||
but the work with which it is combined will remain governed by version
|
||||
3 of the GNU General Public License.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of
|
||||
the GNU Affero General Public License from time to time. Such new versions
|
||||
will be similar in spirit to the present version, but may differ in detail to
|
||||
address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the
|
||||
Program specifies that a certain numbered version of the GNU Affero General
|
||||
Public License "or any later version" applies to it, you have the
|
||||
option of following the terms and conditions either of that numbered
|
||||
version or of any later version published by the Free Software
|
||||
Foundation. If the Program does not specify a version number of the
|
||||
GNU Affero General Public License, you may choose any version ever published
|
||||
by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future
|
||||
versions of the GNU Affero General Public License can be used, that proxy's
|
||||
public statement of acceptance of a version permanently authorizes you
|
||||
to choose that version for the Program.
|
||||
|
||||
Later license versions may give you additional or different
|
||||
permissions. However, no additional obligations are imposed on any
|
||||
author or copyright holder as a result of your choosing to follow a
|
||||
later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||
SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
|
||||
If the disclaimer of warranty and limitation of liability provided
|
||||
above cannot be given local legal effect according to their terms,
|
||||
reviewing courts shall apply local law that most closely approximates
|
||||
an absolute waiver of all civil liability in connection with the
|
||||
Program, unless a warranty or assumption of liability accompanies a
|
||||
copy of the Program in return for a fee.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest
|
||||
possible use to the public, the best way to achieve this is to make it
|
||||
free software which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest
|
||||
to attach them to the start of each source file to most effectively
|
||||
state the exclusion of warranty; and each file should have at least
|
||||
the "copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU Affero General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
This program is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU Affero General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU Affero General Public License
|
||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If your software can interact with users remotely through a computer
|
||||
network, you should also make sure that it provides a way for users to
|
||||
get its source. For example, if your program is a web application, its
|
||||
interface could display a "Source" link that leads users to an archive
|
||||
of the code. There are many ways you could offer source, and different
|
||||
solutions will be better for different programs; see section 13 for the
|
||||
specific requirements.
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school,
|
||||
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||
For more information on this, and how to apply and follow the GNU AGPL, see
|
||||
<https://www.gnu.org/licenses/>.
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
.PHONY: help lint generate breaking format check clean deps install install-buf install-plugins install-npm install-playwright
|
||||
.DEFAULT_GOAL := help
|
||||
|
||||
# Variables
|
||||
PROTO_DIR := proto
|
||||
GEN_CLIENT_DIR := src/generated/client
|
||||
GEN_SERVER_DIR := src/generated/server
|
||||
DOCS_API_DIR := docs/api
|
||||
|
||||
# Go install settings
|
||||
GO_PROXY := GOPROXY=direct
|
||||
GO_PRIVATE := GOPRIVATE=github.com/SebastienMelki
|
||||
GO_INSTALL := $(GO_PROXY) $(GO_PRIVATE) go install
|
||||
|
||||
# Required tool versions
|
||||
BUF_VERSION := v1.64.0
|
||||
SEBUF_VERSION := v0.7.0
|
||||
|
||||
help: ## Show this help message
|
||||
@echo 'Usage: make [target]'
|
||||
@echo ''
|
||||
@echo 'Targets:'
|
||||
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf " %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
|
||||
|
||||
install: install-buf install-plugins install-npm install-playwright deps ## Install everything (buf, sebuf plugins, npm deps, proto deps, browsers)
|
||||
|
||||
install-buf: ## Install buf CLI
|
||||
@if command -v buf >/dev/null 2>&1; then \
|
||||
echo "buf already installed: $$(buf --version)"; \
|
||||
else \
|
||||
echo "Installing buf..."; \
|
||||
$(GO_INSTALL) github.com/bufbuild/buf/cmd/buf@$(BUF_VERSION); \
|
||||
echo "buf installed!"; \
|
||||
fi
|
||||
|
||||
install-plugins: ## Install sebuf protoc plugins (requires Go)
|
||||
@echo "Installing sebuf protoc plugins $(SEBUF_VERSION)..."
|
||||
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-ts-client@$(SEBUF_VERSION)
|
||||
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-ts-server@$(SEBUF_VERSION)
|
||||
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-openapiv3@$(SEBUF_VERSION)
|
||||
@echo "Plugins installed!"
|
||||
|
||||
install-npm: ## Install npm dependencies
|
||||
npm install
|
||||
|
||||
install-playwright: ## Install Playwright browsers for e2e tests
|
||||
npx playwright install chromium
|
||||
|
||||
deps: ## Install/update buf proto dependencies
|
||||
cd $(PROTO_DIR) && buf dep update
|
||||
|
||||
lint: ## Lint protobuf files
|
||||
cd $(PROTO_DIR) && buf lint
|
||||
|
||||
generate: clean ## Generate code from proto definitions
|
||||
@mkdir -p $(GEN_CLIENT_DIR) $(GEN_SERVER_DIR) $(DOCS_API_DIR)
|
||||
cd $(PROTO_DIR) && buf generate
|
||||
@echo "Code generation complete!"
|
||||
|
||||
breaking: ## Check for breaking changes against main
|
||||
cd $(PROTO_DIR) && buf breaking --against '.git#branch=main,subdir=proto'
|
||||
|
||||
format: ## Format protobuf files
|
||||
cd $(PROTO_DIR) && buf format -w
|
||||
|
||||
check: lint generate ## Run all checks (lint + generate)
|
||||
|
||||
clean: ## Clean generated files
|
||||
@rm -rf $(GEN_CLIENT_DIR)
|
||||
@rm -rf $(GEN_SERVER_DIR)
|
||||
@rm -rf $(DOCS_API_DIR)
|
||||
@echo "Clean complete!"
|
||||
+112
@@ -0,0 +1,112 @@
|
||||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
| Version | Supported |
|
||||
| ------- | ------------------ |
|
||||
| main | :white_check_mark: |
|
||||
|
||||
Only the latest version on the `main` branch is actively maintained and receives security updates.
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
**Please do NOT report security vulnerabilities through public GitHub issues.**
|
||||
|
||||
If you discover a security vulnerability in World Monitor, please report it responsibly:
|
||||
|
||||
1. **GitHub Private Vulnerability Reporting**: Use [GitHub's private vulnerability reporting](https://github.com/koala73/worldmonitor/security/advisories/new) to submit your report directly through the repository.
|
||||
|
||||
2. **Direct Contact**: Alternatively, reach out to the repository owner [@koala73](https://github.com/koala73) directly through GitHub.
|
||||
|
||||
### What to Include
|
||||
|
||||
- A description of the vulnerability and its potential impact
|
||||
- Steps to reproduce the issue
|
||||
- Affected components (edge functions, client-side code, data layers, etc.)
|
||||
- Any potential fixes or mitigations you've identified
|
||||
|
||||
### Response Timeline
|
||||
|
||||
- **Acknowledgment**: Within 48 hours of your report
|
||||
- **Initial Assessment**: Within 1 week
|
||||
- **Fix/Patch**: Depending on severity, critical issues will be prioritized
|
||||
|
||||
### What to Expect
|
||||
|
||||
- You will receive an acknowledgment of your report
|
||||
- We will work with you to understand and validate the issue
|
||||
- We will keep you informed of progress toward a fix
|
||||
- Credit will be given to reporters in the fix commit (unless you prefer anonymity)
|
||||
|
||||
## Security Considerations
|
||||
|
||||
World Monitor is a client-side intelligence dashboard that aggregates publicly available data. Here are the key security areas:
|
||||
|
||||
### API Keys & Secrets
|
||||
|
||||
- **Web deployment**: API keys are stored server-side in Vercel Edge Functions
|
||||
- **Desktop runtime**: API keys are stored in the OS keychain (macOS Keychain / Windows Credential Manager) via a consolidated vault entry, never on disk in plaintext
|
||||
- No API keys should ever be committed to the repository
|
||||
- Environment variables (`.env.local`) are gitignored
|
||||
- The RSS proxy uses domain allowlisting to prevent SSRF
|
||||
|
||||
### Edge Functions & Sebuf Handlers
|
||||
|
||||
- All 17 domain APIs are served through Sebuf (a Proto-first RPC framework) via Vercel Edge Functions
|
||||
- Edge functions and handlers should validate/sanitize all input
|
||||
- CORS headers are configured per-function
|
||||
- Rate limiting and circuit breakers protect against abuse
|
||||
|
||||
### Client-Side Security
|
||||
|
||||
- No sensitive data is stored in localStorage or sessionStorage
|
||||
- External content (RSS feeds, news) is sanitized before rendering
|
||||
- Map data layers use trusted, vetted data sources
|
||||
- Content Security Policy restricts script-src to `'self'` (no unsafe-inline/eval)
|
||||
|
||||
### Desktop Runtime Security (Tauri)
|
||||
|
||||
- **IPC origin validation**: Sensitive Tauri commands (secrets, cache, token) are gated to trusted windows only; external-origin windows (e.g., YouTube login) are blocked
|
||||
- **DevTools**: Disabled in production builds; gated behind an opt-in Cargo feature for development
|
||||
- **Sidecar authentication**: A per-session CSPRNG token (`LOCAL_API_TOKEN`) authenticates all renderer-to-sidecar requests, preventing other local processes from accessing the API
|
||||
- **Capability isolation**: The YouTube login window runs under a restricted capability with no access to secret or cache IPC commands
|
||||
- **Fetch patch trust boundary**: The global fetch interceptor injects the sidecar token with a 5-minute TTL; the renderer is the intended client — if renderer integrity is compromised, Tauri IPC provides strictly more access than the fetch patch
|
||||
|
||||
### Data Sources
|
||||
|
||||
- World Monitor aggregates publicly available OSINT data
|
||||
- No classified or restricted data sources are used
|
||||
- State-affiliated sources are flagged with propaganda risk ratings
|
||||
- All data is consumed read-only — the platform does not modify upstream sources
|
||||
|
||||
## Scope
|
||||
|
||||
The following are **in scope** for security reports:
|
||||
|
||||
- Vulnerabilities in the World Monitor codebase
|
||||
- Edge function security issues (SSRF, injection, auth bypass)
|
||||
- XSS or content injection through RSS feeds or external data
|
||||
- API key exposure or secret leakage
|
||||
- Tauri IPC command privilege escalation or capability bypass
|
||||
- Sidecar authentication bypass or token leakage
|
||||
- Dependency vulnerabilities with a viable attack vector
|
||||
|
||||
The following are **out of scope**:
|
||||
|
||||
- Vulnerabilities in third-party services we consume (report to the upstream provider)
|
||||
- Social engineering attacks
|
||||
- Denial of service attacks
|
||||
- Issues in forked copies of the repository
|
||||
- Security issues in user-provided environment configurations
|
||||
|
||||
## Best Practices for Contributors
|
||||
|
||||
- Never commit API keys, tokens, or secrets
|
||||
- Use environment variables for all sensitive configuration
|
||||
- Sanitize external input in edge functions
|
||||
- Keep dependencies updated — run `npm audit` regularly
|
||||
- Follow the principle of least privilege for API access
|
||||
|
||||
---
|
||||
|
||||
Thank you for helping keep World Monitor and its users safe! 🔒
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,150 @@
|
||||
/**
|
||||
* Vercel edge function for sebuf RPC routes.
|
||||
*
|
||||
* Matches /api/{domain}/v1/{rpc} via Vercel dynamic segment routing.
|
||||
* CORS headers are applied to every response (200, 204, 403, 404).
|
||||
*/
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { createRouter } from '../../../server/router';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../../../server/cors';
|
||||
// @ts-expect-error — JS module, no declaration file
|
||||
import { validateApiKey } from '../../_api-key.js';
|
||||
import { mapErrorToResponse } from '../../../server/error-mapper';
|
||||
import { createSeismologyServiceRoutes } from '../../../src/generated/server/worldmonitor/seismology/v1/service_server';
|
||||
import { seismologyHandler } from '../../../server/worldmonitor/seismology/v1/handler';
|
||||
import { createWildfireServiceRoutes } from '../../../src/generated/server/worldmonitor/wildfire/v1/service_server';
|
||||
import { wildfireHandler } from '../../../server/worldmonitor/wildfire/v1/handler';
|
||||
import { createClimateServiceRoutes } from '../../../src/generated/server/worldmonitor/climate/v1/service_server';
|
||||
import { climateHandler } from '../../../server/worldmonitor/climate/v1/handler';
|
||||
import { createPredictionServiceRoutes } from '../../../src/generated/server/worldmonitor/prediction/v1/service_server';
|
||||
import { predictionHandler } from '../../../server/worldmonitor/prediction/v1/handler';
|
||||
import { createDisplacementServiceRoutes } from '../../../src/generated/server/worldmonitor/displacement/v1/service_server';
|
||||
import { displacementHandler } from '../../../server/worldmonitor/displacement/v1/handler';
|
||||
import { createAviationServiceRoutes } from '../../../src/generated/server/worldmonitor/aviation/v1/service_server';
|
||||
import { aviationHandler } from '../../../server/worldmonitor/aviation/v1/handler';
|
||||
import { createResearchServiceRoutes } from '../../../src/generated/server/worldmonitor/research/v1/service_server';
|
||||
import { researchHandler } from '../../../server/worldmonitor/research/v1/handler';
|
||||
import { createUnrestServiceRoutes } from '../../../src/generated/server/worldmonitor/unrest/v1/service_server';
|
||||
import { unrestHandler } from '../../../server/worldmonitor/unrest/v1/handler';
|
||||
import { createConflictServiceRoutes } from '../../../src/generated/server/worldmonitor/conflict/v1/service_server';
|
||||
import { conflictHandler } from '../../../server/worldmonitor/conflict/v1/handler';
|
||||
import { createMaritimeServiceRoutes } from '../../../src/generated/server/worldmonitor/maritime/v1/service_server';
|
||||
import { maritimeHandler } from '../../../server/worldmonitor/maritime/v1/handler';
|
||||
import { createCyberServiceRoutes } from '../../../src/generated/server/worldmonitor/cyber/v1/service_server';
|
||||
import { cyberHandler } from '../../../server/worldmonitor/cyber/v1/handler';
|
||||
import { createEconomicServiceRoutes } from '../../../src/generated/server/worldmonitor/economic/v1/service_server';
|
||||
import { economicHandler } from '../../../server/worldmonitor/economic/v1/handler';
|
||||
import { createInfrastructureServiceRoutes } from '../../../src/generated/server/worldmonitor/infrastructure/v1/service_server';
|
||||
import { infrastructureHandler } from '../../../server/worldmonitor/infrastructure/v1/handler';
|
||||
import { createMarketServiceRoutes } from '../../../src/generated/server/worldmonitor/market/v1/service_server';
|
||||
import { marketHandler } from '../../../server/worldmonitor/market/v1/handler';
|
||||
import { createNewsServiceRoutes } from '../../../src/generated/server/worldmonitor/news/v1/service_server';
|
||||
import { newsHandler } from '../../../server/worldmonitor/news/v1/handler';
|
||||
import { createIntelligenceServiceRoutes } from '../../../src/generated/server/worldmonitor/intelligence/v1/service_server';
|
||||
import { intelligenceHandler } from '../../../server/worldmonitor/intelligence/v1/handler';
|
||||
import { createMilitaryServiceRoutes } from '../../../src/generated/server/worldmonitor/military/v1/service_server';
|
||||
import { militaryHandler } from '../../../server/worldmonitor/military/v1/handler';
|
||||
import { createPositiveEventsServiceRoutes } from '../../../src/generated/server/worldmonitor/positive_events/v1/service_server';
|
||||
import { positiveEventsHandler } from '../../../server/worldmonitor/positive-events/v1/handler';
|
||||
import { createGivingServiceRoutes } from '../../../src/generated/server/worldmonitor/giving/v1/service_server';
|
||||
import { givingHandler } from '../../../server/worldmonitor/giving/v1/handler';
|
||||
import { createTradeServiceRoutes } from '../../../src/generated/server/worldmonitor/trade/v1/service_server';
|
||||
import { tradeHandler } from '../../../server/worldmonitor/trade/v1/handler';
|
||||
import { createSupplyChainServiceRoutes } from '../../../src/generated/server/worldmonitor/supply_chain/v1/service_server';
|
||||
import { supplyChainHandler } from '../../../server/worldmonitor/supply-chain/v1/handler';
|
||||
|
||||
import type { ServerOptions } from '../../../src/generated/server/worldmonitor/seismology/v1/service_server';
|
||||
|
||||
const serverOptions: ServerOptions = { onError: mapErrorToResponse };
|
||||
|
||||
const allRoutes = [
|
||||
...createSeismologyServiceRoutes(seismologyHandler, serverOptions),
|
||||
...createWildfireServiceRoutes(wildfireHandler, serverOptions),
|
||||
...createClimateServiceRoutes(climateHandler, serverOptions),
|
||||
...createPredictionServiceRoutes(predictionHandler, serverOptions),
|
||||
...createDisplacementServiceRoutes(displacementHandler, serverOptions),
|
||||
...createAviationServiceRoutes(aviationHandler, serverOptions),
|
||||
...createResearchServiceRoutes(researchHandler, serverOptions),
|
||||
...createUnrestServiceRoutes(unrestHandler, serverOptions),
|
||||
...createConflictServiceRoutes(conflictHandler, serverOptions),
|
||||
...createMaritimeServiceRoutes(maritimeHandler, serverOptions),
|
||||
...createCyberServiceRoutes(cyberHandler, serverOptions),
|
||||
...createEconomicServiceRoutes(economicHandler, serverOptions),
|
||||
...createInfrastructureServiceRoutes(infrastructureHandler, serverOptions),
|
||||
...createMarketServiceRoutes(marketHandler, serverOptions),
|
||||
...createNewsServiceRoutes(newsHandler, serverOptions),
|
||||
...createIntelligenceServiceRoutes(intelligenceHandler, serverOptions),
|
||||
...createMilitaryServiceRoutes(militaryHandler, serverOptions),
|
||||
...createPositiveEventsServiceRoutes(positiveEventsHandler, serverOptions),
|
||||
...createGivingServiceRoutes(givingHandler, serverOptions),
|
||||
...createTradeServiceRoutes(tradeHandler, serverOptions),
|
||||
...createSupplyChainServiceRoutes(supplyChainHandler, serverOptions),
|
||||
];
|
||||
|
||||
const router = createRouter(allRoutes);
|
||||
|
||||
export default async function handler(request: Request): Promise<Response> {
|
||||
// Origin check first — skip CORS headers for disallowed origins (M-2 fix)
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
let corsHeaders: Record<string, string>;
|
||||
try {
|
||||
corsHeaders = getCorsHeaders(request);
|
||||
} catch {
|
||||
corsHeaders = { 'Access-Control-Allow-Origin': '*' };
|
||||
}
|
||||
|
||||
// OPTIONS preflight
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
// API key validation (origin-aware)
|
||||
const keyCheck = validateApiKey(request);
|
||||
if (keyCheck.required && !keyCheck.valid) {
|
||||
return new Response(JSON.stringify({ error: keyCheck.error }), {
|
||||
status: 401,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
// Route matching
|
||||
const matchedHandler = router.match(request);
|
||||
if (!matchedHandler) {
|
||||
return new Response(JSON.stringify({ error: 'Not found' }), {
|
||||
status: 404,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
// Execute handler with top-level error boundary (H-1 fix)
|
||||
let response: Response;
|
||||
try {
|
||||
response = await matchedHandler(request);
|
||||
} catch (err) {
|
||||
console.error('[gateway] Unhandled handler error:', err);
|
||||
response = new Response(JSON.stringify({ message: 'Internal server error' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Merge CORS headers into response
|
||||
const mergedHeaders = new Headers(response.headers);
|
||||
for (const [key, value] of Object.entries(corsHeaders)) {
|
||||
mergedHeaders.set(key, value);
|
||||
}
|
||||
|
||||
return new Response(response.body, {
|
||||
status: response.status,
|
||||
statusText: response.statusText,
|
||||
headers: mergedHeaders,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
const DESKTOP_ORIGIN_PATTERNS = [
|
||||
/^https?:\/\/tauri\.localhost(:\d+)?$/,
|
||||
/^https?:\/\/[a-z0-9-]+\.tauri\.localhost(:\d+)?$/i,
|
||||
/^tauri:\/\/localhost$/,
|
||||
/^asset:\/\/localhost$/,
|
||||
];
|
||||
|
||||
function isDesktopOrigin(origin) {
|
||||
return Boolean(origin) && DESKTOP_ORIGIN_PATTERNS.some(p => p.test(origin));
|
||||
}
|
||||
|
||||
export function validateApiKey(req) {
|
||||
const key = req.headers.get('X-WorldMonitor-Key');
|
||||
const origin = req.headers.get('Origin') || '';
|
||||
|
||||
if (isDesktopOrigin(origin)) {
|
||||
if (!key) return { valid: false, required: true, error: 'API key required for desktop access' };
|
||||
const validKeys = (process.env.WORLDMONITOR_VALID_KEYS || '').split(',').filter(Boolean);
|
||||
if (!validKeys.includes(key)) return { valid: false, required: true, error: 'Invalid API key' };
|
||||
return { valid: true, required: true };
|
||||
}
|
||||
|
||||
if (key) {
|
||||
const validKeys = (process.env.WORLDMONITOR_VALID_KEYS || '').split(',').filter(Boolean);
|
||||
if (!validKeys.includes(key)) return { valid: false, required: true, error: 'Invalid API key' };
|
||||
return { valid: true, required: true };
|
||||
}
|
||||
|
||||
return { valid: false, required: false };
|
||||
}
|
||||
@@ -1,53 +0,0 @@
|
||||
const statsByEndpoint = new Map();
|
||||
const MAX_ENDPOINTS = 128;
|
||||
const LOG_EVERY = Math.max(0, Number(process.env.CACHE_TELEMETRY_LOG_EVERY || 200));
|
||||
|
||||
function cleanupOldEndpoints() {
|
||||
if (statsByEndpoint.size <= MAX_ENDPOINTS) return;
|
||||
const entries = Array.from(statsByEndpoint.entries())
|
||||
.sort((a, b) => a[1].lastSeen - b[1].lastSeen);
|
||||
const overflow = statsByEndpoint.size - MAX_ENDPOINTS;
|
||||
for (let i = 0; i < overflow; i++) {
|
||||
statsByEndpoint.delete(entries[i][0]);
|
||||
}
|
||||
}
|
||||
|
||||
export function recordCacheTelemetry(endpoint, outcome) {
|
||||
if (!endpoint || !outcome) return;
|
||||
const now = Date.now();
|
||||
const current = statsByEndpoint.get(endpoint) || {
|
||||
total: 0,
|
||||
outcomes: {},
|
||||
firstSeen: now,
|
||||
lastSeen: now,
|
||||
};
|
||||
|
||||
current.total += 1;
|
||||
current.outcomes[outcome] = (current.outcomes[outcome] || 0) + 1;
|
||||
current.lastSeen = now;
|
||||
statsByEndpoint.set(endpoint, current);
|
||||
cleanupOldEndpoints();
|
||||
|
||||
if (LOG_EVERY > 0 && current.total % LOG_EVERY === 0) {
|
||||
console.log(`[CacheTelemetry] ${endpoint} total=${current.total} outcomes=${JSON.stringify(current.outcomes)}`);
|
||||
}
|
||||
}
|
||||
|
||||
export function getCacheTelemetrySnapshot() {
|
||||
const endpoints = Array.from(statsByEndpoint.entries())
|
||||
.sort((a, b) => b[1].lastSeen - a[1].lastSeen)
|
||||
.map(([endpoint, stats]) => ({
|
||||
endpoint,
|
||||
total: stats.total,
|
||||
outcomes: stats.outcomes,
|
||||
firstSeen: new Date(stats.firstSeen).toISOString(),
|
||||
lastSeen: new Date(stats.lastSeen).toISOString(),
|
||||
}));
|
||||
|
||||
return {
|
||||
generatedAt: new Date().toISOString(),
|
||||
endpointCount: endpoints.length,
|
||||
endpoints,
|
||||
note: 'In-memory per instance telemetry (resets on cold start).',
|
||||
};
|
||||
}
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
const ALLOWED_ORIGIN_PATTERNS = [
|
||||
/^https:\/\/(.*\.)?worldmonitor\.app$/,
|
||||
/^https:\/\/worldmonitor-[a-z0-9-]+-elie-habib-projects\.vercel\.app$/,
|
||||
/^https:\/\/worldmonitor-[a-z0-9-]+-elie-[a-z0-9]+\.vercel\.app$/,
|
||||
/^https?:\/\/localhost(:\d+)?$/,
|
||||
/^https?:\/\/127\.0\.0\.1(:\d+)?$/,
|
||||
/^https?:\/\/tauri\.localhost(:\d+)?$/,
|
||||
@@ -19,7 +19,7 @@ export function getCorsHeaders(req, methods = 'GET, OPTIONS') {
|
||||
return {
|
||||
'Access-Control-Allow-Origin': allowOrigin,
|
||||
'Access-Control-Allow-Methods': methods,
|
||||
'Access-Control-Allow-Headers': 'Content-Type',
|
||||
'Access-Control-Allow-Headers': 'Content-Type, Authorization, X-WorldMonitor-Key',
|
||||
'Access-Control-Max-Age': '86400',
|
||||
'Vary': 'Origin',
|
||||
};
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
export function createIpRateLimiter({
|
||||
limit,
|
||||
windowMs,
|
||||
maxEntries = 5000,
|
||||
cleanupIntervalMs = 30 * 1000,
|
||||
}) {
|
||||
const records = new Map();
|
||||
let lastCleanupAt = 0;
|
||||
|
||||
function cleanup(now) {
|
||||
if (now - lastCleanupAt < cleanupIntervalMs && records.size <= maxEntries) {
|
||||
return;
|
||||
}
|
||||
lastCleanupAt = now;
|
||||
|
||||
const cutoff = now - windowMs;
|
||||
for (const [ip, record] of records) {
|
||||
if (record.windowStart < cutoff) {
|
||||
records.delete(ip);
|
||||
}
|
||||
}
|
||||
|
||||
if (records.size <= maxEntries) {
|
||||
return;
|
||||
}
|
||||
|
||||
const overflow = records.size - maxEntries;
|
||||
const oldest = Array.from(records.entries())
|
||||
.sort((a, b) => a[1].windowStart - b[1].windowStart);
|
||||
for (let i = 0; i < overflow; i++) {
|
||||
const entry = oldest[i];
|
||||
if (!entry) break;
|
||||
records.delete(entry[0]);
|
||||
}
|
||||
}
|
||||
|
||||
function check(ip) {
|
||||
const now = Date.now();
|
||||
cleanup(now);
|
||||
|
||||
const key = (ip || 'unknown').trim() || 'unknown';
|
||||
const record = records.get(key);
|
||||
|
||||
if (!record || now - record.windowStart > windowMs) {
|
||||
records.set(key, { count: 1, windowStart: now });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (record.count >= limit) {
|
||||
return false;
|
||||
}
|
||||
|
||||
record.count += 1;
|
||||
return true;
|
||||
}
|
||||
|
||||
return {
|
||||
check,
|
||||
size: () => records.size,
|
||||
};
|
||||
}
|
||||
@@ -1,185 +0,0 @@
|
||||
const isSidecar = (process.env.LOCAL_API_MODE || '').includes('sidecar');
|
||||
|
||||
// ── In-memory cache (desktop/sidecar) ──
|
||||
const mem = new Map();
|
||||
let persistPath = null;
|
||||
let persistTimer = null;
|
||||
let persistInFlight = false;
|
||||
let persistQueued = false;
|
||||
let loaded = false;
|
||||
const MAX_PERSIST_ENTRIES = Math.max(100, Number(process.env.LOCAL_API_CACHE_PERSIST_MAX || 5000));
|
||||
|
||||
async function ensureDesktopCache() {
|
||||
if (loaded) return;
|
||||
loaded = true;
|
||||
try {
|
||||
const { join } = await import('node:path');
|
||||
const { readFileSync } = await import('node:fs');
|
||||
const dir = process.env.LOCAL_API_RESOURCE_DIR || '.';
|
||||
persistPath = join(dir, 'api-cache.json');
|
||||
const data = JSON.parse(readFileSync(persistPath, 'utf8'));
|
||||
const now = Date.now();
|
||||
for (const [k, entry] of Object.entries(data)) {
|
||||
if (entry.expiresAt > now) mem.set(k, entry);
|
||||
}
|
||||
console.log(`[Cache] Loaded ${mem.size} entries from disk`);
|
||||
} catch {
|
||||
// File doesn't exist yet
|
||||
}
|
||||
setInterval(() => {
|
||||
const now = Date.now();
|
||||
for (const [k, v] of mem) {
|
||||
if (v.expiresAt <= now) mem.delete(k);
|
||||
}
|
||||
}, 60_000).unref?.();
|
||||
}
|
||||
|
||||
function buildPersistSnapshot() {
|
||||
const now = Date.now();
|
||||
const payload = Object.create(null);
|
||||
let kept = 0;
|
||||
|
||||
for (const [key, entry] of mem) {
|
||||
if (!entry || entry.expiresAt <= now) continue;
|
||||
payload[key] = entry;
|
||||
kept += 1;
|
||||
if (kept >= MAX_PERSIST_ENTRIES) break;
|
||||
}
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
async function persistToDisk() {
|
||||
if (!persistPath) return;
|
||||
if (persistInFlight) {
|
||||
persistQueued = true;
|
||||
return;
|
||||
}
|
||||
|
||||
persistInFlight = true;
|
||||
try {
|
||||
const snapshot = buildPersistSnapshot();
|
||||
const json = JSON.stringify(snapshot);
|
||||
const { writeFile, rename } = await import('node:fs/promises');
|
||||
const tmp = persistPath + '.tmp';
|
||||
await writeFile(tmp, json, 'utf8');
|
||||
await rename(tmp, persistPath);
|
||||
} catch (err) {
|
||||
console.warn('[Cache] Persist error:', err.message);
|
||||
} finally {
|
||||
persistInFlight = false;
|
||||
if (persistQueued) {
|
||||
persistQueued = false;
|
||||
void persistToDisk();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function debouncedPersist() {
|
||||
if (!persistPath) return;
|
||||
clearTimeout(persistTimer);
|
||||
persistTimer = setTimeout(() => {
|
||||
void persistToDisk();
|
||||
}, 2000);
|
||||
if (persistTimer?.unref) persistTimer.unref();
|
||||
}
|
||||
|
||||
// ── Redis (cloud/Vercel) ──
|
||||
let RedisClass = null;
|
||||
let redis = null;
|
||||
let redisInitFailed = false;
|
||||
|
||||
export async function getRedis() {
|
||||
if (isSidecar) return null;
|
||||
if (redis) return redis;
|
||||
if (redisInitFailed) return null;
|
||||
|
||||
const url = process.env.UPSTASH_REDIS_REST_URL;
|
||||
const token = process.env.UPSTASH_REDIS_REST_TOKEN;
|
||||
if (!url || !token) return null;
|
||||
|
||||
try {
|
||||
if (!RedisClass) {
|
||||
const mod = await import('@upstash/redis');
|
||||
RedisClass = mod.Redis;
|
||||
}
|
||||
redis = new RedisClass({ url, token });
|
||||
return redis;
|
||||
} catch (err) {
|
||||
redisInitFailed = true;
|
||||
console.warn('[Cache] Redis init failed:', err.message);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ── Shared API ──
|
||||
|
||||
export async function getCachedJson(key) {
|
||||
if (isSidecar) {
|
||||
await ensureDesktopCache();
|
||||
const entry = mem.get(key);
|
||||
if (!entry) return null;
|
||||
if (entry.expiresAt <= Date.now()) {
|
||||
mem.delete(key);
|
||||
return null;
|
||||
}
|
||||
return entry.value;
|
||||
}
|
||||
|
||||
const r = await getRedis();
|
||||
if (!r) return null;
|
||||
try {
|
||||
return await r.get(key);
|
||||
} catch (err) {
|
||||
console.warn('[Cache] Read failed:', err.message);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function setCachedJson(key, value, ttlSeconds) {
|
||||
if (isSidecar) {
|
||||
await ensureDesktopCache();
|
||||
mem.set(key, { value, expiresAt: Date.now() + ttlSeconds * 1000 });
|
||||
debouncedPersist();
|
||||
return true;
|
||||
}
|
||||
|
||||
const r = await getRedis();
|
||||
if (!r) return false;
|
||||
try {
|
||||
await r.set(key, value, { ex: ttlSeconds });
|
||||
return true;
|
||||
} catch (err) {
|
||||
console.warn('[Cache] Write failed:', err.message);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export async function mget(...keys) {
|
||||
if (isSidecar) {
|
||||
await ensureDesktopCache();
|
||||
const now = Date.now();
|
||||
return keys.map(k => {
|
||||
const entry = mem.get(k);
|
||||
if (!entry || entry.expiresAt <= now) return null;
|
||||
return entry.value;
|
||||
});
|
||||
}
|
||||
|
||||
const r = await getRedis();
|
||||
if (!r) return keys.map(() => null);
|
||||
try {
|
||||
return await r.mget(...keys);
|
||||
} catch (err) {
|
||||
console.warn('[Cache] mget failed:', err.message);
|
||||
return keys.map(() => null);
|
||||
}
|
||||
}
|
||||
|
||||
export function hashString(input) {
|
||||
let hash = 5381;
|
||||
for (let i = 0; i < input.length; i++) {
|
||||
hash = ((hash << 5) + hash) + input.charCodeAt(i);
|
||||
}
|
||||
return (hash >>> 0).toString(36);
|
||||
}
|
||||
@@ -1,188 +0,0 @@
|
||||
// ACLED Conflict Events API proxy - battles, explosions, violence against civilians
|
||||
// Separate from protest proxy to avoid mixing data flows
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'acled:conflict:v2';
|
||||
const CACHE_TTL_SECONDS = 10 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const RATE_LIMIT = 10;
|
||||
const RATE_WINDOW_MS = 60 * 1000;
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: RATE_LIMIT,
|
||||
windowMs: RATE_WINDOW_MS,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed', data: [] }, {
|
||||
status: 405,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed', data: [] }, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited', data: [] }, {
|
||||
status: 429,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Retry-After': '60',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const token = process.env.ACLED_ACCESS_TOKEN;
|
||||
if (!token) {
|
||||
return Response.json({ error: 'ACLED not configured', data: [], configured: false }, {
|
||||
status: 200,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (cached && typeof cached === 'object' && Array.isArray(cached.data)) {
|
||||
recordCacheTelemetry('/api/acled-conflict', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (fallbackCache.data && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/acled-conflict', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const endDate = new Date().toISOString().split('T')[0];
|
||||
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
|
||||
const params = new URLSearchParams({
|
||||
event_type: 'Battles|Explosions/Remote violence|Violence against civilians',
|
||||
event_date: `${startDate}|${endDate}`,
|
||||
event_date_where: 'BETWEEN',
|
||||
limit: '500',
|
||||
_format: 'json',
|
||||
});
|
||||
|
||||
const response = await fetch(`https://acleddata.com/api/acled/read?${params}`, {
|
||||
headers: {
|
||||
Accept: 'application/json',
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const text = await response.text();
|
||||
return Response.json({ error: `ACLED API error: ${response.status}`, details: text.substring(0, 200), data: [] }, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const rawData = await response.json();
|
||||
const events = Array.isArray(rawData?.data) ? rawData.data : [];
|
||||
const sanitizedEvents = events.map((e) => ({
|
||||
event_id_cnty: e.event_id_cnty,
|
||||
event_date: e.event_date,
|
||||
event_type: e.event_type,
|
||||
sub_event_type: e.sub_event_type,
|
||||
actor1: e.actor1,
|
||||
actor2: e.actor2,
|
||||
country: e.country,
|
||||
admin1: e.admin1,
|
||||
location: e.location,
|
||||
latitude: e.latitude,
|
||||
longitude: e.longitude,
|
||||
fatalities: e.fatalities,
|
||||
notes: typeof e.notes === 'string' ? e.notes.substring(0, 500) : undefined,
|
||||
source: e.source,
|
||||
tags: e.tags,
|
||||
}));
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: sanitizedEvents.length,
|
||||
data: sanitizedEvents,
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/acled-conflict', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'MISS',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (fallbackCache.data) {
|
||||
recordCacheTelemetry('/api/acled-conflict', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
'X-Cache': 'STALE',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/acled-conflict', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, data: [] }, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
-200
@@ -1,200 +0,0 @@
|
||||
// ACLED API proxy - keeps token server-side only
|
||||
// Token is stored in ACLED_ACCESS_TOKEN (no VITE_ prefix)
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'acled:protests:v2';
|
||||
const CACHE_TTL_SECONDS = 10 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
|
||||
// In-memory fallback cache when Redis is unavailable.
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const RATE_LIMIT = 10; // requests per minute
|
||||
const RATE_WINDOW_MS = 60 * 1000;
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: RATE_LIMIT,
|
||||
windowMs: RATE_WINDOW_MS,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed', data: [] }, {
|
||||
status: 405,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed', data: [] }, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited', data: [] }, {
|
||||
status: 429,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Retry-After': '60',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const token = process.env.ACLED_ACCESS_TOKEN;
|
||||
if (!token) {
|
||||
return Response.json({
|
||||
error: 'ACLED not configured',
|
||||
data: [],
|
||||
configured: false,
|
||||
}, {
|
||||
status: 200,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (cached && typeof cached === 'object' && Array.isArray(cached.data)) {
|
||||
recordCacheTelemetry('/api/acled', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (fallbackCache.data && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/acled', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const endDate = new Date().toISOString().split('T')[0];
|
||||
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
|
||||
const params = new URLSearchParams({
|
||||
event_type: 'Protests',
|
||||
event_date: `${startDate}|${endDate}`,
|
||||
event_date_where: 'BETWEEN',
|
||||
limit: '500',
|
||||
_format: 'json',
|
||||
});
|
||||
|
||||
const response = await fetch(`https://acleddata.com/api/acled/read?${params}`, {
|
||||
headers: {
|
||||
Accept: 'application/json',
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const text = await response.text();
|
||||
return Response.json({
|
||||
error: `ACLED API error: ${response.status}`,
|
||||
details: text.substring(0, 200),
|
||||
data: [],
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const rawData = await response.json();
|
||||
const events = Array.isArray(rawData?.data) ? rawData.data : [];
|
||||
const sanitizedEvents = events.map((e) => ({
|
||||
event_id_cnty: e.event_id_cnty,
|
||||
event_date: e.event_date,
|
||||
event_type: e.event_type,
|
||||
sub_event_type: e.sub_event_type,
|
||||
actor1: e.actor1,
|
||||
actor2: e.actor2,
|
||||
country: e.country,
|
||||
admin1: e.admin1,
|
||||
location: e.location,
|
||||
latitude: e.latitude,
|
||||
longitude: e.longitude,
|
||||
fatalities: e.fatalities,
|
||||
notes: typeof e.notes === 'string' ? e.notes.substring(0, 500) : undefined,
|
||||
source: e.source,
|
||||
tags: e.tags,
|
||||
}));
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: sanitizedEvents.length,
|
||||
data: sanitizedEvents,
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/acled', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'MISS',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (fallbackCache.data) {
|
||||
recordCacheTelemetry('/api/acled', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
'X-Cache': 'STALE',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/acled', 'ERROR');
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${toErrorMessage(error)}`,
|
||||
data: [],
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
+49
-167
@@ -1,98 +1,37 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
|
||||
const CACHE_TTL_SECONDS = 8;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const CACHE_VERSION = 'v1';
|
||||
const MEMORY_CACHE_MAX_ENTRIES = 8;
|
||||
const MEMORY_FALLBACK_MAX_AGE_MS = 60 * 1000;
|
||||
const memoryCache = new Map();
|
||||
const inFlightByKey = new Map();
|
||||
|
||||
function getErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'Failed to fetch AIS snapshot');
|
||||
}
|
||||
|
||||
function getMemoryCachedSnapshot(cacheKey, allowStale = false) {
|
||||
const entry = memoryCache.get(cacheKey);
|
||||
if (!entry) return null;
|
||||
|
||||
const now = Date.now();
|
||||
const age = now - entry.timestamp;
|
||||
if (age > MEMORY_FALLBACK_MAX_AGE_MS) {
|
||||
memoryCache.delete(cacheKey);
|
||||
return null;
|
||||
}
|
||||
|
||||
if (!allowStale && age > CACHE_TTL_MS) {
|
||||
return null;
|
||||
}
|
||||
|
||||
entry.lastSeen = now;
|
||||
return entry.data;
|
||||
}
|
||||
|
||||
function setMemoryCachedSnapshot(cacheKey, data) {
|
||||
const now = Date.now();
|
||||
memoryCache.set(cacheKey, {
|
||||
data,
|
||||
timestamp: now,
|
||||
lastSeen: now,
|
||||
});
|
||||
|
||||
if (memoryCache.size <= MEMORY_CACHE_MAX_ENTRIES) return;
|
||||
|
||||
const overflow = memoryCache.size - MEMORY_CACHE_MAX_ENTRIES;
|
||||
const oldestEntries = Array.from(memoryCache.entries())
|
||||
.sort((a, b) => a[1].lastSeen - b[1].lastSeen);
|
||||
for (let i = 0; i < overflow; i++) {
|
||||
const entry = oldestEntries[i];
|
||||
if (!entry) break;
|
||||
memoryCache.delete(entry[0]);
|
||||
}
|
||||
}
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
function getRelayBaseUrl() {
|
||||
const relayUrl = process.env.WS_RELAY_URL;
|
||||
if (!relayUrl) return null;
|
||||
return relayUrl
|
||||
.replace('wss://', 'https://')
|
||||
.replace('ws://', 'http://')
|
||||
.replace(/\/$/, '');
|
||||
return relayUrl.replace('wss://', 'https://').replace('ws://', 'http://').replace(/\/$/, '');
|
||||
}
|
||||
|
||||
function isValidSnapshot(data) {
|
||||
return Boolean(
|
||||
data &&
|
||||
typeof data === 'object' &&
|
||||
data.status &&
|
||||
typeof data.status === 'object' &&
|
||||
Array.isArray(data.disruptions) &&
|
||||
Array.isArray(data.density)
|
||||
);
|
||||
function getRelayHeaders(baseHeaders = {}) {
|
||||
const headers = { ...baseHeaders };
|
||||
const relaySecret = process.env.RELAY_SHARED_SECRET || '';
|
||||
if (relaySecret) {
|
||||
const relayHeader = (process.env.RELAY_AUTH_HEADER || 'x-relay-key').toLowerCase();
|
||||
headers[relayHeader] = relaySecret;
|
||||
headers.Authorization = `Bearer ${relaySecret}`;
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
async function fetchWithTimeout(url, options, timeoutMs = 15000) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
@@ -100,107 +39,50 @@ export default async function handler(req) {
|
||||
});
|
||||
}
|
||||
|
||||
const requestUrl = new URL(req.url);
|
||||
const includeCandidates = requestUrl.searchParams.get('candidates') === 'true';
|
||||
const cacheKey = `ais-snapshot:${CACHE_VERSION}:${includeCandidates ? 'full' : 'lite'}`;
|
||||
const redisCached = await getCachedJson(cacheKey);
|
||||
if (isValidSnapshot(redisCached)) {
|
||||
setMemoryCachedSnapshot(cacheKey, redisCached);
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'REDIS-HIT');
|
||||
return new Response(JSON.stringify(redisCached), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const memoryCached = getMemoryCachedSnapshot(cacheKey);
|
||||
if (isValidSnapshot(memoryCached)) {
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'MEMORY-HIT');
|
||||
return new Response(JSON.stringify(memoryCached), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
...corsHeaders,
|
||||
},
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const relayBaseUrl = getRelayBaseUrl();
|
||||
if (!relayBaseUrl) {
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'NO-RELAY-CONFIG');
|
||||
return new Response(JSON.stringify({ vessels: [], skipped: true, reason: 'AIS relay not configured' }), {
|
||||
status: 200,
|
||||
return new Response(JSON.stringify({ error: 'WS_RELAY_URL is not configured' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
let requestPromise = inFlightByKey.get(cacheKey);
|
||||
if (!requestPromise) {
|
||||
requestPromise = (async () => {
|
||||
const upstreamUrl = `${relayBaseUrl}/ais/snapshot?candidates=${includeCandidates ? 'true' : 'false'}`;
|
||||
const response = await fetch(upstreamUrl, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`AIS relay HTTP ${response.status}`);
|
||||
}
|
||||
const requestUrl = new URL(req.url);
|
||||
const relayUrl = `${relayBaseUrl}/ais/snapshot${requestUrl.search || ''}`;
|
||||
const response = await fetchWithTimeout(relayUrl, {
|
||||
headers: getRelayHeaders({ Accept: 'application/json' }),
|
||||
}, 12000);
|
||||
|
||||
const data = await response.json();
|
||||
if (!isValidSnapshot(data)) {
|
||||
throw new Error('Invalid AIS snapshot payload');
|
||||
}
|
||||
return data;
|
||||
})();
|
||||
inFlightByKey.set(cacheKey, requestPromise);
|
||||
}
|
||||
const body = await response.text();
|
||||
const headers = {
|
||||
'Content-Type': response.headers.get('content-type') || 'application/json',
|
||||
'Cache-Control': response.headers.get('cache-control') || 'no-cache',
|
||||
...corsHeaders,
|
||||
};
|
||||
|
||||
const data = await requestPromise;
|
||||
if (!isValidSnapshot(data)) {
|
||||
throw new Error('Invalid AIS snapshot payload');
|
||||
}
|
||||
|
||||
setMemoryCachedSnapshot(cacheKey, data);
|
||||
void setCachedJson(cacheKey, data, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'MISS');
|
||||
|
||||
return new Response(JSON.stringify(data), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
|
||||
'X-Cache': 'MISS',
|
||||
...corsHeaders,
|
||||
},
|
||||
return new Response(body, {
|
||||
status: response.status,
|
||||
headers,
|
||||
});
|
||||
} catch (error) {
|
||||
const staleMemory = getMemoryCachedSnapshot(cacheKey, true);
|
||||
if (isValidSnapshot(staleMemory)) {
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'MEMORY-ERROR-FALLBACK');
|
||||
return new Response(JSON.stringify(staleMemory), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
|
||||
'X-Cache': 'MEMORY-ERROR-FALLBACK',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'ERROR');
|
||||
return new Response(JSON.stringify({ error: getErrorMessage(error) }), {
|
||||
status: 502,
|
||||
const isTimeout = error?.name === 'AbortError';
|
||||
return new Response(JSON.stringify({
|
||||
error: isTimeout ? 'Relay timeout' : 'Relay request failed',
|
||||
details: error?.message || String(error),
|
||||
}), {
|
||||
status: isTimeout ? 504 : 502,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
} finally {
|
||||
inFlightByKey.delete(cacheKey);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
// Fetch AI/ML papers from ArXiv
|
||||
// Categories: cs.AI, cs.LG (Machine Learning), cs.CL (Computation and Language)
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const category = searchParams.get('category') || 'cs.AI'; // cs.AI, cs.LG, cs.CL
|
||||
const maxResults = searchParams.get('max_results') || '50';
|
||||
const sortBy = searchParams.get('sortBy') || 'submittedDate'; // submittedDate, lastUpdatedDate, relevance
|
||||
|
||||
// ArXiv API search query
|
||||
// Search for papers in specified category, sorted by date
|
||||
const query = `cat:${category}`;
|
||||
const apiUrl = `https://export.arxiv.org/api/query?search_query=${encodeURIComponent(query)}&start=0&max_results=${maxResults}&sortBy=${sortBy}&sortOrder=descending`;
|
||||
|
||||
const response = await fetch(apiUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'WorldMonitor/1.0 (AI Research Tracker)',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`ArXiv API returned ${response.status}`);
|
||||
}
|
||||
|
||||
const xmlData = await response.text();
|
||||
|
||||
// Parse XML to extract key information
|
||||
// Return raw XML for client-side parsing or transform here
|
||||
return new Response(xmlData, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/xml',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', // 1 hour cache
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: 'Failed to fetch ArXiv data',
|
||||
message: error.message
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCacheTelemetrySnapshot } from './_cache-telemetry.js';
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(getCacheTelemetrySnapshot()), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,198 +0,0 @@
|
||||
import { getCachedJson, setCachedJson, mget, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
|
||||
const MODEL = 'llama-3.1-8b-instant';
|
||||
const CACHE_TTL_SECONDS = 86400;
|
||||
const CACHE_VERSION = 'v1';
|
||||
const MAX_BATCH_SIZE = 20;
|
||||
|
||||
const VALID_LEVELS = ['critical', 'high', 'medium', 'low', 'info'];
|
||||
const VALID_CATEGORIES = [
|
||||
'conflict', 'protest', 'disaster', 'diplomatic', 'economic',
|
||||
'terrorism', 'cyber', 'health', 'environmental', 'military',
|
||||
'crime', 'infrastructure', 'tech', 'general',
|
||||
];
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.GROQ_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ results: [], fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return new Response(JSON.stringify({ error: 'Payload too large' }), {
|
||||
status: 413,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
let body;
|
||||
try {
|
||||
body = await request.json();
|
||||
} catch {
|
||||
return new Response(JSON.stringify({ error: 'Invalid JSON body' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const { titles, variant = 'full' } = body;
|
||||
if (!Array.isArray(titles) || titles.length === 0) {
|
||||
return new Response(JSON.stringify({ error: 'titles array required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const batch = titles.slice(0, MAX_BATCH_SIZE);
|
||||
const results = new Array(batch.length).fill(null);
|
||||
const uncachedIndices = [];
|
||||
|
||||
const cacheKeys = batch.map(
|
||||
(t) => `classify:${CACHE_VERSION}:${hashString(t.toLowerCase() + ':' + variant)}`
|
||||
);
|
||||
const cached = await mget(...cacheKeys);
|
||||
for (let i = 0; i < cached.length; i++) {
|
||||
const val = cached[i];
|
||||
if (val && typeof val === 'object' && val.level) {
|
||||
results[i] = { level: val.level, category: val.category, cached: true };
|
||||
} else {
|
||||
uncachedIndices.push(i);
|
||||
}
|
||||
}
|
||||
|
||||
if (uncachedIndices.length === 0) {
|
||||
return new Response(JSON.stringify({ results }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const uncachedTitles = uncachedIndices.map((i) => batch[i]);
|
||||
const isTech = variant === 'tech';
|
||||
const numberedList = uncachedTitles.map((t, i) => `${i + 1}. ${t}`).join('\n');
|
||||
|
||||
const systemPrompt = `You classify news headlines into threat level and category. Return ONLY a valid JSON array, no other text.
|
||||
|
||||
Levels: critical, high, medium, low, info
|
||||
Categories: conflict, protest, disaster, diplomatic, economic, terrorism, cyber, health, environmental, military, crime, infrastructure, tech, general
|
||||
|
||||
${isTech ? 'Focus: technology, startups, AI, cybersecurity. Most tech news is "low" or "info" unless it involves outages, breaches, or major disruptions.' : 'Focus: geopolitical events, conflicts, disasters, diplomacy. Classify by real-world severity and impact.'}
|
||||
|
||||
Return a JSON array with one object per headline in order: [{"level":"...","category":"..."},...]`;
|
||||
|
||||
try {
|
||||
const response = await fetch(GROQ_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: numberedList },
|
||||
],
|
||||
temperature: 0,
|
||||
max_tokens: uncachedTitles.length * 60,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
console.error('[ClassifyBatch] Groq error:', response.status);
|
||||
return new Response(JSON.stringify({ results, fallback: true }), {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const raw = data.choices?.[0]?.message?.content?.trim();
|
||||
if (!raw) {
|
||||
return new Response(JSON.stringify({ results, fallback: true }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
const match = raw.match(/\[[\s\S]*\]/);
|
||||
if (match) {
|
||||
try { parsed = JSON.parse(match[0]); } catch { /* fall through */ }
|
||||
}
|
||||
}
|
||||
|
||||
if (!Array.isArray(parsed)) {
|
||||
return new Response(JSON.stringify({ results, fallback: true }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const cacheWrites = [];
|
||||
for (let i = 0; i < uncachedIndices.length; i++) {
|
||||
const classification = parsed[i];
|
||||
if (!classification) continue;
|
||||
|
||||
const level = VALID_LEVELS.includes(classification.level) ? classification.level : null;
|
||||
const category = VALID_CATEGORIES.includes(classification.category) ? classification.category : null;
|
||||
if (!level || !category) continue;
|
||||
|
||||
const idx = uncachedIndices[i];
|
||||
results[idx] = { level, category, cached: false };
|
||||
|
||||
const cacheKey = `classify:${CACHE_VERSION}:${hashString(batch[idx].toLowerCase() + ':' + variant)}`;
|
||||
cacheWrites.push(
|
||||
setCachedJson(cacheKey, { level, category, timestamp: Date.now() }, CACHE_TTL_SECONDS)
|
||||
);
|
||||
}
|
||||
|
||||
if (cacheWrites.length > 0) {
|
||||
await Promise.allSettled(cacheWrites);
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({ results }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[ClassifyBatch] Error:', error.message);
|
||||
return new Response(JSON.stringify({ results, fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,164 +0,0 @@
|
||||
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
|
||||
const MODEL = 'llama-3.1-8b-instant';
|
||||
const CACHE_TTL_SECONDS = 86400;
|
||||
const CACHE_VERSION = 'v1';
|
||||
|
||||
const VALID_LEVELS = ['critical', 'high', 'medium', 'low', 'info'];
|
||||
const VALID_CATEGORIES = [
|
||||
'conflict', 'protest', 'disaster', 'diplomatic', 'economic',
|
||||
'terrorism', 'cyber', 'health', 'environmental', 'military',
|
||||
'crime', 'infrastructure', 'tech', 'general',
|
||||
];
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'GET, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.GROQ_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const title = url.searchParams.get('title');
|
||||
const variant = url.searchParams.get('variant') || 'full';
|
||||
|
||||
if (!title) {
|
||||
return new Response(JSON.stringify({ error: 'title param required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const cacheKey = `classify:${CACHE_VERSION}:${hashString(title.toLowerCase() + ':' + variant)}`;
|
||||
|
||||
try {
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.level) {
|
||||
return new Response(JSON.stringify({
|
||||
level: cached.level,
|
||||
category: cached.category,
|
||||
confidence: 0.9,
|
||||
source: 'llm',
|
||||
cached: true,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const isTech = variant === 'tech';
|
||||
const systemPrompt = `You classify news headlines into threat level and category. Return ONLY valid JSON, no other text.
|
||||
|
||||
Levels: critical, high, medium, low, info
|
||||
Categories: conflict, protest, disaster, diplomatic, economic, terrorism, cyber, health, environmental, military, crime, infrastructure, tech, general
|
||||
|
||||
${isTech ? 'Focus: technology, startups, AI, cybersecurity. Most tech news is "low" or "info" unless it involves outages, breaches, or major disruptions.' : 'Focus: geopolitical events, conflicts, disasters, diplomacy. Classify by real-world severity and impact.'}
|
||||
|
||||
Return: {"level":"...","category":"..."}`;
|
||||
|
||||
const response = await fetch(GROQ_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: title },
|
||||
],
|
||||
temperature: 0,
|
||||
max_tokens: 50,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
console.error('[Classify] Groq error:', response.status);
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const raw = data.choices?.[0]?.message?.content?.trim();
|
||||
if (!raw) {
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
console.warn('[Classify] Invalid JSON from LLM:', raw);
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const level = VALID_LEVELS.includes(parsed.level) ? parsed.level : null;
|
||||
const category = VALID_CATEGORIES.includes(parsed.category) ? parsed.category : null;
|
||||
if (!level || !category) {
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
await setCachedJson(cacheKey, { level, category, timestamp: Date.now() }, CACHE_TTL_SECONDS);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
level,
|
||||
category,
|
||||
confidence: 0.9,
|
||||
source: 'llm',
|
||||
cached: false,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600',
|
||||
},
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('[Classify] Error:', error.message);
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,206 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'climate:anomalies:v1';
|
||||
const CACHE_TTL_SECONDS = 6 * 60 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: 15,
|
||||
windowMs: 60 * 1000,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(data && typeof data === 'object' && Array.isArray(data.anomalies));
|
||||
}
|
||||
|
||||
const MONITORED_ZONES = [
|
||||
{ name: 'Ukraine', lat: 48.4, lon: 31.2 },
|
||||
{ name: 'Middle East', lat: 33.0, lon: 44.0 },
|
||||
{ name: 'Sahel', lat: 14.0, lon: 0.0 },
|
||||
{ name: 'Horn of Africa', lat: 8.0, lon: 42.0 },
|
||||
{ name: 'South Asia', lat: 25.0, lon: 78.0 },
|
||||
{ name: 'California', lat: 36.8, lon: -119.4 },
|
||||
{ name: 'Amazon', lat: -3.4, lon: -60.0 },
|
||||
{ name: 'Australia', lat: -25.0, lon: 134.0 },
|
||||
{ name: 'Mediterranean', lat: 38.0, lon: 20.0 },
|
||||
{ name: 'Taiwan Strait', lat: 24.0, lon: 120.0 },
|
||||
{ name: 'Myanmar', lat: 19.8, lon: 96.7 },
|
||||
{ name: 'Central Africa', lat: 4.0, lon: 22.0 },
|
||||
{ name: 'Southern Africa', lat: -25.0, lon: 28.0 },
|
||||
{ name: 'Central Asia', lat: 42.0, lon: 65.0 },
|
||||
{ name: 'Caribbean', lat: 19.0, lon: -72.0 },
|
||||
];
|
||||
|
||||
function classifySeverity(tempDelta, precipDelta) {
|
||||
const absTemp = Math.abs(tempDelta);
|
||||
const absPrecip = Math.abs(precipDelta);
|
||||
if (absTemp >= 5 || absPrecip >= 80) return 'extreme';
|
||||
if (absTemp >= 3 || absPrecip >= 40) return 'moderate';
|
||||
return 'normal';
|
||||
}
|
||||
|
||||
function classifyType(tempDelta, precipDelta) {
|
||||
const absTemp = Math.abs(tempDelta);
|
||||
const absPrecip = Math.abs(precipDelta);
|
||||
if (absTemp >= absPrecip / 20) {
|
||||
if (tempDelta > 0 && precipDelta < -20) return 'mixed';
|
||||
if (tempDelta > 3) return 'warm';
|
||||
if (tempDelta < -3) return 'cold';
|
||||
}
|
||||
if (precipDelta > 40) return 'wet';
|
||||
if (precipDelta < -40) return 'dry';
|
||||
if (tempDelta > 0) return 'warm';
|
||||
return 'cold';
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited' }, {
|
||||
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const endDate = new Date();
|
||||
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000);
|
||||
const start = startDate.toISOString().split('T')[0];
|
||||
const end = endDate.toISOString().split('T')[0];
|
||||
|
||||
const fetchZone = async (zone) => {
|
||||
try {
|
||||
const params = new URLSearchParams({
|
||||
latitude: String(zone.lat),
|
||||
longitude: String(zone.lon),
|
||||
start_date: start,
|
||||
end_date: end,
|
||||
daily: 'temperature_2m_mean,precipitation_sum',
|
||||
timezone: 'UTC',
|
||||
});
|
||||
|
||||
const resp = await fetch(`https://archive-api.open-meteo.com/v1/archive?${params}`, {
|
||||
headers: { Accept: 'application/json' },
|
||||
});
|
||||
|
||||
if (!resp.ok) return null;
|
||||
const data = await resp.json();
|
||||
const temps = data.daily?.temperature_2m_mean || [];
|
||||
const precips = data.daily?.precipitation_sum || [];
|
||||
|
||||
if (temps.length < 14) return null;
|
||||
|
||||
const validTemps = temps.filter(t => t !== null);
|
||||
const validPrecips = precips.filter(p => p !== null);
|
||||
|
||||
const last7Temps = validTemps.slice(-7);
|
||||
const baseline30Temps = validTemps.slice(0, -7);
|
||||
const last7Precips = validPrecips.slice(-7);
|
||||
const baseline30Precips = validPrecips.slice(0, -7);
|
||||
|
||||
const avg = arr => arr.length ? arr.reduce((s, v) => s + v, 0) / arr.length : 0;
|
||||
|
||||
const tempDelta = avg(last7Temps) - avg(baseline30Temps);
|
||||
const precipDelta = avg(last7Precips) - avg(baseline30Precips);
|
||||
const severity = classifySeverity(tempDelta, precipDelta);
|
||||
|
||||
return {
|
||||
zone: zone.name,
|
||||
lat: zone.lat,
|
||||
lon: zone.lon,
|
||||
tempDelta: Math.round(tempDelta * 10) / 10,
|
||||
precipDelta: Math.round(precipDelta * 10) / 10,
|
||||
severity,
|
||||
type: classifyType(tempDelta, precipDelta),
|
||||
period: `${start} to ${end}`,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
const results = await Promise.allSettled(MONITORED_ZONES.map(fetchZone));
|
||||
const anomalies = results
|
||||
.filter(r => r.status === 'fulfilled' && r.value)
|
||||
.map(r => r.value);
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
anomalies,
|
||||
timestamp: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, anomalies: [] }, {
|
||||
status: 500, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,64 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
function clampLimit(rawLimit) {
|
||||
const parsed = Number.parseInt(rawLimit || '', 10);
|
||||
if (!Number.isFinite(parsed)) return 50;
|
||||
return Math.max(1, Math.min(100, parsed));
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const dateRange = url.searchParams.get('dateRange') || '7d';
|
||||
const limit = clampLimit(url.searchParams.get('limit'));
|
||||
|
||||
const token = process.env.CLOUDFLARE_API_TOKEN;
|
||||
if (!token) {
|
||||
// Signal to client that outages feature is not configured
|
||||
return new Response(JSON.stringify({ configured: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(
|
||||
`https://api.cloudflare.com/client/v4/radar/annotations/outages?dateRange=${dateRange}&limit=${limit}`,
|
||||
{ headers: { 'Authorization': `Bearer ${token}` } }
|
||||
);
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=120, s-maxage=120, stale-while-revalidate=60', ...corsHeaders },
|
||||
});
|
||||
} catch (error) {
|
||||
// Return empty result on error so client circuit breaker doesn't trigger unnecessarily
|
||||
return new Response(JSON.stringify({ success: true, result: { annotations: [] } }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,154 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCachedJson, hashString, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const ALLOWED_CURRENCIES = ['usd', 'eur', 'gbp', 'jpy', 'cny', 'btc', 'eth'];
|
||||
const MAX_COIN_IDS = 20;
|
||||
const COIN_ID_PATTERN = /^[a-z0-9-]+$/;
|
||||
|
||||
const CACHE_TTL_SECONDS = 120; // 2 minutes
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const RESPONSE_CACHE_CONTROL = 'public, max-age=120, s-maxage=120, stale-while-revalidate=60';
|
||||
const CACHE_VERSION = 'v2';
|
||||
|
||||
// In-memory fallback cache for the current instance.
|
||||
let fallbackCache = { key: '', payload: null, timestamp: 0 };
|
||||
|
||||
function validateCoinIds(idsParam) {
|
||||
if (!idsParam) return 'bitcoin,ethereum,solana';
|
||||
|
||||
const ids = idsParam.split(',')
|
||||
.map(id => id.trim().toLowerCase())
|
||||
.filter(id => COIN_ID_PATTERN.test(id) && id.length <= 50)
|
||||
.slice(0, MAX_COIN_IDS);
|
||||
|
||||
return ids.length > 0 ? ids.join(',') : 'bitcoin,ethereum,solana';
|
||||
}
|
||||
|
||||
function validateCurrency(val) {
|
||||
const currency = (val || 'usd').toLowerCase();
|
||||
return ALLOWED_CURRENCIES.includes(currency) ? currency : 'usd';
|
||||
}
|
||||
|
||||
function validateBoolean(val, defaultVal) {
|
||||
if (val === 'true' || val === 'false') return val;
|
||||
return defaultVal;
|
||||
}
|
||||
|
||||
function getHeaders(cors, xCache, cacheControl = RESPONSE_CACHE_CONTROL) {
|
||||
return {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': cacheControl,
|
||||
'X-Cache': xCache,
|
||||
};
|
||||
}
|
||||
|
||||
function isValidPayload(payload) {
|
||||
return Boolean(
|
||||
payload &&
|
||||
typeof payload === 'object' &&
|
||||
typeof payload.body === 'string' &&
|
||||
Number.isFinite(payload.status)
|
||||
);
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
|
||||
const ids = validateCoinIds(url.searchParams.get('ids'));
|
||||
const vsCurrencies = validateCurrency(url.searchParams.get('vs_currencies'));
|
||||
const include24hrChange = validateBoolean(url.searchParams.get('include_24hr_change'), 'true');
|
||||
|
||||
const now = Date.now();
|
||||
const cacheKey = `${ids}:${vsCurrencies}:${include24hrChange}`;
|
||||
const redisKey = `coingecko:${CACHE_VERSION}:${hashString(cacheKey)}`;
|
||||
|
||||
const redisCached = await getCachedJson(redisKey);
|
||||
if (isValidPayload(redisCached)) {
|
||||
recordCacheTelemetry('/api/coingecko', 'REDIS-HIT');
|
||||
return new Response(redisCached.body, {
|
||||
status: redisCached.status,
|
||||
headers: getHeaders(cors, 'REDIS-HIT'),
|
||||
});
|
||||
}
|
||||
|
||||
if (
|
||||
isValidPayload(fallbackCache.payload) &&
|
||||
fallbackCache.key === cacheKey &&
|
||||
now - fallbackCache.timestamp < CACHE_TTL_MS
|
||||
) {
|
||||
recordCacheTelemetry('/api/coingecko', 'MEMORY-HIT');
|
||||
return new Response(fallbackCache.payload.body, {
|
||||
status: fallbackCache.payload.status,
|
||||
headers: getHeaders(cors, 'MEMORY-HIT'),
|
||||
});
|
||||
}
|
||||
|
||||
const endpoint = url.searchParams.get('endpoint');
|
||||
|
||||
try {
|
||||
let geckoUrl;
|
||||
if (endpoint === 'markets') {
|
||||
geckoUrl = `https://api.coingecko.com/api/v3/coins/markets?vs_currency=${vsCurrencies}&ids=${ids}&order=market_cap_desc&sparkline=true&price_change_percentage=24h`;
|
||||
} else {
|
||||
geckoUrl = `https://api.coingecko.com/api/v3/simple/price?ids=${ids}&vs_currencies=${vsCurrencies}&include_24hr_change=${include24hrChange}`;
|
||||
}
|
||||
const response = await fetch(geckoUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
// If rate limited, return cached data if available
|
||||
if (
|
||||
response.status === 429 &&
|
||||
isValidPayload(fallbackCache.payload) &&
|
||||
fallbackCache.key === cacheKey
|
||||
) {
|
||||
recordCacheTelemetry('/api/coingecko', 'STALE');
|
||||
return new Response(fallbackCache.payload.body, {
|
||||
status: fallbackCache.payload.status,
|
||||
headers: getHeaders(cors, 'STALE'),
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
|
||||
// Cache successful responses
|
||||
if (response.ok) {
|
||||
const payload = { body: data, status: response.status };
|
||||
fallbackCache = { key: cacheKey, payload, timestamp: Date.now() };
|
||||
void setCachedJson(redisKey, payload, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/coingecko', 'MISS');
|
||||
} else {
|
||||
recordCacheTelemetry('/api/coingecko', 'UPSTREAM-ERROR');
|
||||
}
|
||||
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: getHeaders(cors, 'MISS'),
|
||||
});
|
||||
} catch (error) {
|
||||
// Return cached data on error if available
|
||||
if (isValidPayload(fallbackCache.payload) && fallbackCache.key === cacheKey) {
|
||||
recordCacheTelemetry('/api/coingecko', 'ERROR-FALLBACK');
|
||||
return new Response(fallbackCache.payload.body, {
|
||||
status: fallbackCache.payload.status,
|
||||
headers: getHeaders(cors, 'ERROR-FALLBACK', 'public, max-age=120'),
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/coingecko', 'ERROR');
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,191 +0,0 @@
|
||||
/**
|
||||
* Country Intelligence Brief Endpoint
|
||||
* Generates AI-powered country situation briefs using Groq
|
||||
* Redis cached (2h TTL) for cross-user deduplication
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
|
||||
const MODEL = 'llama-3.1-8b-instant';
|
||||
const CACHE_TTL_SECONDS = 7200; // 2 hours
|
||||
const CACHE_VERSION = 'ci-v2';
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.GROQ_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ intel: null, fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return new Response(JSON.stringify({ error: 'Payload too large' }), {
|
||||
status: 413,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const { country, code, context } = await request.json();
|
||||
|
||||
if (!country || !code) {
|
||||
return new Response(JSON.stringify({ error: 'country and code required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Cache key includes country code + context hash (context changes as data updates)
|
||||
const contextHash = context ? hashString(JSON.stringify(context)).slice(0, 8) : 'no-ctx';
|
||||
const cacheKey = `${CACHE_VERSION}:${code}:${contextHash}`;
|
||||
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.brief) {
|
||||
console.log('[CountryIntel] Cache hit:', code);
|
||||
return new Response(JSON.stringify({ ...cached, cached: true }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
// Build data context section
|
||||
const dataLines = [];
|
||||
if (context?.score != null) {
|
||||
const changeStr = context.change24h ? ` (${context.change24h > 0 ? '+' : ''}${context.change24h} in 24h)` : '';
|
||||
dataLines.push(`Instability Score: ${context.score}/100 (${context.level || 'unknown'}) — trend: ${context.trend || 'unknown'}${changeStr}`);
|
||||
}
|
||||
if (context?.components) {
|
||||
const c = context.components;
|
||||
dataLines.push(`Score Components: Unrest ${c.unrest ?? '?'}/100, Security ${c.security ?? '?'}/100, Information ${c.information ?? '?'}/100`);
|
||||
}
|
||||
if (context?.protests != null) dataLines.push(`Active protests in/near country (7d): ${context.protests}`);
|
||||
if (context?.militaryFlights != null) dataLines.push(`Military aircraft detected in/near country: ${context.militaryFlights}`);
|
||||
if (context?.militaryVessels != null) dataLines.push(`Military vessels detected in/near country: ${context.militaryVessels}`);
|
||||
if (context?.outages != null) dataLines.push(`Internet outages: ${context.outages}`);
|
||||
if (context?.earthquakes != null) dataLines.push(`Recent earthquakes: ${context.earthquakes}`);
|
||||
if (context?.stockIndex) dataLines.push(`Stock Market Index: ${context.stockIndex}`);
|
||||
if (context?.convergenceScore != null) {
|
||||
dataLines.push(`Signal convergence score: ${context.convergenceScore}/100 (multiple signal types detected: ${(context.signalTypes || []).join(', ')})`);
|
||||
}
|
||||
if (context?.regionalConvergence?.length > 0) {
|
||||
dataLines.push(`\nRegional convergence alerts:`);
|
||||
context.regionalConvergence.forEach(r => dataLines.push(`- ${r}`));
|
||||
}
|
||||
if (context?.headlines?.length > 0) {
|
||||
dataLines.push(`\nRecent headlines mentioning ${country} (${context.headlines.length} found):`);
|
||||
context.headlines.slice(0, 15).forEach((h, i) => dataLines.push(`${i + 1}. ${h}`));
|
||||
}
|
||||
|
||||
const dataSection = dataLines.length > 0
|
||||
? `\nCURRENT SENSOR DATA:\n${dataLines.join('\n')}`
|
||||
: '\nNo real-time sensor data available for this country.';
|
||||
|
||||
const dateStr = new Date().toISOString().split('T')[0];
|
||||
|
||||
const systemPrompt = `You are a senior intelligence analyst providing comprehensive country situation briefs. Current date: ${dateStr}. Donald Trump is the current US President (second term, inaugurated Jan 2025).
|
||||
|
||||
Write a thorough, data-driven intelligence brief for the requested country. Structure:
|
||||
|
||||
1. **Current Situation** — What is happening right now. Reference specific data: instability scores, protest counts, military presence, outages. Explain what the numbers mean in context.
|
||||
|
||||
2. **Military & Security Posture** — Analyze military activity in/near the country. What forces are present? What does the positioning suggest? What are foreign nations doing in this theater?
|
||||
|
||||
3. **Key Risk Factors** — What drives instability or stability. Connect the dots between different signals (protests + outages = potential crackdown? military buildup + diplomatic tensions = escalation risk?). Reference specific headlines.
|
||||
|
||||
4. **Regional Context** — How does this country's situation affect or relate to its neighbors and the broader region? Reference any convergence alerts.
|
||||
|
||||
5. **Outlook & Watch Items** — What to monitor in the near term. Be specific about indicators that would signal escalation or de-escalation.
|
||||
|
||||
Rules:
|
||||
- Be specific and analytical. Reference the data provided (scores, counts, headlines, convergence).
|
||||
- If data shows low activity, say so — don't manufacture threats.
|
||||
- Connect signals: explain what combinations of data points suggest.
|
||||
- 5-6 paragraphs, 300-400 words.
|
||||
- No speculation beyond what the data supports.
|
||||
- Use plain language, not jargon.
|
||||
- If military assets are 0, don't speculate about military presence — say monitoring shows no current military activity.
|
||||
- When referencing a specific headline from the numbered list, cite it as [N] where N is the headline number (e.g. "tensions escalated [3]"). Only cite headlines you directly reference.`;
|
||||
|
||||
const userPrompt = `Country: ${country} (${code})${dataSection}`;
|
||||
|
||||
const groqRes = await fetch(GROQ_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
temperature: 0.4,
|
||||
max_tokens: 900,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!groqRes.ok) {
|
||||
const errText = await groqRes.text();
|
||||
console.error('[CountryIntel] Groq error:', groqRes.status, errText);
|
||||
return new Response(JSON.stringify({ error: 'AI service error', fallback: true }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const groqData = await groqRes.json();
|
||||
const brief = groqData.choices?.[0]?.message?.content || '';
|
||||
|
||||
const result = {
|
||||
brief,
|
||||
country,
|
||||
code,
|
||||
model: MODEL,
|
||||
generatedAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
if (brief) {
|
||||
await setCachedJson(cacheKey, result, CACHE_TTL_SECONDS);
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[CountryIntel] Error:', err);
|
||||
return new Response(JSON.stringify({ error: 'Internal error' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,371 +0,0 @@
|
||||
import { strict as assert } from 'node:assert';
|
||||
import test from 'node:test';
|
||||
import handler, {
|
||||
__resetCyberThreatsState,
|
||||
__testDedupeThreats,
|
||||
__testParseFeodoRecords,
|
||||
} from './cyber-threats.js';
|
||||
|
||||
const ORIGINAL_FETCH = globalThis.fetch;
|
||||
const ORIGINAL_URLHAUS_KEY = process.env.URLHAUS_AUTH_KEY;
|
||||
const ORIGINAL_OTX_KEY = process.env.OTX_API_KEY;
|
||||
const ORIGINAL_ABUSEIPDB_KEY = process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
function makeRequest(path = '/api/cyber-threats', ip = '198.51.100.10') {
|
||||
const headers = new Headers();
|
||||
headers.set('x-forwarded-for', ip);
|
||||
return new Request(`https://worldmonitor.app${path}`, { headers });
|
||||
}
|
||||
|
||||
function jsonResponse(body, status = 200) {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
function textResponse(body, status = 200) {
|
||||
return new Response(body, {
|
||||
status,
|
||||
headers: { 'content-type': 'text/plain' },
|
||||
});
|
||||
}
|
||||
|
||||
// Mock that handles all 5 source URLs + geo enrichment
|
||||
function createMockFetch({ feodo, urlhaus, c2intel, otx, abuseipdb, geo } = {}) {
|
||||
return async (url) => {
|
||||
const target = String(url);
|
||||
if (target.includes('feodotracker.abuse.ch') && feodo) return feodo(target);
|
||||
if (target.includes('urlhaus-api.abuse.ch') && urlhaus) return urlhaus(target);
|
||||
if (target.includes('raw.githubusercontent.com') && target.includes('C2IntelFeeds') && c2intel) return c2intel(target);
|
||||
if (target.includes('otx.alienvault.com') && otx) return otx(target);
|
||||
if (target.includes('api.abuseipdb.com') && abuseipdb) return abuseipdb(target);
|
||||
if ((target.includes('ipwho.is') || target.includes('ipapi.co')) && geo) return geo(target);
|
||||
// Default: return 404 for unconfigured sources
|
||||
return new Response('not found', { status: 404 });
|
||||
};
|
||||
}
|
||||
|
||||
test.afterEach(() => {
|
||||
globalThis.fetch = ORIGINAL_FETCH;
|
||||
process.env.URLHAUS_AUTH_KEY = ORIGINAL_URLHAUS_KEY;
|
||||
process.env.OTX_API_KEY = ORIGINAL_OTX_KEY;
|
||||
process.env.ABUSEIPDB_API_KEY = ORIGINAL_ABUSEIPDB_KEY;
|
||||
__resetCyberThreatsState();
|
||||
});
|
||||
|
||||
test('Feodo parser accepts online and recent offline entries, filters stale', () => {
|
||||
const nowMs = Date.parse('2026-02-15T12:00:00.000Z');
|
||||
const records = [
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
first_seen: '2026-02-14 10:00:00 UTC',
|
||||
last_online: '2026-02-15 10:00:00 UTC',
|
||||
malware: 'QakBot',
|
||||
},
|
||||
{
|
||||
ip_address: '5.6.7.8',
|
||||
status: 'offline',
|
||||
first_seen: '2026-02-14 10:00:00 UTC',
|
||||
last_online: '2026-02-15 10:00:00 UTC',
|
||||
malware: 'Emotet',
|
||||
},
|
||||
{
|
||||
ip_address: '9.9.9.9',
|
||||
status: 'online',
|
||||
first_seen: '2025-10-01 10:00:00 UTC',
|
||||
last_online: '2025-10-02 10:00:00 UTC',
|
||||
malware: 'generic',
|
||||
},
|
||||
{
|
||||
ip_address: '2.2.2.2',
|
||||
first_seen: '2026-02-14 10:00:00 UTC',
|
||||
last_online: '2026-02-15 10:00:00 UTC',
|
||||
malware: 'generic',
|
||||
},
|
||||
];
|
||||
|
||||
const parsed = __testParseFeodoRecords(records, { nowMs, days: 14 });
|
||||
// online + offline (recent) + no-status (recent) = 3; stale (9.9.9.9) filtered
|
||||
assert.equal(parsed.length, 3);
|
||||
assert.equal(parsed[0].indicator, '1.2.3.4');
|
||||
assert.equal(parsed[0].severity, 'critical');
|
||||
assert.equal(parsed[1].indicator, '5.6.7.8');
|
||||
assert.equal(parsed[1].severity, 'medium');
|
||||
assert.equal(parsed[0].firstSeen?.endsWith('Z'), true);
|
||||
assert.equal(parsed[0].lastSeen?.endsWith('Z'), true);
|
||||
});
|
||||
|
||||
test('dedupes by source + indicatorType + indicator', () => {
|
||||
const deduped = __testDedupeThreats([
|
||||
{
|
||||
id: 'a',
|
||||
source: 'feodo',
|
||||
type: 'c2_server',
|
||||
indicatorType: 'ip',
|
||||
indicator: '1.2.3.4',
|
||||
severity: 'high',
|
||||
tags: ['a'],
|
||||
firstSeen: '2026-02-10T00:00:00.000Z',
|
||||
lastSeen: '2026-02-11T00:00:00.000Z',
|
||||
},
|
||||
{
|
||||
id: 'b',
|
||||
source: 'feodo',
|
||||
type: 'c2_server',
|
||||
indicatorType: 'ip',
|
||||
indicator: '1.2.3.4',
|
||||
severity: 'critical',
|
||||
tags: ['b'],
|
||||
firstSeen: '2026-02-12T00:00:00.000Z',
|
||||
lastSeen: '2026-02-13T00:00:00.000Z',
|
||||
},
|
||||
{
|
||||
id: 'c',
|
||||
source: 'urlhaus',
|
||||
type: 'malicious_url',
|
||||
indicatorType: 'domain',
|
||||
indicator: 'bad.example',
|
||||
severity: 'medium',
|
||||
tags: [],
|
||||
firstSeen: '2026-02-11T00:00:00.000Z',
|
||||
lastSeen: '2026-02-11T01:00:00.000Z',
|
||||
},
|
||||
]);
|
||||
|
||||
assert.equal(deduped.length, 2);
|
||||
const feodo = deduped.find((item) => item.source === 'feodo');
|
||||
assert.equal(feodo?.severity, 'critical');
|
||||
assert.equal(feodo?.tags.includes('a'), true);
|
||||
assert.equal(feodo?.tags.includes('b'), true);
|
||||
});
|
||||
|
||||
test('API aggregates from all 5 sources', async () => {
|
||||
process.env.URLHAUS_AUTH_KEY = 'test-key';
|
||||
process.env.OTX_API_KEY = 'test-otx';
|
||||
process.env.ABUSEIPDB_API_KEY = 'test-abuse';
|
||||
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]),
|
||||
urlhaus: () => jsonResponse({
|
||||
urls: [{
|
||||
url: 'http://5.5.5.5/malware.exe',
|
||||
host: '5.5.5.5',
|
||||
url_status: 'online',
|
||||
threat: 'malware_download',
|
||||
tags: ['malware'],
|
||||
dateadded: '2026-02-14T08:00:00.000Z',
|
||||
latitude: 48.86,
|
||||
longitude: 2.35,
|
||||
country: 'FR',
|
||||
}],
|
||||
}),
|
||||
c2intel: () => textResponse(
|
||||
'#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP\n10.10.10.11,Possible Metasploit C2 IP',
|
||||
),
|
||||
otx: () => jsonResponse({
|
||||
results: [{
|
||||
indicator: '20.20.20.20',
|
||||
title: 'APT threat',
|
||||
tags: ['apt', 'c2'],
|
||||
created: '2026-02-13T00:00:00.000Z',
|
||||
modified: '2026-02-14T00:00:00.000Z',
|
||||
}],
|
||||
}),
|
||||
abuseipdb: () => jsonResponse({
|
||||
data: [{
|
||||
ipAddress: '30.30.30.30',
|
||||
abuseConfidenceScore: 98,
|
||||
lastReportedAt: '2026-02-15T06:00:00.000Z',
|
||||
countryCode: 'CN',
|
||||
latitude: 39.9,
|
||||
longitude: 116.4,
|
||||
}],
|
||||
}),
|
||||
geo: () => jsonResponse({ success: true, latitude: 40.0, longitude: -74.0, country_code: 'US' }),
|
||||
});
|
||||
|
||||
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.20'));
|
||||
assert.equal(response.status, 200);
|
||||
|
||||
const body = await response.json();
|
||||
assert.equal(body.success, true);
|
||||
assert.equal(body.sources.feodo.ok, true);
|
||||
assert.equal(body.sources.urlhaus.ok, true);
|
||||
assert.equal(body.sources.c2intel.ok, true);
|
||||
assert.equal(body.sources.otx.ok, true);
|
||||
assert.equal(body.sources.abuseipdb.ok, true);
|
||||
// 5 sources, all with coords (3 native + 3 via geo enrichment mock)
|
||||
assert.equal(body.data.length >= 5, true);
|
||||
});
|
||||
|
||||
test('API works with only free sources when keys missing', async () => {
|
||||
delete process.env.URLHAUS_AUTH_KEY;
|
||||
delete process.env.OTX_API_KEY;
|
||||
delete process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]),
|
||||
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
|
||||
});
|
||||
|
||||
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.11'));
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(response.headers.get('X-Cache'), 'MISS');
|
||||
|
||||
const body = await response.json();
|
||||
assert.equal(body.success, true);
|
||||
assert.equal(body.partial, false);
|
||||
assert.equal(body.sources.feodo.ok, true);
|
||||
assert.equal(body.sources.c2intel.ok, true);
|
||||
assert.equal(body.sources.urlhaus.ok, false);
|
||||
assert.equal(body.sources.urlhaus.reason, 'missing_auth_key');
|
||||
assert.equal(body.sources.otx.ok, false);
|
||||
assert.equal(body.sources.otx.reason, 'missing_api_key');
|
||||
assert.equal(body.sources.abuseipdb.ok, false);
|
||||
assert.equal(body.sources.abuseipdb.reason, 'missing_api_key');
|
||||
assert.equal(Array.isArray(body.data), true);
|
||||
});
|
||||
|
||||
test('API marks partial=true when URLhaus is enabled but fails', async () => {
|
||||
process.env.URLHAUS_AUTH_KEY = 'test-key';
|
||||
delete process.env.OTX_API_KEY;
|
||||
delete process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]),
|
||||
urlhaus: () => new Response('boom', { status: 500 }),
|
||||
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
|
||||
});
|
||||
|
||||
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.12'));
|
||||
assert.equal(response.status, 200);
|
||||
|
||||
const body = await response.json();
|
||||
assert.equal(body.success, true);
|
||||
assert.equal(body.partial, true);
|
||||
assert.equal(body.sources.urlhaus.ok, false);
|
||||
assert.equal(body.sources.urlhaus.reason, 'urlhaus_http_500');
|
||||
});
|
||||
|
||||
test('API returns memory cache hit on repeated request', async () => {
|
||||
delete process.env.URLHAUS_AUTH_KEY;
|
||||
delete process.env.OTX_API_KEY;
|
||||
delete process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
let feodoCalls = 0;
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => {
|
||||
feodoCalls += 1;
|
||||
return jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]);
|
||||
},
|
||||
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
|
||||
});
|
||||
|
||||
const first = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.13'));
|
||||
assert.equal(first.status, 200);
|
||||
assert.equal(first.headers.get('X-Cache'), 'MISS');
|
||||
assert.equal(feodoCalls, 1);
|
||||
|
||||
globalThis.fetch = async () => {
|
||||
throw new Error('network should not be hit for memory cache');
|
||||
};
|
||||
|
||||
const second = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.13'));
|
||||
assert.equal(second.status, 200);
|
||||
assert.equal(second.headers.get('X-Cache'), 'MEMORY-HIT');
|
||||
assert.equal(feodoCalls, 1);
|
||||
});
|
||||
|
||||
test('API returns stale fallback when upstream fails after fresh cache TTL', async () => {
|
||||
delete process.env.URLHAUS_AUTH_KEY;
|
||||
delete process.env.OTX_API_KEY;
|
||||
delete process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
const baseNow = Date.parse('2026-02-15T12:00:00.000Z');
|
||||
const originalDateNow = Date.now;
|
||||
Date.now = () => baseNow;
|
||||
|
||||
try {
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]),
|
||||
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
|
||||
});
|
||||
|
||||
const first = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.14'));
|
||||
assert.equal(first.status, 200);
|
||||
assert.equal(first.headers.get('X-Cache'), 'MISS');
|
||||
|
||||
Date.now = () => baseNow + (11 * 60 * 1000);
|
||||
globalThis.fetch = async () => {
|
||||
throw new Error('forced upstream failure');
|
||||
};
|
||||
|
||||
const stale = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.14'));
|
||||
assert.equal(stale.status, 200);
|
||||
assert.equal(stale.headers.get('X-Cache'), 'STALE');
|
||||
|
||||
const body = await stale.json();
|
||||
assert.equal(body.success, true);
|
||||
assert.equal(Array.isArray(body.data), true);
|
||||
assert.equal(body.data.length >= 1, true);
|
||||
} finally {
|
||||
Date.now = originalDateNow;
|
||||
}
|
||||
});
|
||||
@@ -1,77 +1,16 @@
|
||||
// Tech Events API - Parses Techmeme ICS feed and dev.events RSS, returns structured events
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
/**
|
||||
* Comprehensive city geocoding database (500+ cities worldwide).
|
||||
* Extracted from the legacy api/tech-events.js endpoint.
|
||||
*/
|
||||
|
||||
const ICS_URL = 'https://www.techmeme.com/newsy_events.ics';
|
||||
const DEV_EVENTS_RSS = 'https://dev.events/rss.xml';
|
||||
export interface CityCoord {
|
||||
lat: number;
|
||||
lng: number;
|
||||
country: string;
|
||||
virtual?: boolean;
|
||||
}
|
||||
|
||||
// Curated major tech events that may fall off limited RSS feeds
|
||||
// These are manually maintained for important conferences
|
||||
const CURATED_EVENTS = [
|
||||
{
|
||||
id: 'step-dubai-2026',
|
||||
title: 'STEP Dubai 2026',
|
||||
type: 'conference',
|
||||
location: 'Dubai Internet City, Dubai',
|
||||
coords: { lat: 25.0956, lng: 55.1548, country: 'UAE', original: 'Dubai Internet City, Dubai' },
|
||||
startDate: '2026-02-11',
|
||||
endDate: '2026-02-12',
|
||||
url: 'https://dubai.stepconference.com',
|
||||
source: 'curated',
|
||||
description: 'Intelligence Everywhere: The AI Economy - 8,000+ attendees, 400+ startups',
|
||||
},
|
||||
{
|
||||
id: 'gitex-global-2026',
|
||||
title: 'GITEX Global 2026',
|
||||
type: 'conference',
|
||||
location: 'Dubai World Trade Centre, Dubai',
|
||||
coords: { lat: 25.2285, lng: 55.2867, country: 'UAE', original: 'Dubai World Trade Centre, Dubai' },
|
||||
startDate: '2026-12-07',
|
||||
endDate: '2026-12-11',
|
||||
url: 'https://www.gitex.com',
|
||||
source: 'curated',
|
||||
description: 'World\'s largest tech & startup show',
|
||||
},
|
||||
{
|
||||
id: 'token2049-dubai-2026',
|
||||
title: 'TOKEN2049 Dubai 2026',
|
||||
type: 'conference',
|
||||
location: 'Dubai, UAE',
|
||||
coords: { lat: 25.2048, lng: 55.2708, country: 'UAE', original: 'Dubai, UAE' },
|
||||
startDate: '2026-04-29',
|
||||
endDate: '2026-04-30',
|
||||
url: 'https://www.token2049.com',
|
||||
source: 'curated',
|
||||
description: 'Premier crypto event in Dubai',
|
||||
},
|
||||
{
|
||||
id: 'collision-2026',
|
||||
title: 'Collision 2026',
|
||||
type: 'conference',
|
||||
location: 'Toronto, Canada',
|
||||
coords: { lat: 43.6532, lng: -79.3832, country: 'Canada', original: 'Toronto, Canada' },
|
||||
startDate: '2026-06-22',
|
||||
endDate: '2026-06-25',
|
||||
url: 'https://collisionconf.com',
|
||||
source: 'curated',
|
||||
description: 'North America\'s fastest growing tech conference',
|
||||
},
|
||||
{
|
||||
id: 'web-summit-2026',
|
||||
title: 'Web Summit 2026',
|
||||
type: 'conference',
|
||||
location: 'Lisbon, Portugal',
|
||||
coords: { lat: 38.7223, lng: -9.1393, country: 'Portugal', original: 'Lisbon, Portugal' },
|
||||
startDate: '2026-11-02',
|
||||
endDate: '2026-11-05',
|
||||
url: 'https://websummit.com',
|
||||
source: 'curated',
|
||||
description: 'The world\'s premier tech conference',
|
||||
},
|
||||
];
|
||||
|
||||
// Comprehensive city geocoding database (500+ cities worldwide)
|
||||
const CITY_COORDS = {
|
||||
export const CITY_COORDS: Record<string, CityCoord> = {
|
||||
// North America - USA
|
||||
'san francisco': { lat: 37.7749, lng: -122.4194, country: 'USA' },
|
||||
'san jose': { lat: 37.3382, lng: -121.8863, country: 'USA' },
|
||||
@@ -164,7 +103,7 @@ const CITY_COORDS = {
|
||||
'tijuana': { lat: 32.5149, lng: -117.0382, country: 'Mexico' },
|
||||
'cancun': { lat: 21.1619, lng: -86.8515, country: 'Mexico' },
|
||||
'panama city': { lat: 8.9824, lng: -79.5199, country: 'Panama' },
|
||||
'san jose': { lat: 9.9281, lng: -84.0907, country: 'Costa Rica' },
|
||||
'san jose cr': { lat: 9.9281, lng: -84.0907, country: 'Costa Rica' },
|
||||
|
||||
// South America
|
||||
'sao paulo': { lat: -23.5505, lng: -46.6333, country: 'Brazil' },
|
||||
@@ -176,9 +115,9 @@ const CITY_COORDS = {
|
||||
'buenos aires': { lat: -34.6037, lng: -58.3816, country: 'Argentina' },
|
||||
'santiago': { lat: -33.4489, lng: -70.6693, country: 'Chile' },
|
||||
'bogota': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
|
||||
'bogotá': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
|
||||
'bogot\u00e1': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
|
||||
'medellin': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
|
||||
'medellín': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
|
||||
'medell\u00edn': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
|
||||
'lima': { lat: -12.0464, lng: -77.0428, country: 'Peru' },
|
||||
'caracas': { lat: 10.4806, lng: -66.9036, country: 'Venezuela' },
|
||||
'montevideo': { lat: -34.9011, lng: -56.1645, country: 'Uruguay' },
|
||||
@@ -186,7 +125,7 @@ const CITY_COORDS = {
|
||||
|
||||
// Europe - UK & Ireland
|
||||
'london': { lat: 51.5074, lng: -0.1278, country: 'UK' },
|
||||
'cambridge': { lat: 52.2053, lng: 0.1218, country: 'UK' },
|
||||
'cambridge uk': { lat: 52.2053, lng: 0.1218, country: 'UK' },
|
||||
'oxford': { lat: 51.7520, lng: -1.2577, country: 'UK' },
|
||||
'manchester': { lat: 53.4808, lng: -2.2426, country: 'UK' },
|
||||
'birmingham': { lat: 52.4862, lng: -1.8904, country: 'UK' },
|
||||
@@ -214,12 +153,12 @@ const CITY_COORDS = {
|
||||
'monaco': { lat: 43.7384, lng: 7.4246, country: 'Monaco' },
|
||||
'berlin': { lat: 52.5200, lng: 13.4050, country: 'Germany' },
|
||||
'munich': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
|
||||
'münchen': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
|
||||
'm\u00fcnchen': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
|
||||
'frankfurt': { lat: 50.1109, lng: 8.6821, country: 'Germany' },
|
||||
'hamburg': { lat: 53.5511, lng: 9.9937, country: 'Germany' },
|
||||
'cologne': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
|
||||
'köln': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
|
||||
'düsseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
|
||||
'k\u00f6ln': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
|
||||
'd\u00fcsseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
|
||||
'dusseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
|
||||
'stuttgart': { lat: 48.7758, lng: 9.1829, country: 'Germany' },
|
||||
'hanover': { lat: 52.3759, lng: 9.7320, country: 'Germany' },
|
||||
@@ -237,9 +176,9 @@ const CITY_COORDS = {
|
||||
'ghent': { lat: 51.0543, lng: 3.7174, country: 'Belgium' },
|
||||
'luxembourg': { lat: 49.6116, lng: 6.1319, country: 'Luxembourg' },
|
||||
'zurich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
|
||||
'zürich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
|
||||
'z\u00fcrich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
|
||||
'geneva': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
|
||||
'genève': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
|
||||
'gen\u00e8ve': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
|
||||
'basel': { lat: 47.5596, lng: 7.5886, country: 'Switzerland' },
|
||||
'bern': { lat: 46.9480, lng: 7.4474, country: 'Switzerland' },
|
||||
'lausanne': { lat: 46.5197, lng: 6.6323, country: 'Switzerland' },
|
||||
@@ -257,7 +196,7 @@ const CITY_COORDS = {
|
||||
'seville': { lat: 37.3891, lng: -5.9845, country: 'Spain' },
|
||||
'sevilla': { lat: 37.3891, lng: -5.9845, country: 'Spain' },
|
||||
'malaga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
|
||||
'málaga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
|
||||
'm\u00e1laga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
|
||||
'bilbao': { lat: 43.2630, lng: -2.9350, country: 'Spain' },
|
||||
'lisbon': { lat: 38.7223, lng: -9.1393, country: 'Portugal' },
|
||||
'lisboa': { lat: 38.7223, lng: -9.1393, country: 'Portugal' },
|
||||
@@ -283,11 +222,11 @@ const CITY_COORDS = {
|
||||
// Europe - Northern
|
||||
'stockholm': { lat: 59.3293, lng: 18.0686, country: 'Sweden' },
|
||||
'gothenburg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
|
||||
'göteborg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
|
||||
'malmö': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
|
||||
'g\u00f6teborg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
|
||||
'malm\u00f6': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
|
||||
'malmo': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
|
||||
'copenhagen': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
|
||||
'københavn': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
|
||||
'k\u00f8benhavn': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
|
||||
'aarhus': { lat: 56.1629, lng: 10.2039, country: 'Denmark' },
|
||||
'oslo': { lat: 59.9139, lng: 10.7522, country: 'Norway' },
|
||||
'bergen': { lat: 60.3913, lng: 5.3221, country: 'Norway' },
|
||||
@@ -300,16 +239,16 @@ const CITY_COORDS = {
|
||||
'warsaw': { lat: 52.2297, lng: 21.0122, country: 'Poland' },
|
||||
'warszawa': { lat: 52.2297, lng: 21.0122, country: 'Poland' },
|
||||
'krakow': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
|
||||
'kraków': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
|
||||
'krak\u00f3w': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
|
||||
'wroclaw': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
|
||||
'wrocław': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
|
||||
'wroc\u0142aw': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
|
||||
'gdansk': { lat: 54.3520, lng: 18.6466, country: 'Poland' },
|
||||
'prague': { lat: 50.0755, lng: 14.4378, country: 'Czech Republic' },
|
||||
'praha': { lat: 50.0755, lng: 14.4378, country: 'Czech Republic' },
|
||||
'brno': { lat: 49.1951, lng: 16.6068, country: 'Czech Republic' },
|
||||
'budapest': { lat: 47.4979, lng: 19.0402, country: 'Hungary' },
|
||||
'bucharest': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
|
||||
'bucurești': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
|
||||
'bucure\u0219ti': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
|
||||
'cluj-napoca': { lat: 46.7712, lng: 23.6236, country: 'Romania' },
|
||||
'sofia': { lat: 42.6977, lng: 23.3219, country: 'Bulgaria' },
|
||||
'belgrade': { lat: 44.7866, lng: 20.4489, country: 'Serbia' },
|
||||
@@ -464,274 +403,3 @@ const CITY_COORDS = {
|
||||
'virtual': { lat: 0, lng: 0, country: 'Virtual', virtual: true },
|
||||
'hybrid': { lat: 0, lng: 0, country: 'Virtual', virtual: true },
|
||||
};
|
||||
|
||||
function normalizeLocation(location) {
|
||||
if (!location) return null;
|
||||
|
||||
// Clean up the location string
|
||||
let normalized = location.toLowerCase().trim();
|
||||
|
||||
// Remove common suffixes/prefixes
|
||||
normalized = normalized.replace(/^hybrid:\s*/i, '');
|
||||
normalized = normalized.replace(/,\s*(usa|us|uk|canada)$/i, '');
|
||||
|
||||
// Direct lookup
|
||||
if (CITY_COORDS[normalized]) {
|
||||
return { ...CITY_COORDS[normalized], original: location };
|
||||
}
|
||||
|
||||
// Try removing state/country suffix
|
||||
const parts = normalized.split(',');
|
||||
if (parts.length > 1) {
|
||||
const city = parts[0].trim();
|
||||
if (CITY_COORDS[city]) {
|
||||
return { ...CITY_COORDS[city], original: location };
|
||||
}
|
||||
}
|
||||
|
||||
// Try fuzzy match (contains)
|
||||
for (const [key, coords] of Object.entries(CITY_COORDS)) {
|
||||
if (normalized.includes(key) || key.includes(normalized)) {
|
||||
return { ...coords, original: location };
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
function parseICS(icsText) {
|
||||
const events = [];
|
||||
const eventBlocks = icsText.split('BEGIN:VEVENT').slice(1);
|
||||
|
||||
for (const block of eventBlocks) {
|
||||
const summaryMatch = block.match(/SUMMARY:(.+)/);
|
||||
const locationMatch = block.match(/LOCATION:(.+)/);
|
||||
const dtstartMatch = block.match(/DTSTART;VALUE=DATE:(\d+)/);
|
||||
const dtendMatch = block.match(/DTEND;VALUE=DATE:(\d+)/);
|
||||
const urlMatch = block.match(/URL:(.+)/);
|
||||
const uidMatch = block.match(/UID:(.+)/);
|
||||
|
||||
if (summaryMatch && dtstartMatch) {
|
||||
const summary = summaryMatch[1].trim();
|
||||
const location = locationMatch ? locationMatch[1].trim() : null;
|
||||
const startDate = dtstartMatch[1];
|
||||
const endDate = dtendMatch ? dtendMatch[1] : startDate;
|
||||
const url = urlMatch ? urlMatch[1].trim() : null;
|
||||
const uid = uidMatch ? uidMatch[1].trim() : null;
|
||||
|
||||
// Determine event type
|
||||
let type = 'other';
|
||||
if (summary.startsWith('Earnings:')) type = 'earnings';
|
||||
else if (summary.startsWith('IPO')) type = 'ipo';
|
||||
else if (location) type = 'conference';
|
||||
|
||||
// Parse coordinates if location exists
|
||||
const coords = normalizeLocation(location);
|
||||
|
||||
events.push({
|
||||
id: uid,
|
||||
title: summary,
|
||||
type,
|
||||
location: location,
|
||||
coords: coords,
|
||||
startDate: `${startDate.slice(0, 4)}-${startDate.slice(4, 6)}-${startDate.slice(6, 8)}`,
|
||||
endDate: `${endDate.slice(0, 4)}-${endDate.slice(4, 6)}-${endDate.slice(6, 8)}`,
|
||||
url: url,
|
||||
source: 'techmeme',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return events.sort((a, b) => a.startDate.localeCompare(b.startDate));
|
||||
}
|
||||
|
||||
function parseDevEventsRSS(rssText) {
|
||||
const events = [];
|
||||
|
||||
// Simple regex-based RSS parsing for edge runtime
|
||||
const itemMatches = rssText.matchAll(/<item>([\s\S]*?)<\/item>/g);
|
||||
|
||||
for (const match of itemMatches) {
|
||||
const item = match[1];
|
||||
|
||||
const titleMatch = item.match(/<title><!\[CDATA\[(.*?)\]\]><\/title>|<title>(.*?)<\/title>/);
|
||||
const linkMatch = item.match(/<link>(.*?)<\/link>/);
|
||||
const descMatch = item.match(/<description><!\[CDATA\[(.*?)\]\]><\/description>|<description>(.*?)<\/description>/s);
|
||||
const guidMatch = item.match(/<guid[^>]*>(.*?)<\/guid>/);
|
||||
|
||||
const title = titleMatch ? (titleMatch[1] || titleMatch[2]) : null;
|
||||
const link = linkMatch ? linkMatch[1] : null;
|
||||
const description = descMatch ? (descMatch[1] || descMatch[2]) : '';
|
||||
const guid = guidMatch ? guidMatch[1] : null;
|
||||
|
||||
if (!title) continue;
|
||||
|
||||
// Parse date from description: "EventName is happening on Month Day, Year"
|
||||
const dateMatch = description.match(/on\s+(\w+\s+\d{1,2},?\s+\d{4})/i);
|
||||
let startDate = null;
|
||||
if (dateMatch) {
|
||||
const parsed = new Date(dateMatch[1]);
|
||||
if (!isNaN(parsed.getTime())) {
|
||||
startDate = parsed.toISOString().split('T')[0];
|
||||
}
|
||||
}
|
||||
|
||||
// Parse location from description: various formats
|
||||
let location = null;
|
||||
const locationMatch = description.match(/(?:in|at)\s+([A-Za-z\s]+,\s*[A-Za-z\s]+)(?:\.|$)/i) ||
|
||||
description.match(/Location:\s*([^<\n]+)/i);
|
||||
if (locationMatch) {
|
||||
location = locationMatch[1].trim();
|
||||
}
|
||||
// Check for "Online" events
|
||||
if (description.toLowerCase().includes('online')) {
|
||||
location = 'Online';
|
||||
}
|
||||
|
||||
// Skip events without valid dates or in the past
|
||||
if (!startDate) continue;
|
||||
const eventDate = new Date(startDate);
|
||||
const now = new Date();
|
||||
now.setHours(0, 0, 0, 0);
|
||||
if (eventDate < now) continue;
|
||||
|
||||
const coords = location && location !== 'Online' ? normalizeLocation(location) : null;
|
||||
if (location === 'Online') {
|
||||
// Mark as virtual
|
||||
if (coords) coords.virtual = true;
|
||||
}
|
||||
|
||||
events.push({
|
||||
id: guid || `dev-events-${title.slice(0, 20)}`,
|
||||
title: title,
|
||||
type: 'conference',
|
||||
location: location,
|
||||
coords: coords || (location === 'Online' ? { virtual: true, original: 'Online' } : null),
|
||||
startDate: startDate,
|
||||
endDate: startDate, // RSS doesn't have end date
|
||||
url: link,
|
||||
source: 'dev.events',
|
||||
});
|
||||
}
|
||||
|
||||
return events;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const type = url.searchParams.get('type'); // 'all', 'conferences', 'earnings', 'ipo'
|
||||
const mappable = url.searchParams.get('mappable') === 'true'; // Only return events with coords
|
||||
const limit = parseInt(url.searchParams.get('limit')) || 0; // Max events (0 = unlimited)
|
||||
const days = parseInt(url.searchParams.get('days')) || 0; // Events within N days (0 = unlimited)
|
||||
|
||||
try {
|
||||
// Fetch both sources in parallel
|
||||
const [icsResponse, rssResponse] = await Promise.allSettled([
|
||||
fetch(ICS_URL, {
|
||||
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0)' },
|
||||
}),
|
||||
fetch(DEV_EVENTS_RSS, {
|
||||
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0)' },
|
||||
}),
|
||||
]);
|
||||
|
||||
let events = [];
|
||||
|
||||
// Parse Techmeme ICS
|
||||
if (icsResponse.status === 'fulfilled' && icsResponse.value.ok) {
|
||||
const icsText = await icsResponse.value.text();
|
||||
events.push(...parseICS(icsText));
|
||||
} else {
|
||||
console.warn('Failed to fetch Techmeme ICS');
|
||||
}
|
||||
|
||||
// Parse dev.events RSS
|
||||
if (rssResponse.status === 'fulfilled' && rssResponse.value.ok) {
|
||||
const rssText = await rssResponse.value.text();
|
||||
const devEvents = parseDevEventsRSS(rssText);
|
||||
events.push(...devEvents);
|
||||
} else {
|
||||
console.warn('Failed to fetch dev.events RSS');
|
||||
}
|
||||
|
||||
// Add curated events (major conferences that may fall off limited RSS feeds)
|
||||
const now = new Date();
|
||||
now.setHours(0, 0, 0, 0);
|
||||
for (const curated of CURATED_EVENTS) {
|
||||
const eventDate = new Date(curated.startDate);
|
||||
if (eventDate >= now) {
|
||||
events.push(curated);
|
||||
}
|
||||
}
|
||||
|
||||
// Deduplicate by title similarity (rough match)
|
||||
const seen = new Set();
|
||||
events = events.filter(e => {
|
||||
const key = e.title.toLowerCase().replace(/[^a-z0-9]/g, '').slice(0, 30);
|
||||
if (seen.has(key)) return false;
|
||||
seen.add(key);
|
||||
return true;
|
||||
});
|
||||
|
||||
// Sort by date
|
||||
events.sort((a, b) => a.startDate.localeCompare(b.startDate));
|
||||
|
||||
// Filter by type if specified
|
||||
if (type && type !== 'all') {
|
||||
events = events.filter(e => e.type === type);
|
||||
}
|
||||
|
||||
// Filter to only mappable events if requested
|
||||
if (mappable) {
|
||||
events = events.filter(e => e.coords && !e.coords.virtual);
|
||||
}
|
||||
|
||||
// Filter by time range if specified
|
||||
if (days > 0) {
|
||||
const cutoff = new Date();
|
||||
cutoff.setDate(cutoff.getDate() + days);
|
||||
events = events.filter(e => new Date(e.startDate) <= cutoff);
|
||||
}
|
||||
|
||||
// Apply limit if specified
|
||||
if (limit > 0) {
|
||||
events = events.slice(0, limit);
|
||||
}
|
||||
|
||||
// Add metadata
|
||||
const conferences = events.filter(e => e.type === 'conference');
|
||||
const mappableCount = conferences.filter(e => e.coords && !e.coords.virtual).length;
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
success: true,
|
||||
count: events.length,
|
||||
conferenceCount: conferences.length,
|
||||
mappableCount,
|
||||
lastUpdated: new Date().toISOString(),
|
||||
events,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Tech events error:', error);
|
||||
return new Response(JSON.stringify({
|
||||
success: false,
|
||||
error: error.message,
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -1,11 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler() {
|
||||
return new Response(JSON.stringify({ error: 'Not found' }), {
|
||||
status: 404,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
},
|
||||
});
|
||||
}
|
||||
+31
-1
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const RELEASES_URL = 'https://api.github.com/repos/koala73/worldmonitor/releases/latest';
|
||||
@@ -11,9 +12,35 @@ const PLATFORM_PATTERNS = {
|
||||
'linux-appimage': (name) => name.endsWith('_amd64.AppImage'),
|
||||
};
|
||||
|
||||
const VARIANT_IDENTIFIERS = {
|
||||
full: ['worldmonitor'],
|
||||
world: ['worldmonitor'],
|
||||
tech: ['techmonitor'],
|
||||
finance: ['financemonitor'],
|
||||
};
|
||||
|
||||
function canonicalAssetName(name) {
|
||||
return String(name || '').toLowerCase().replace(/[^a-z0-9]+/g, '');
|
||||
}
|
||||
|
||||
function findAssetForVariant(assets, variant, platformMatcher) {
|
||||
const identifiers = VARIANT_IDENTIFIERS[variant] ?? null;
|
||||
if (!identifiers) return null;
|
||||
|
||||
return assets.find((asset) => {
|
||||
const assetName = String(asset?.name || '');
|
||||
const normalizedAssetName = canonicalAssetName(assetName);
|
||||
const hasVariantIdentifier = identifiers.some((identifier) =>
|
||||
normalizedAssetName.includes(identifier)
|
||||
);
|
||||
return hasVariantIdentifier && platformMatcher(assetName);
|
||||
}) ?? null;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const url = new URL(req.url);
|
||||
const platform = url.searchParams.get('platform');
|
||||
const variant = (url.searchParams.get('variant') || '').toLowerCase();
|
||||
|
||||
if (!platform || !PLATFORM_PATTERNS[platform]) {
|
||||
return Response.redirect(RELEASES_PAGE, 302);
|
||||
@@ -33,7 +60,10 @@ export default async function handler(req) {
|
||||
|
||||
const release = await res.json();
|
||||
const matcher = PLATFORM_PATTERNS[platform];
|
||||
const asset = release.assets?.find((a) => matcher(a.name));
|
||||
const assets = Array.isArray(release.assets) ? release.assets : [];
|
||||
const asset = variant
|
||||
? findAssetForVariant(assets, variant, matcher)
|
||||
: assets.find((a) => matcher(String(a?.name || '')));
|
||||
|
||||
if (!asset) {
|
||||
return Response.redirect(RELEASES_PAGE, 302);
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const response = await fetch(
|
||||
'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson',
|
||||
{
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,163 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_TTL = 900;
|
||||
let cachedResponse = null;
|
||||
let cacheTimestamp = 0;
|
||||
|
||||
const ETF_LIST = [
|
||||
{ ticker: 'IBIT', issuer: 'BlackRock' },
|
||||
{ ticker: 'FBTC', issuer: 'Fidelity' },
|
||||
{ ticker: 'ARKB', issuer: 'ARK/21Shares' },
|
||||
{ ticker: 'BITB', issuer: 'Bitwise' },
|
||||
{ ticker: 'GBTC', issuer: 'Grayscale' },
|
||||
{ ticker: 'HODL', issuer: 'VanEck' },
|
||||
{ ticker: 'BRRR', issuer: 'Valkyrie' },
|
||||
{ ticker: 'EZBC', issuer: 'Franklin' },
|
||||
{ ticker: 'BTCO', issuer: 'Invesco' },
|
||||
{ ticker: 'BTCW', issuer: 'WisdomTree' },
|
||||
];
|
||||
|
||||
async function fetchChart(ticker) {
|
||||
const url = `https://query1.finance.yahoo.com/v8/finance/chart/${ticker}?range=5d&interval=1d`;
|
||||
const controller = new AbortController();
|
||||
const id = setTimeout(() => controller.abort(), 8000);
|
||||
try {
|
||||
const res = await fetch(url, { signal: controller.signal });
|
||||
if (!res.ok) return null;
|
||||
return await res.json();
|
||||
} catch {
|
||||
return null;
|
||||
} finally {
|
||||
clearTimeout(id);
|
||||
}
|
||||
}
|
||||
|
||||
function parseChartData(chart, ticker, issuer) {
|
||||
try {
|
||||
const result = chart?.chart?.result?.[0];
|
||||
if (!result) return null;
|
||||
|
||||
const quote = result.indicators?.quote?.[0];
|
||||
const closes = quote?.close || [];
|
||||
const volumes = quote?.volume || [];
|
||||
|
||||
const validCloses = closes.filter(p => p != null);
|
||||
const validVolumes = volumes.filter(v => v != null);
|
||||
|
||||
if (validCloses.length < 2) return null;
|
||||
|
||||
const latestPrice = validCloses[validCloses.length - 1];
|
||||
const prevPrice = validCloses[validCloses.length - 2];
|
||||
const priceChange = prevPrice ? ((latestPrice - prevPrice) / prevPrice * 100) : 0;
|
||||
|
||||
const latestVolume = validVolumes.length > 0 ? validVolumes[validVolumes.length - 1] : 0;
|
||||
const avgVolume = validVolumes.length > 1
|
||||
? validVolumes.slice(0, -1).reduce((a, b) => a + b, 0) / (validVolumes.length - 1)
|
||||
: latestVolume;
|
||||
|
||||
// Estimate flow direction from price change + volume
|
||||
const volumeRatio = avgVolume > 0 ? latestVolume / avgVolume : 1;
|
||||
const direction = priceChange > 0.1 ? 'inflow' : priceChange < -0.1 ? 'outflow' : 'neutral';
|
||||
const estFlowMagnitude = latestVolume * latestPrice * (priceChange > 0 ? 1 : -1) * 0.1;
|
||||
|
||||
return {
|
||||
ticker,
|
||||
issuer,
|
||||
price: +latestPrice.toFixed(2),
|
||||
priceChange: +priceChange.toFixed(2),
|
||||
volume: latestVolume,
|
||||
avgVolume: Math.round(avgVolume),
|
||||
volumeRatio: +volumeRatio.toFixed(2),
|
||||
direction,
|
||||
estFlow: Math.round(estFlowMagnitude),
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function buildFallbackResult() {
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
summary: {
|
||||
etfCount: 0,
|
||||
totalVolume: 0,
|
||||
totalEstFlow: 0,
|
||||
netDirection: 'UNAVAILABLE',
|
||||
inflowCount: 0,
|
||||
outflowCount: 0,
|
||||
},
|
||||
etfs: [],
|
||||
unavailable: true,
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: cors });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
|
||||
return new Response(JSON.stringify(cachedResponse), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=1800` },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const charts = await Promise.allSettled(
|
||||
ETF_LIST.map(etf => fetchChart(etf.ticker))
|
||||
);
|
||||
|
||||
const etfs = [];
|
||||
for (let i = 0; i < ETF_LIST.length; i++) {
|
||||
const chart = charts[i].status === 'fulfilled' ? charts[i].value : null;
|
||||
if (chart) {
|
||||
const parsed = parseChartData(chart, ETF_LIST[i].ticker, ETF_LIST[i].issuer);
|
||||
if (parsed) etfs.push(parsed);
|
||||
}
|
||||
}
|
||||
|
||||
const totalVolume = etfs.reduce((sum, e) => sum + e.volume, 0);
|
||||
const totalEstFlow = etfs.reduce((sum, e) => sum + e.estFlow, 0);
|
||||
const inflowCount = etfs.filter(e => e.direction === 'inflow').length;
|
||||
const outflowCount = etfs.filter(e => e.direction === 'outflow').length;
|
||||
|
||||
const result = {
|
||||
timestamp: new Date().toISOString(),
|
||||
summary: {
|
||||
etfCount: etfs.length,
|
||||
totalVolume,
|
||||
totalEstFlow,
|
||||
netDirection: totalEstFlow > 0 ? 'NET INFLOW' : totalEstFlow < 0 ? 'NET OUTFLOW' : 'NEUTRAL',
|
||||
inflowCount,
|
||||
outflowCount,
|
||||
},
|
||||
etfs: etfs.sort((a, b) => b.volume - a.volume),
|
||||
};
|
||||
|
||||
cachedResponse = result;
|
||||
cacheTimestamp = now;
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=1800` },
|
||||
});
|
||||
} catch (err) {
|
||||
const fallback = cachedResponse || buildFallbackResult();
|
||||
cachedResponse = fallback;
|
||||
cacheTimestamp = now;
|
||||
return new Response(JSON.stringify(fallback), {
|
||||
status: 200,
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const response = await fetch('https://nasstatus.faa.gov/api/airport-status-information', {
|
||||
headers: { 'Accept': 'application/xml' },
|
||||
});
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/xml',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(`<error>${error.message}</error>`, {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/xml' },
|
||||
});
|
||||
}
|
||||
}
|
||||
-115
@@ -1,115 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const SYMBOL_PATTERN = /^[A-Za-z0-9.^]+$/;
|
||||
const MAX_SYMBOLS = 20;
|
||||
const MAX_SYMBOL_LENGTH = 10;
|
||||
|
||||
function validateSymbols(symbolsParam) {
|
||||
if (!symbolsParam) return null;
|
||||
|
||||
const symbols = symbolsParam
|
||||
.split(',')
|
||||
.map(s => s.trim().toUpperCase())
|
||||
.filter(s => s.length <= MAX_SYMBOL_LENGTH && SYMBOL_PATTERN.test(s))
|
||||
.slice(0, MAX_SYMBOLS);
|
||||
|
||||
return symbols.length > 0 ? symbols : null;
|
||||
}
|
||||
|
||||
async function fetchQuote(symbol, apiKey) {
|
||||
const url = `https://finnhub.io/api/v1/quote?symbol=${encodeURIComponent(symbol)}&token=${apiKey}`;
|
||||
const response = await fetch(url, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return { symbol, error: `HTTP ${response.status}` };
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
// Finnhub returns { c, d, dp, h, l, o, pc, t } where:
|
||||
// c = current price, d = change, dp = percent change
|
||||
// h = high, l = low, o = open, pc = previous close, t = timestamp
|
||||
if (data.c === 0 && data.h === 0 && data.l === 0) {
|
||||
return { symbol, error: 'No data available' };
|
||||
}
|
||||
|
||||
return {
|
||||
symbol,
|
||||
price: data.c,
|
||||
change: data.d,
|
||||
changePercent: data.dp,
|
||||
high: data.h,
|
||||
low: data.l,
|
||||
open: data.o,
|
||||
previousClose: data.pc,
|
||||
timestamp: data.t,
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.FINNHUB_API_KEY;
|
||||
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ quotes: [], skipped: true, reason: 'FINNHUB_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const symbols = validateSymbols(url.searchParams.get('symbols'));
|
||||
|
||||
if (!symbols) {
|
||||
return new Response(JSON.stringify({ error: 'Invalid or missing symbols parameter' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Fetch all quotes in parallel (Finnhub allows 60 req/min on free tier)
|
||||
const quotes = await Promise.all(
|
||||
symbols.map(symbol => fetchQuote(symbol, apiKey))
|
||||
);
|
||||
|
||||
return new Response(JSON.stringify({ quotes }), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,145 +0,0 @@
|
||||
/**
|
||||
* NASA FIRMS Satellite Fire Detection API
|
||||
* Proxies requests to NASA FIRMS to avoid CORS and protect API key
|
||||
* Returns parsed fire data for monitored conflict regions
|
||||
*
|
||||
* GET ?region=Ukraine&days=1 — fires for one region
|
||||
* GET ?days=1 — fires for all monitored regions
|
||||
*/
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const FIRMS_API_KEY = process.env.NASA_FIRMS_API_KEY || process.env.FIRMS_API_KEY || '';
|
||||
const FIRMS_BASE = 'https://firms.modaps.eosdis.nasa.gov/api/area/csv';
|
||||
const SOURCE = 'VIIRS_SNPP_NRT';
|
||||
|
||||
// Bounding boxes as west,south,east,north
|
||||
const MONITORED_REGIONS = {
|
||||
'Ukraine': { bbox: '22,44,40,53' },
|
||||
'Russia': { bbox: '20,50,180,82' },
|
||||
'Iran': { bbox: '44,25,63,40' },
|
||||
'Israel/Gaza': { bbox: '34,29,36,34' },
|
||||
'Syria': { bbox: '35,32,42,37' },
|
||||
'Taiwan': { bbox: '119,21,123,26' },
|
||||
'North Korea': { bbox: '124,37,131,43' },
|
||||
'Saudi Arabia': { bbox: '34,16,56,32' },
|
||||
'Turkey': { bbox: '26,36,45,42' },
|
||||
};
|
||||
|
||||
// Map VIIRS confidence letters to numeric
|
||||
function parseConfidence(c) {
|
||||
if (c === 'h') return 95;
|
||||
if (c === 'n') return 50;
|
||||
if (c === 'l') return 20;
|
||||
return parseInt(c) || 0;
|
||||
}
|
||||
|
||||
function parseCSV(csv) {
|
||||
const lines = csv.trim().split('\n');
|
||||
if (lines.length < 2) return [];
|
||||
|
||||
const headers = lines[0].split(',').map(h => h.trim());
|
||||
const results = [];
|
||||
|
||||
for (let i = 1; i < lines.length; i++) {
|
||||
const vals = lines[i].split(',').map(v => v.trim());
|
||||
if (vals.length < headers.length) continue;
|
||||
|
||||
const row = {};
|
||||
headers.forEach((h, idx) => { row[h] = vals[idx]; });
|
||||
|
||||
results.push({
|
||||
lat: parseFloat(row.latitude),
|
||||
lon: parseFloat(row.longitude),
|
||||
brightness: parseFloat(row.bright_ti4) || 0,
|
||||
scan: parseFloat(row.scan) || 0,
|
||||
track: parseFloat(row.track) || 0,
|
||||
acq_date: row.acq_date || '',
|
||||
acq_time: row.acq_time || '',
|
||||
satellite: row.satellite || '',
|
||||
confidence: parseConfidence(row.confidence),
|
||||
bright_t31: parseFloat(row.bright_ti5) || 0,
|
||||
frp: parseFloat(row.frp) || 0,
|
||||
daynight: row.daynight || '',
|
||||
});
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (request.method === 'OPTIONS') return new Response(null, { status: 204, headers: cors });
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
if (!FIRMS_API_KEY) {
|
||||
return json({ regions: {}, totalCount: 0, skipped: true, reason: 'NASA_FIRMS_API_KEY not configured', source: SOURCE, days: 0, timestamp: new Date().toISOString() });
|
||||
}
|
||||
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const regionName = searchParams.get('region');
|
||||
const days = Math.min(parseInt(searchParams.get('days')) || 1, 5);
|
||||
|
||||
const regions = regionName
|
||||
? { [regionName]: MONITORED_REGIONS[regionName] }
|
||||
: MONITORED_REGIONS;
|
||||
|
||||
if (regionName && !MONITORED_REGIONS[regionName]) {
|
||||
return json({ error: `Unknown region: ${regionName}` }, 400);
|
||||
}
|
||||
|
||||
const allFires = {};
|
||||
let totalCount = 0;
|
||||
|
||||
// Fetch regions in parallel (max 10)
|
||||
const entries = Object.entries(regions);
|
||||
const results = await Promise.allSettled(
|
||||
entries.map(async ([name, { bbox }]) => {
|
||||
const url = `${FIRMS_BASE}/${FIRMS_API_KEY}/${SOURCE}/${bbox}/${days}`;
|
||||
const res = await fetch(url, {
|
||||
headers: { 'Accept': 'text/csv' },
|
||||
});
|
||||
if (!res.ok) throw new Error(`FIRMS ${res.status} for ${name}`);
|
||||
const csv = await res.text();
|
||||
return { name, fires: parseCSV(csv) };
|
||||
})
|
||||
);
|
||||
|
||||
for (const result of results) {
|
||||
if (result.status === 'fulfilled') {
|
||||
const { name, fires } = result.value;
|
||||
allFires[name] = fires;
|
||||
totalCount += fires.length;
|
||||
} else {
|
||||
console.error('[FIRMS]', result.reason?.message);
|
||||
}
|
||||
}
|
||||
|
||||
return json({
|
||||
regions: allFires,
|
||||
totalCount,
|
||||
source: SOURCE,
|
||||
days,
|
||||
timestamp: new Date().toISOString(),
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[FIRMS] Error:', err);
|
||||
return json({ error: 'Failed to fetch fire data' }, 500);
|
||||
}
|
||||
}
|
||||
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', // 10 min cache
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,89 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const seriesId = url.searchParams.get('series_id');
|
||||
const observationStart = url.searchParams.get('observation_start');
|
||||
const observationEnd = url.searchParams.get('observation_end');
|
||||
|
||||
if (!seriesId) {
|
||||
return new Response(JSON.stringify({ error: 'Missing series_id parameter' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.FRED_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({
|
||||
observations: [],
|
||||
skipped: true,
|
||||
reason: 'FRED_API_KEY not configured',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const params = new URLSearchParams({
|
||||
series_id: seriesId,
|
||||
api_key: apiKey,
|
||||
file_type: 'json',
|
||||
sort_order: 'desc',
|
||||
limit: '10',
|
||||
});
|
||||
|
||||
if (observationStart) params.set('observation_start', observationStart);
|
||||
if (observationEnd) params.set('observation_end', observationEnd);
|
||||
|
||||
const fredUrl = `https://api.stlouisfed.org/fred/series/observations?${params}`;
|
||||
const response = await fetch(fredUrl, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
return new Response(JSON.stringify(data), {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: error.message }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const MAX_RECORDS = 20;
|
||||
const DEFAULT_RECORDS = 10;
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const query = url.searchParams.get('query');
|
||||
const maxrecords = Math.min(
|
||||
parseInt(url.searchParams.get('maxrecords') || DEFAULT_RECORDS, 10),
|
||||
MAX_RECORDS
|
||||
);
|
||||
const timespan = url.searchParams.get('timespan') || '72h';
|
||||
|
||||
if (!query || query.length < 2) {
|
||||
return new Response(JSON.stringify({ error: 'Query parameter required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const gdeltUrl = new URL('https://api.gdeltproject.org/api/v2/doc/doc');
|
||||
gdeltUrl.searchParams.set('query', query);
|
||||
gdeltUrl.searchParams.set('mode', 'artlist');
|
||||
gdeltUrl.searchParams.set('maxrecords', maxrecords.toString());
|
||||
gdeltUrl.searchParams.set('format', 'json');
|
||||
gdeltUrl.searchParams.set('sort', 'date');
|
||||
gdeltUrl.searchParams.set('timespan', timespan);
|
||||
|
||||
const response = await fetch(gdeltUrl.toString());
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`GDELT returned ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
const articles = (data.articles || []).map(article => ({
|
||||
title: article.title,
|
||||
url: article.url,
|
||||
source: article.domain || article.source?.domain,
|
||||
date: article.seendate,
|
||||
image: article.socialimage,
|
||||
language: article.language,
|
||||
tone: article.tone,
|
||||
}));
|
||||
|
||||
return new Response(JSON.stringify({ articles, query }), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: error.message, articles: [] }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,86 +0,0 @@
|
||||
// GDELT Geo API proxy with security hardening
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const ALLOWED_FORMATS = ['geojson', 'json', 'csv'];
|
||||
const MAX_RECORDS = 500;
|
||||
const MIN_RECORDS = 1;
|
||||
const ALLOWED_TIMESPANS = ['1d', '7d', '14d', '30d', '60d', '90d'];
|
||||
|
||||
function validateMaxRecords(val) {
|
||||
const num = parseInt(val, 10);
|
||||
if (isNaN(num)) return 250;
|
||||
return Math.max(MIN_RECORDS, Math.min(MAX_RECORDS, num));
|
||||
}
|
||||
|
||||
function validateFormat(val) {
|
||||
return ALLOWED_FORMATS.includes(val) ? val : 'geojson';
|
||||
}
|
||||
|
||||
function validateTimespan(val) {
|
||||
return ALLOWED_TIMESPANS.includes(val) ? val : '7d';
|
||||
}
|
||||
|
||||
function sanitizeQuery(val) {
|
||||
if (!val || typeof val !== 'string') return 'protest';
|
||||
return val.slice(0, 200).replace(/[<>\"']/g, '');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const query = sanitizeQuery(url.searchParams.get('query'));
|
||||
const format = validateFormat(url.searchParams.get('format') || 'geojson');
|
||||
const maxrecords = validateMaxRecords(url.searchParams.get('maxrecords') || '250');
|
||||
const timespan = validateTimespan(url.searchParams.get('timespan') || '7d');
|
||||
|
||||
try {
|
||||
const response = await fetch(
|
||||
`https://api.gdeltproject.org/api/v2/geo/geo?query=${encodeURIComponent(query)}&format=${format}&maxrecords=${maxrecords}×pan=${timespan}`
|
||||
);
|
||||
|
||||
if (!response.ok) {
|
||||
return new Response(JSON.stringify({ error: 'Upstream service unavailable' }), {
|
||||
status: 502,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': format === 'csv' ? 'text/csv' : 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[GDELT] Fetch error:', error.message);
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch GDELT data' }), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,90 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
// Fetch trending GitHub repositories
|
||||
// Uses unofficial GitHub trending scraper API
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const language = searchParams.get('language') || 'python'; // python, javascript, typescript, etc.
|
||||
const since = searchParams.get('since') || 'daily'; // daily, weekly, monthly
|
||||
const spoken_language = searchParams.get('spoken_language') || ''; // en, zh, etc.
|
||||
|
||||
// Using GitHub trending API (unofficial)
|
||||
// Alternative: https://gh-trending-api.herokuapp.com/repositories
|
||||
const baseUrl = 'https://api.gitterapp.com/repositories';
|
||||
const queryParams = new URLSearchParams({
|
||||
language: language,
|
||||
since: since,
|
||||
});
|
||||
|
||||
if (spoken_language) {
|
||||
queryParams.append('spoken_language_code', spoken_language);
|
||||
}
|
||||
|
||||
const apiUrl = `${baseUrl}?${queryParams.toString()}`;
|
||||
|
||||
const response = await fetch(apiUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'WorldMonitor/1.0 (Tech Tracker)',
|
||||
},
|
||||
signal: AbortSignal.timeout(10000), // 10 second timeout
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
// Fallback: try alternative API
|
||||
const fallbackUrl = `https://gh-trending-api.herokuapp.com/repositories/${language}?since=${since}`;
|
||||
const fallbackResponse = await fetch(fallbackUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
signal: AbortSignal.timeout(10000),
|
||||
});
|
||||
|
||||
if (!fallbackResponse.ok) {
|
||||
throw new Error(`GitHub trending API returned ${fallbackResponse.status}`);
|
||||
}
|
||||
|
||||
const data = await fallbackResponse.json();
|
||||
return new Response(JSON.stringify(data), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300', // 30 min cache
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
return new Response(JSON.stringify(data), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300', // 30 min cache
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: 'Failed to fetch GitHub trending data',
|
||||
message: error.message
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1,296 +0,0 @@
|
||||
/**
|
||||
* Groq API Summarization Endpoint with Redis Caching
|
||||
* Uses Llama 3.1 8B Instant for high-throughput summarization
|
||||
* Free tier: 14,400 requests/day (14x more than 70B model)
|
||||
* Server-side Redis cache for cross-user deduplication
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
|
||||
const MODEL = 'llama-3.1-8b-instant'; // 14.4K RPD vs 1K for 70b
|
||||
const CACHE_TTL_SECONDS = 86400; // 24 hours
|
||||
|
||||
const CACHE_VERSION = 'v3';
|
||||
|
||||
function getCacheKey(headlines, mode, geoContext = '', variant = 'full', lang = 'en') {
|
||||
const sorted = headlines.slice(0, 8).sort().join('|');
|
||||
const geoHash = geoContext ? ':g' + hashString(geoContext).slice(0, 6) : '';
|
||||
const hash = hashString(`${mode}:${sorted}`);
|
||||
const normalizedVariant = typeof variant === 'string' && variant ? variant.toLowerCase() : 'full';
|
||||
const normalizedLang = typeof lang === 'string' && lang ? lang.toLowerCase() : 'en';
|
||||
|
||||
if (mode === 'translate') {
|
||||
const targetLang = normalizedVariant || normalizedLang;
|
||||
return `summary:${CACHE_VERSION}:${mode}:${targetLang}:${hash}${geoHash}`;
|
||||
}
|
||||
|
||||
return `summary:${CACHE_VERSION}:${mode}:${normalizedVariant}:${normalizedLang}:${hash}${geoHash}`;
|
||||
}
|
||||
|
||||
// Deduplicate similar headlines (same story from different sources)
|
||||
function deduplicateHeadlines(headlines) {
|
||||
const seen = new Set();
|
||||
const unique = [];
|
||||
|
||||
for (const headline of headlines) {
|
||||
// Normalize: lowercase, remove punctuation, collapse whitespace
|
||||
const normalized = headline.toLowerCase()
|
||||
.replace(/[^\w\s]/g, '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
|
||||
// Extract key words (4+ chars) for similarity check
|
||||
const words = new Set(normalized.split(' ').filter(w => w.length >= 4));
|
||||
|
||||
// Check if this headline is too similar to any we've seen
|
||||
let isDuplicate = false;
|
||||
for (const seenWords of seen) {
|
||||
const intersection = [...words].filter(w => seenWords.has(w));
|
||||
const similarity = intersection.length / Math.min(words.size, seenWords.size);
|
||||
if (similarity > 0.6) {
|
||||
isDuplicate = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!isDuplicate) {
|
||||
seen.add(words);
|
||||
unique.push(headline);
|
||||
}
|
||||
}
|
||||
|
||||
return unique;
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.GROQ_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ summary: null, fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return new Response(JSON.stringify({ error: 'Payload too large' }), {
|
||||
status: 413,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const { headlines, mode = 'brief', geoContext = '', variant = 'full', lang = 'en' } = await request.json();
|
||||
|
||||
if (!headlines || !Array.isArray(headlines) || headlines.length === 0) {
|
||||
return new Response(JSON.stringify({ error: 'Headlines array required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Check cache first
|
||||
const cacheKey = getCacheKey(headlines, mode, geoContext, variant, lang);
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.summary) {
|
||||
console.log('[Groq] Cache hit:', cacheKey);
|
||||
return new Response(JSON.stringify({
|
||||
summary: cached.summary,
|
||||
model: cached.model || MODEL,
|
||||
provider: 'cache',
|
||||
cached: true,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Deduplicate similar headlines (same story from multiple sources)
|
||||
const uniqueHeadlines = deduplicateHeadlines(headlines.slice(0, 8));
|
||||
const headlineText = uniqueHeadlines.map((h, i) => `${i + 1}. ${h}`).join('\n');
|
||||
|
||||
let systemPrompt, userPrompt;
|
||||
|
||||
// Include intelligence synthesis context in prompt if available
|
||||
const intelSection = geoContext ? `\n\n${geoContext}` : '';
|
||||
|
||||
// Current date context for LLM (models may have outdated knowledge)
|
||||
const isTechVariant = variant === 'tech';
|
||||
const dateContext = `Current date: ${new Date().toISOString().split('T')[0]}.${isTechVariant ? '' : ' Donald Trump is the current US President (second term, inaugurated Jan 2025).'}`;
|
||||
|
||||
// Language instruction
|
||||
const langInstruction = lang && lang !== 'en' ? `\nIMPORTANT: Output the summary in ${lang.toUpperCase()} language.` : '';
|
||||
|
||||
if (mode === 'brief') {
|
||||
if (isTechVariant) {
|
||||
// Tech variant: focus on startups, AI, funding, product launches
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Summarize the key tech/startup development in 2-3 sentences.
|
||||
Rules:
|
||||
- Focus ONLY on technology, startups, AI, funding, product launches, or developer news
|
||||
- IGNORE political news, trade policy, tariffs, government actions unless directly about tech regulation
|
||||
- Lead with the company/product/technology name
|
||||
- Start directly: "OpenAI announced...", "A new $50M Series B...", "GitHub released..."
|
||||
- No bullet points, no meta-commentary${langInstruction}`;
|
||||
} else {
|
||||
// Full variant: geopolitical focus
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Summarize the key development in 2-3 sentences.
|
||||
Rules:
|
||||
- Lead with WHAT happened and WHERE - be specific
|
||||
- NEVER start with "Breaking news", "Good evening", "Tonight", or TV-style openings
|
||||
- Start directly with the subject: "Iran's regime...", "The US Treasury...", "Protests in..."
|
||||
- CRITICAL FOCAL POINTS are the main actors - mention them by name
|
||||
- If focal points show news + signals convergence, that's the lead
|
||||
- No bullet points, no meta-commentary${langInstruction}`;
|
||||
}
|
||||
userPrompt = `Summarize the top story:\n${headlineText}${intelSection}`;
|
||||
} else if (mode === 'analysis') {
|
||||
if (isTechVariant) {
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Analyze the tech/startup trend in 2-3 sentences.
|
||||
Rules:
|
||||
- Focus ONLY on technology implications: funding trends, AI developments, market shifts, product strategy
|
||||
- IGNORE political implications, trade wars, government unless directly about tech policy
|
||||
- Lead with the insight for tech industry
|
||||
- Connect to startup ecosystem, VC trends, or technical implications`;
|
||||
} else {
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Provide analysis in 2-3 sentences. Be direct and specific.
|
||||
Rules:
|
||||
- Lead with the insight - what's significant and why
|
||||
- NEVER start with "Breaking news", "Tonight", "The key/dominant narrative is"
|
||||
- Start with substance: "Iran faces...", "The escalation in...", "Multiple signals suggest..."
|
||||
- CRITICAL FOCAL POINTS are your main actors - explain WHY they matter
|
||||
- If focal points show news-signal correlation, flag as escalation
|
||||
- Connect dots, be specific about implications`;
|
||||
}
|
||||
userPrompt = isTechVariant
|
||||
? `What's the key tech trend or development?\n${headlineText}${intelSection}`
|
||||
: `What's the key pattern or risk?\n${headlineText}${intelSection}`;
|
||||
} else if (mode === 'translate') {
|
||||
const targetLang = variant; // In translate mode, variant param holds the target language code (e.g., 'fr', 'es')
|
||||
systemPrompt = `You are a professional news translator. Translate the following news headlines/summaries into ${targetLang}.
|
||||
Rules:
|
||||
- Maintain the original tone and journalistic style.
|
||||
- Do NOT add any conversational filler (e.g., "Here is the translation").
|
||||
- Output ONLY the translated text.
|
||||
- If the text is already in ${targetLang}, return it as is.`;
|
||||
userPrompt = `Translate to ${targetLang}:\n${headlines[0]}`;
|
||||
} else {
|
||||
systemPrompt = isTechVariant
|
||||
? `${dateContext}\n\nSynthesize tech news in 2 sentences. Focus on startups, AI, funding, products. Ignore politics unless directly about tech regulation.${langInstruction}`
|
||||
: `${dateContext}\n\nSynthesize in 2 sentences max. Lead with substance. NEVER start with "Breaking news" or "Tonight" - just state the insight directly. CRITICAL focal points with news-signal convergence are significant.${langInstruction}`;
|
||||
userPrompt = `Key takeaway:\n${headlineText}${intelSection}`;
|
||||
}
|
||||
|
||||
const response = await fetch(GROQ_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
temperature: 0.3,
|
||||
max_tokens: 150,
|
||||
top_p: 0.9,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.error('[Groq] API error:', response.status, errorText);
|
||||
|
||||
// Return fallback signal for rate limiting
|
||||
if (response.status === 429) {
|
||||
return new Response(JSON.stringify({ error: 'Rate limited', fallback: true }), {
|
||||
status: 429,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({ error: 'Groq API error', fallback: true }), {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const summary = data.choices?.[0]?.message?.content?.trim();
|
||||
|
||||
if (!summary) {
|
||||
return new Response(JSON.stringify({ error: 'Empty response', fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Store in cache
|
||||
await setCachedJson(cacheKey, {
|
||||
summary,
|
||||
model: MODEL,
|
||||
timestamp: Date.now(),
|
||||
}, CACHE_TTL_SECONDS);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
summary,
|
||||
model: MODEL,
|
||||
provider: 'groq',
|
||||
cached: false,
|
||||
tokens: data.usage?.total_tokens || 0,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
|
||||
},
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('[Groq] Error:', error.name, error.message, error.stack?.split('\n')[1]);
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
errorType: error.name,
|
||||
fallback: true
|
||||
}), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
// Fetch Hacker News front page stories
|
||||
// Uses official HackerNews Firebase API
|
||||
const ALLOWED_STORY_TYPES = new Set(['top', 'new', 'best', 'ask', 'show', 'job']);
|
||||
const DEFAULT_LIMIT = 30;
|
||||
const MAX_LIMIT = 60;
|
||||
const MAX_CONCURRENCY = 10;
|
||||
|
||||
function parseLimit(rawLimit) {
|
||||
const parsed = Number.parseInt(rawLimit || '', 10);
|
||||
if (!Number.isFinite(parsed)) return DEFAULT_LIMIT;
|
||||
return Math.max(1, Math.min(MAX_LIMIT, parsed));
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const requestedType = searchParams.get('type') || 'top';
|
||||
const storyType = ALLOWED_STORY_TYPES.has(requestedType) ? requestedType : 'top';
|
||||
const limit = parseLimit(searchParams.get('limit'));
|
||||
|
||||
// HackerNews official Firebase API
|
||||
const storiesUrl = `https://hacker-news.firebaseio.com/v0/${storyType}stories.json`;
|
||||
|
||||
// Fetch story IDs
|
||||
const storiesResponse = await fetch(storiesUrl, {
|
||||
signal: AbortSignal.timeout(10000),
|
||||
});
|
||||
|
||||
if (!storiesResponse.ok) {
|
||||
throw new Error(`HackerNews API returned ${storiesResponse.status}`);
|
||||
}
|
||||
|
||||
const storyIds = await storiesResponse.json();
|
||||
if (!Array.isArray(storyIds)) {
|
||||
throw new Error('HackerNews API returned unexpected payload');
|
||||
}
|
||||
const limitedIds = storyIds.slice(0, limit);
|
||||
|
||||
// Fetch story details in bounded batches to avoid unbounded fan-out.
|
||||
const stories = [];
|
||||
for (let i = 0; i < limitedIds.length; i += MAX_CONCURRENCY) {
|
||||
const batchIds = limitedIds.slice(i, i + MAX_CONCURRENCY);
|
||||
const storyPromises = batchIds.map(async (id) => {
|
||||
const storyUrl = `https://hacker-news.firebaseio.com/v0/item/${id}.json`;
|
||||
try {
|
||||
const response = await fetch(storyUrl, {
|
||||
signal: AbortSignal.timeout(5000),
|
||||
});
|
||||
if (response.ok) {
|
||||
return await response.json();
|
||||
}
|
||||
return null;
|
||||
} catch (error) {
|
||||
console.error(`Failed to fetch story ${id}:`, error);
|
||||
return null;
|
||||
}
|
||||
});
|
||||
const batchResults = await Promise.all(storyPromises);
|
||||
stories.push(...batchResults.filter((story) => story !== null));
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
type: storyType,
|
||||
stories: stories,
|
||||
total: stories.length,
|
||||
timestamp: new Date().toISOString()
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', // 5 min cache
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: 'Failed to fetch Hacker News data',
|
||||
message: error.message
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
-148
@@ -1,148 +0,0 @@
|
||||
// HDX HAPI (Humanitarian API) proxy
|
||||
// Returns aggregated conflict event counts per country
|
||||
// Source: ACLED data aggregated monthly by HDX
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_KEY = 'hapi:conflict-events:v2';
|
||||
const CACHE_TTL_SECONDS = 6 * 60 * 60; // 6 hours
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const RESPONSE_CACHE_CONTROL = 'public, max-age=1800';
|
||||
|
||||
// In-memory fallback when Redis is unavailable.
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(
|
||||
data &&
|
||||
typeof data === 'object' &&
|
||||
Array.isArray(data.countries)
|
||||
);
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/hapi', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/hapi', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const appId = btoa('worldmonitor:monitor@worldmonitor.app');
|
||||
const response = await fetch(
|
||||
`https://hapi.humdata.org/api/v2/coordination-context/conflict-events?output_format=json&limit=1000&offset=0&app_identifier=${appId}`,
|
||||
{
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`HAPI API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const rawData = await response.json();
|
||||
const records = rawData.data || [];
|
||||
|
||||
// Each record is (country, event_type, month) — aggregate across event types per country
|
||||
// Keep only the most recent month per country
|
||||
const byCountry = {};
|
||||
for (const r of records) {
|
||||
const iso3 = r.location_code || '';
|
||||
if (!iso3) continue;
|
||||
|
||||
const month = r.reference_period_start || '';
|
||||
const eventType = (r.event_type || '').toLowerCase();
|
||||
const events = r.events || 0;
|
||||
const fatalities = r.fatalities || 0;
|
||||
|
||||
if (!byCountry[iso3]) {
|
||||
byCountry[iso3] = { iso3, locationName: r.location_name || '', month, eventsTotal: 0, eventsPoliticalViolence: 0, eventsCivilianTargeting: 0, eventsDemonstrations: 0, fatalitiesTotalPoliticalViolence: 0, fatalitiesTotalCivilianTargeting: 0 };
|
||||
}
|
||||
|
||||
const c = byCountry[iso3];
|
||||
if (month > c.month) {
|
||||
// Newer month — reset
|
||||
c.month = month;
|
||||
c.eventsTotal = 0; c.eventsPoliticalViolence = 0; c.eventsCivilianTargeting = 0; c.eventsDemonstrations = 0; c.fatalitiesTotalPoliticalViolence = 0; c.fatalitiesTotalCivilianTargeting = 0;
|
||||
}
|
||||
if (month === c.month) {
|
||||
c.eventsTotal += events;
|
||||
if (eventType.includes('political_violence')) { c.eventsPoliticalViolence += events; c.fatalitiesTotalPoliticalViolence += fatalities; }
|
||||
if (eventType.includes('civilian_targeting')) { c.eventsCivilianTargeting += events; c.fatalitiesTotalCivilianTargeting += fatalities; }
|
||||
if (eventType.includes('demonstration')) { c.eventsDemonstrations += events; }
|
||||
}
|
||||
}
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: Object.keys(byCountry).length,
|
||||
countries: Object.values(byCountry),
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/hapi', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'MISS',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/hapi', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'STALE',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/hapi', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, countries: [] }, {
|
||||
status: 500,
|
||||
headers: { ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
import { strict as assert } from 'node:assert';
|
||||
import test from 'node:test';
|
||||
import { XMLLoader } from '@loaders.gl/xml';
|
||||
import { WMSCapabilitiesLoader, WMSErrorLoader, _WMSFeatureInfoLoader } from '@loaders.gl/wms';
|
||||
|
||||
const WMS_CAPABILITIES_XML = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<WMS_Capabilities version="1.3.0">
|
||||
<Service>
|
||||
<Name>WMS</Name>
|
||||
<Title>Test Service</Title>
|
||||
<KeywordList>
|
||||
<Keyword>alerts</Keyword>
|
||||
<Keyword>world</Keyword>
|
||||
</KeywordList>
|
||||
</Service>
|
||||
<Capability>
|
||||
<Request>
|
||||
<GetMap>
|
||||
<Format>image/png</Format>
|
||||
<Format>image/jpeg</Format>
|
||||
</GetMap>
|
||||
</Request>
|
||||
<Exception>
|
||||
<Format>application/vnd.ogc.se_xml</Format>
|
||||
</Exception>
|
||||
<Layer>
|
||||
<Title>Root Layer</Title>
|
||||
<CRS>EPSG:4326</CRS>
|
||||
<EX_GeographicBoundingBox>
|
||||
<westBoundLongitude>-180</westBoundLongitude>
|
||||
<eastBoundLongitude>180</eastBoundLongitude>
|
||||
<southBoundLatitude>-90</southBoundLatitude>
|
||||
<northBoundLatitude>90</northBoundLatitude>
|
||||
</EX_GeographicBoundingBox>
|
||||
<Layer queryable="1">
|
||||
<Name>alerts</Name>
|
||||
<Title>Alerts</Title>
|
||||
<BoundingBox CRS="EPSG:4326" minx="-10" miny="-20" maxx="30" maxy="40" />
|
||||
<Dimension name="time" units="ISO8601" default="2024-01-01" nearestValue="1">
|
||||
2024-01-01/2024-12-31/P1D
|
||||
</Dimension>
|
||||
</Layer>
|
||||
</Layer>
|
||||
</Capability>
|
||||
</WMS_Capabilities>`;
|
||||
|
||||
test('XMLLoader keeps namespace stripping + array paths stable', () => {
|
||||
const xml = '<root><ns:Child attr="x">ok</ns:Child><ns:Child attr="y">yo</ns:Child></root>';
|
||||
const parsed = XMLLoader.parseTextSync(xml, {
|
||||
xml: {
|
||||
removeNSPrefix: true,
|
||||
arrayPaths: ['root.Child'],
|
||||
},
|
||||
});
|
||||
|
||||
assert.deepEqual(parsed, {
|
||||
root: {
|
||||
Child: [
|
||||
{ value: 'ok', attr: 'x' },
|
||||
{ value: 'yo', attr: 'y' },
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test('WMSCapabilitiesLoader parses core typed fields from XML capabilities', () => {
|
||||
const parsed = WMSCapabilitiesLoader.parseTextSync(WMS_CAPABILITIES_XML);
|
||||
|
||||
assert.equal(parsed.version, '1.3.0');
|
||||
assert.equal(parsed.name, 'WMS');
|
||||
assert.deepEqual(parsed.requests.GetMap.mimeTypes, ['image/png', 'image/jpeg']);
|
||||
|
||||
assert.equal(parsed.layers.length, 1);
|
||||
const rootLayer = parsed.layers[0];
|
||||
assert.deepEqual(rootLayer.geographicBoundingBox, [[-180, -90], [180, 90]]);
|
||||
|
||||
const alertsLayer = rootLayer.layers[0];
|
||||
assert.equal(alertsLayer.name, 'alerts');
|
||||
assert.equal(alertsLayer.queryable, true);
|
||||
assert.deepEqual(alertsLayer.boundingBoxes[0], {
|
||||
crs: 'EPSG:4326',
|
||||
boundingBox: [[-10, -20], [30, 40]],
|
||||
});
|
||||
assert.deepEqual(alertsLayer.dimensions[0], {
|
||||
name: 'time',
|
||||
units: 'ISO8601',
|
||||
extent: '2024-01-01/2024-12-31/P1D',
|
||||
defaultValue: '2024-01-01',
|
||||
nearestValue: true,
|
||||
});
|
||||
});
|
||||
|
||||
test('WMSErrorLoader extracts namespaced error text and honors throw options', () => {
|
||||
const namespacedErrorXml =
|
||||
'<?xml version="1.0"?><ogc:ServiceExceptionReport><ogc:ServiceException code="LayerNotDefined">Bad layer</ogc:ServiceException></ogc:ServiceExceptionReport>';
|
||||
|
||||
const defaultMessage = WMSErrorLoader.parseTextSync(namespacedErrorXml);
|
||||
assert.equal(defaultMessage, 'WMS Service error: Bad layer');
|
||||
|
||||
const minimalMessage = WMSErrorLoader.parseTextSync(namespacedErrorXml, {
|
||||
wms: { minimalErrors: true },
|
||||
});
|
||||
assert.equal(minimalMessage, 'Bad layer');
|
||||
|
||||
assert.throws(
|
||||
() => WMSErrorLoader.parseTextSync(namespacedErrorXml, { wms: { throwOnError: true } }),
|
||||
/WMS Service error: Bad layer/
|
||||
);
|
||||
});
|
||||
|
||||
test('WMS feature info parsing remains stable for single and repeated FIELDS nodes', () => {
|
||||
const singleFieldsXml = '<?xml version="1.0"?><FeatureInfoResponse><FIELDS id="1" label="one"/></FeatureInfoResponse>';
|
||||
const manyFieldsXml = '<?xml version="1.0"?><FeatureInfoResponse><FIELDS id="1"/><FIELDS id="2"/></FeatureInfoResponse>';
|
||||
|
||||
const single = _WMSFeatureInfoLoader.parseTextSync(singleFieldsXml);
|
||||
const many = _WMSFeatureInfoLoader.parseTextSync(manyFieldsXml);
|
||||
|
||||
assert.equal(single.features.length, 1);
|
||||
assert.deepEqual(single.features[0]?.attributes, { id: '1', label: 'one' });
|
||||
assert.equal(many.features.length, 2);
|
||||
assert.equal(many.features[0]?.attributes?.id, '1');
|
||||
assert.equal(many.features[1]?.attributes?.id, '2');
|
||||
});
|
||||
@@ -1,284 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_TTL = 300;
|
||||
let cachedResponse = null;
|
||||
let cacheTimestamp = 0;
|
||||
|
||||
async function fetchJSON(url, timeout = 8000) {
|
||||
const controller = new AbortController();
|
||||
const id = setTimeout(() => controller.abort(), timeout);
|
||||
try {
|
||||
const res = await fetch(url, { signal: controller.signal });
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
return await res.json();
|
||||
} finally {
|
||||
clearTimeout(id);
|
||||
}
|
||||
}
|
||||
|
||||
function rateOfChange(prices, days) {
|
||||
if (!prices || prices.length < days + 1) return null;
|
||||
const recent = prices[prices.length - 1];
|
||||
const past = prices[prices.length - 1 - days];
|
||||
if (!past || past === 0) return null;
|
||||
return ((recent - past) / past) * 100;
|
||||
}
|
||||
|
||||
function sma(prices, period) {
|
||||
if (!prices || prices.length < period) return null;
|
||||
const slice = prices.slice(-period);
|
||||
return slice.reduce((a, b) => a + b, 0) / period;
|
||||
}
|
||||
|
||||
function extractClosePrices(chart) {
|
||||
try {
|
||||
const result = chart?.chart?.result?.[0];
|
||||
return result?.indicators?.quote?.[0]?.close?.filter(p => p != null) || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function extractVolumes(chart) {
|
||||
try {
|
||||
const result = chart?.chart?.result?.[0];
|
||||
return result?.indicators?.quote?.[0]?.volume?.filter(v => v != null) || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function extractAlignedPriceVolume(chart) {
|
||||
try {
|
||||
const result = chart?.chart?.result?.[0];
|
||||
const closes = result?.indicators?.quote?.[0]?.close || [];
|
||||
const volumes = result?.indicators?.quote?.[0]?.volume || [];
|
||||
const pairs = [];
|
||||
for (let i = 0; i < closes.length; i++) {
|
||||
if (closes[i] != null && volumes[i] != null) {
|
||||
pairs.push({ price: closes[i], volume: volumes[i] });
|
||||
}
|
||||
}
|
||||
return pairs;
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function buildFallbackResult() {
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
verdict: 'UNKNOWN',
|
||||
bullishCount: 0,
|
||||
totalCount: 0,
|
||||
signals: {
|
||||
liquidity: { status: 'UNKNOWN', value: null, sparkline: [] },
|
||||
flowStructure: { status: 'UNKNOWN', btcReturn5: null, qqqReturn5: null },
|
||||
macroRegime: { status: 'UNKNOWN', qqqRoc20: null, xlpRoc20: null },
|
||||
technicalTrend: {
|
||||
status: 'UNKNOWN',
|
||||
btcPrice: null,
|
||||
sma50: null,
|
||||
sma200: null,
|
||||
vwap30d: null,
|
||||
mayerMultiple: null,
|
||||
sparkline: [],
|
||||
},
|
||||
hashRate: { status: 'UNKNOWN', change30d: null },
|
||||
miningCost: { status: 'UNKNOWN' },
|
||||
fearGreed: { status: 'UNKNOWN', value: null, history: [] },
|
||||
},
|
||||
meta: { qqqSparkline: [] },
|
||||
unavailable: true,
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: cors });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
|
||||
return new Response(JSON.stringify(cachedResponse), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=600` },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const yahooBase = 'https://query1.finance.yahoo.com/v8/finance/chart';
|
||||
const [jpyChart, btcChart, qqqChart, xlpChart, fearGreed, mempoolHash] = await Promise.allSettled([
|
||||
fetchJSON(`${yahooBase}/JPY=X?range=1y&interval=1d`),
|
||||
fetchJSON(`${yahooBase}/BTC-USD?range=1y&interval=1d`),
|
||||
fetchJSON(`${yahooBase}/QQQ?range=1y&interval=1d`),
|
||||
fetchJSON(`${yahooBase}/XLP?range=1y&interval=1d`),
|
||||
fetchJSON('https://api.alternative.me/fng/?limit=30&format=json'),
|
||||
fetchJSON('https://mempool.space/api/v1/mining/hashrate/1m'),
|
||||
]);
|
||||
|
||||
const jpyPrices = jpyChart.status === 'fulfilled' ? extractClosePrices(jpyChart.value) : [];
|
||||
const btcPrices = btcChart.status === 'fulfilled' ? extractClosePrices(btcChart.value) : [];
|
||||
const btcVolumes = btcChart.status === 'fulfilled' ? extractVolumes(btcChart.value) : [];
|
||||
const btcAligned = btcChart.status === 'fulfilled' ? extractAlignedPriceVolume(btcChart.value) : [];
|
||||
const qqqPrices = qqqChart.status === 'fulfilled' ? extractClosePrices(qqqChart.value) : [];
|
||||
const xlpPrices = xlpChart.status === 'fulfilled' ? extractClosePrices(xlpChart.value) : [];
|
||||
|
||||
// 1. Liquidity Signal (JPY 30d ROC)
|
||||
const jpyRoc30 = rateOfChange(jpyPrices, 30);
|
||||
const liquidityStatus = jpyRoc30 !== null
|
||||
? (jpyRoc30 < -2 ? 'SQUEEZE' : 'NORMAL')
|
||||
: 'UNKNOWN';
|
||||
|
||||
// 2. Flow Structure (BTC vs QQQ 5d return)
|
||||
const btcReturn5 = rateOfChange(btcPrices, 5);
|
||||
const qqqReturn5 = rateOfChange(qqqPrices, 5);
|
||||
let flowStatus = 'UNKNOWN';
|
||||
if (btcReturn5 !== null && qqqReturn5 !== null) {
|
||||
const gap = btcReturn5 - qqqReturn5;
|
||||
flowStatus = Math.abs(gap) > 5 ? 'PASSIVE GAP' : 'ALIGNED';
|
||||
}
|
||||
|
||||
// 3. Macro Regime (QQQ/XLP 20d ROC)
|
||||
const qqqRoc20 = rateOfChange(qqqPrices, 20);
|
||||
const xlpRoc20 = rateOfChange(xlpPrices, 20);
|
||||
let regimeStatus = 'UNKNOWN';
|
||||
if (qqqRoc20 !== null && xlpRoc20 !== null) {
|
||||
regimeStatus = qqqRoc20 > xlpRoc20 ? 'RISK-ON' : 'DEFENSIVE';
|
||||
}
|
||||
|
||||
// 4. Technical Trend (BTC vs SMA50 + VWAP)
|
||||
const btcSma50 = sma(btcPrices, 50);
|
||||
const btcSma200 = sma(btcPrices, 200);
|
||||
const btcCurrent = btcPrices.length > 0 ? btcPrices[btcPrices.length - 1] : null;
|
||||
|
||||
// Compute VWAP from aligned price/volume pairs (30d)
|
||||
let btcVwap = null;
|
||||
if (btcAligned.length >= 30) {
|
||||
const last30 = btcAligned.slice(-30);
|
||||
let sumPV = 0, sumV = 0;
|
||||
for (const { price, volume } of last30) {
|
||||
sumPV += price * volume;
|
||||
sumV += volume;
|
||||
}
|
||||
if (sumV > 0) btcVwap = +(sumPV / sumV).toFixed(0);
|
||||
}
|
||||
|
||||
let trendStatus = 'UNKNOWN';
|
||||
let mayerMultiple = null;
|
||||
if (btcCurrent && btcSma50) {
|
||||
const aboveSma = btcCurrent > btcSma50 * 1.02;
|
||||
const belowSma = btcCurrent < btcSma50 * 0.98;
|
||||
const aboveVwap = btcVwap ? btcCurrent > btcVwap : null;
|
||||
if (aboveSma && aboveVwap !== false) trendStatus = 'BULLISH';
|
||||
else if (belowSma && aboveVwap !== true) trendStatus = 'BEARISH';
|
||||
else trendStatus = 'NEUTRAL';
|
||||
}
|
||||
if (btcCurrent && btcSma200) {
|
||||
mayerMultiple = +(btcCurrent / btcSma200).toFixed(2);
|
||||
}
|
||||
|
||||
// 5. Hash Rate
|
||||
let hashStatus = 'UNKNOWN';
|
||||
let hashChange = null;
|
||||
if (mempoolHash.status === 'fulfilled') {
|
||||
const hr = mempoolHash.value?.hashrates || mempoolHash.value;
|
||||
if (Array.isArray(hr) && hr.length >= 2) {
|
||||
const recent = hr[hr.length - 1]?.avgHashrate || hr[hr.length - 1];
|
||||
const older = hr[0]?.avgHashrate || hr[0];
|
||||
if (recent && older && older > 0) {
|
||||
hashChange = +((recent - older) / older * 100).toFixed(1);
|
||||
hashStatus = hashChange > 3 ? 'GROWING' : hashChange < -3 ? 'DECLINING' : 'STABLE';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 6. Mining Cost (hashrate-based model)
|
||||
let miningStatus = 'UNKNOWN';
|
||||
if (btcCurrent && hashChange !== null) {
|
||||
miningStatus = btcCurrent > 60000 ? 'PROFITABLE' : btcCurrent > 40000 ? 'TIGHT' : 'SQUEEZE';
|
||||
}
|
||||
|
||||
// 7. Fear & Greed
|
||||
let fgValue = null;
|
||||
let fgLabel = 'UNKNOWN';
|
||||
let fgHistory = [];
|
||||
if (fearGreed.status === 'fulfilled' && fearGreed.value?.data) {
|
||||
const data = fearGreed.value.data;
|
||||
const parsed = parseInt(data[0]?.value, 10);
|
||||
fgValue = Number.isFinite(parsed) ? parsed : null;
|
||||
fgLabel = data[0]?.value_classification || 'UNKNOWN';
|
||||
fgHistory = data.slice(0, 30).map(d => ({
|
||||
value: parseInt(d.value, 10),
|
||||
date: new Date(parseInt(d.timestamp, 10) * 1000).toISOString().slice(0, 10),
|
||||
})).reverse();
|
||||
}
|
||||
|
||||
// Sparkline data
|
||||
const btcSparkline = btcPrices.slice(-30);
|
||||
const qqqSparkline = qqqPrices.slice(-30);
|
||||
const jpySparkline = jpyPrices.slice(-30);
|
||||
|
||||
// Overall Verdict
|
||||
let bullishCount = 0;
|
||||
let totalCount = 0;
|
||||
const signals = [
|
||||
{ name: 'Liquidity', status: liquidityStatus, bullish: liquidityStatus === 'NORMAL' },
|
||||
{ name: 'Flow Structure', status: flowStatus, bullish: flowStatus === 'ALIGNED' },
|
||||
{ name: 'Macro Regime', status: regimeStatus, bullish: regimeStatus === 'RISK-ON' },
|
||||
{ name: 'Technical Trend', status: trendStatus, bullish: trendStatus === 'BULLISH' },
|
||||
{ name: 'Hash Rate', status: hashStatus, bullish: hashStatus === 'GROWING' },
|
||||
{ name: 'Mining Cost', status: miningStatus, bullish: miningStatus === 'PROFITABLE' },
|
||||
{ name: 'Fear & Greed', status: fgLabel, bullish: fgValue !== null && fgValue > 50 },
|
||||
];
|
||||
|
||||
for (const s of signals) {
|
||||
if (s.status !== 'UNKNOWN') {
|
||||
totalCount++;
|
||||
if (s.bullish) bullishCount++;
|
||||
}
|
||||
}
|
||||
|
||||
const verdict = totalCount === 0 ? 'UNKNOWN' : (bullishCount / totalCount >= 0.57 ? 'BUY' : 'CASH');
|
||||
|
||||
const result = {
|
||||
timestamp: new Date().toISOString(),
|
||||
verdict,
|
||||
bullishCount,
|
||||
totalCount,
|
||||
signals: {
|
||||
liquidity: { status: liquidityStatus, value: jpyRoc30 !== null ? +jpyRoc30.toFixed(2) : null, sparkline: jpySparkline },
|
||||
flowStructure: { status: flowStatus, btcReturn5: btcReturn5 !== null ? +btcReturn5.toFixed(2) : null, qqqReturn5: qqqReturn5 !== null ? +qqqReturn5.toFixed(2) : null },
|
||||
macroRegime: { status: regimeStatus, qqqRoc20: qqqRoc20 !== null ? +qqqRoc20.toFixed(2) : null, xlpRoc20: xlpRoc20 !== null ? +xlpRoc20.toFixed(2) : null },
|
||||
technicalTrend: { status: trendStatus, btcPrice: btcCurrent, sma50: btcSma50 ? +btcSma50.toFixed(0) : null, sma200: btcSma200 ? +btcSma200.toFixed(0) : null, vwap30d: btcVwap, mayerMultiple, sparkline: btcSparkline },
|
||||
hashRate: { status: hashStatus, change30d: hashChange },
|
||||
miningCost: { status: miningStatus },
|
||||
fearGreed: { status: fgLabel, value: fgValue, history: fgHistory },
|
||||
},
|
||||
meta: { qqqSparkline },
|
||||
};
|
||||
|
||||
cachedResponse = result;
|
||||
cacheTimestamp = now;
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=600` },
|
||||
});
|
||||
} catch (err) {
|
||||
const fallback = cachedResponse || buildFallbackResult();
|
||||
cachedResponse = fallback;
|
||||
cacheTimestamp = now;
|
||||
return new Response(JSON.stringify(fallback), {
|
||||
status: 200,
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const response = await fetch(
|
||||
'https://msi.nga.mil/api/publications/broadcast-warn?output=json&status=A'
|
||||
);
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json', ...cors, 'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60' },
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: error.message }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
+7
-1
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
/**
|
||||
* Dynamic OG Image Generator for Story Sharing
|
||||
* Returns an SVG image (1200x630) — rich intelligence card for social previews.
|
||||
@@ -24,12 +25,17 @@ const LEVEL_LABELS = {
|
||||
low: 'LOW RISK',
|
||||
};
|
||||
|
||||
function normalizeLevel(rawLevel) {
|
||||
const level = String(rawLevel || '').toLowerCase();
|
||||
return Object.prototype.hasOwnProperty.call(LEVEL_COLORS, level) ? level : 'normal';
|
||||
}
|
||||
|
||||
export default function handler(req, res) {
|
||||
const url = new URL(req.url, `https://${req.headers.host}`);
|
||||
const countryCode = (url.searchParams.get('c') || '').toUpperCase();
|
||||
const type = url.searchParams.get('t') || 'ciianalysis';
|
||||
const score = url.searchParams.get('s');
|
||||
const level = url.searchParams.get('l') || 'normal';
|
||||
const level = normalizeLevel(url.searchParams.get('l'));
|
||||
|
||||
const countryName = COUNTRY_NAMES[countryCode] || countryCode || 'Global';
|
||||
const levelColor = LEVEL_COLORS[level] || '#eab308';
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
import { strict as assert } from 'node:assert';
|
||||
import test from 'node:test';
|
||||
import handler from './og-story.js';
|
||||
|
||||
function renderOgStory(query = '') {
|
||||
const req = {
|
||||
url: `https://worldmonitor.app/api/og-story${query ? `?${query}` : ''}`,
|
||||
headers: { host: 'worldmonitor.app' },
|
||||
};
|
||||
|
||||
let statusCode = 0;
|
||||
let body = '';
|
||||
const headers = {};
|
||||
|
||||
const res = {
|
||||
setHeader(name, value) {
|
||||
headers[String(name).toLowerCase()] = String(value);
|
||||
},
|
||||
status(code) {
|
||||
statusCode = code;
|
||||
return this;
|
||||
},
|
||||
send(payload) {
|
||||
body = String(payload);
|
||||
},
|
||||
};
|
||||
|
||||
handler(req, res);
|
||||
return { statusCode, body, headers };
|
||||
}
|
||||
|
||||
test('normalizes unsupported level values to prevent SVG script injection', () => {
|
||||
const injectedLevel = encodeURIComponent('</text><script>alert(1)</script><text>');
|
||||
const response = renderOgStory(`c=US&s=50&l=${injectedLevel}`);
|
||||
|
||||
assert.equal(response.statusCode, 200);
|
||||
assert.equal(/<script/i.test(response.body), false);
|
||||
assert.match(response.body, />NORMAL<\/text>/);
|
||||
});
|
||||
|
||||
test('uses a known level when it is allowlisted', () => {
|
||||
const response = renderOgStory('c=US&s=88&l=critical');
|
||||
|
||||
assert.equal(response.statusCode, 200);
|
||||
assert.match(response.body, />CRITICAL<\/text>/);
|
||||
assert.match(response.body, /#ef4444/);
|
||||
});
|
||||
|
||||
@@ -1,294 +0,0 @@
|
||||
/**
|
||||
* OpenRouter API Summarization Endpoint with Redis Caching
|
||||
* Fallback when Groq is rate-limited
|
||||
* Uses OpenRouter auto-routed free model
|
||||
* Free tier: 50 requests/day (20/min)
|
||||
* Server-side Redis cache for cross-user deduplication
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const OPENROUTER_API_URL = 'https://openrouter.ai/api/v1/chat/completions';
|
||||
const MODEL = 'openrouter/free';
|
||||
const CACHE_TTL_SECONDS = 86400; // 24 hours
|
||||
|
||||
const CACHE_VERSION = 'v3';
|
||||
|
||||
function getCacheKey(headlines, mode, geoContext = '', variant = 'full', lang = 'en') {
|
||||
const sorted = headlines.slice(0, 8).sort().join('|');
|
||||
const geoHash = geoContext ? ':g' + hashString(geoContext).slice(0, 6) : '';
|
||||
const hash = hashString(`${mode}:${sorted}`);
|
||||
const normalizedVariant = typeof variant === 'string' && variant ? variant.toLowerCase() : 'full';
|
||||
const normalizedLang = typeof lang === 'string' && lang ? lang.toLowerCase() : 'en';
|
||||
|
||||
if (mode === 'translate') {
|
||||
const targetLang = normalizedVariant || normalizedLang;
|
||||
return `summary:${CACHE_VERSION}:${mode}:${targetLang}:${hash}${geoHash}`;
|
||||
}
|
||||
|
||||
return `summary:${CACHE_VERSION}:${mode}:${normalizedVariant}:${normalizedLang}:${hash}${geoHash}`;
|
||||
}
|
||||
|
||||
// Deduplicate similar headlines (same story from different sources)
|
||||
function deduplicateHeadlines(headlines) {
|
||||
const seen = new Set();
|
||||
const unique = [];
|
||||
|
||||
for (const headline of headlines) {
|
||||
// Normalize: lowercase, remove punctuation, collapse whitespace
|
||||
const normalized = headline.toLowerCase()
|
||||
.replace(/[^\w\s]/g, '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
|
||||
// Extract key words (4+ chars) for similarity check
|
||||
const words = new Set(normalized.split(' ').filter(w => w.length >= 4));
|
||||
|
||||
// Check if this headline is too similar to any we've seen
|
||||
let isDuplicate = false;
|
||||
for (const seenWords of seen) {
|
||||
const intersection = [...words].filter(w => seenWords.has(w));
|
||||
const similarity = intersection.length / Math.min(words.size, seenWords.size);
|
||||
if (similarity > 0.6) {
|
||||
isDuplicate = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!isDuplicate) {
|
||||
seen.add(words);
|
||||
unique.push(headline);
|
||||
}
|
||||
}
|
||||
|
||||
return unique;
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.OPENROUTER_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ summary: null, fallback: true, skipped: true, reason: 'OPENROUTER_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return new Response(JSON.stringify({ error: 'Payload too large' }), {
|
||||
status: 413,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const { headlines, mode = 'brief', geoContext = '', variant = 'full', lang = 'en' } = await request.json();
|
||||
|
||||
if (!headlines || !Array.isArray(headlines) || headlines.length === 0) {
|
||||
return new Response(JSON.stringify({ error: 'Headlines array required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Check cache first (shared with Groq endpoint)
|
||||
const cacheKey = getCacheKey(headlines, mode, geoContext, variant, lang);
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.summary) {
|
||||
console.log('[OpenRouter] Cache hit:', cacheKey);
|
||||
return new Response(JSON.stringify({
|
||||
summary: cached.summary,
|
||||
model: cached.model || MODEL,
|
||||
provider: 'cache',
|
||||
cached: true,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Deduplicate similar headlines (same story from different sources)
|
||||
const uniqueHeadlines = deduplicateHeadlines(headlines.slice(0, 8));
|
||||
const headlineText = uniqueHeadlines.map((h, i) => `${i + 1}. ${h}`).join('\n');
|
||||
|
||||
let systemPrompt, userPrompt;
|
||||
|
||||
// Include intelligence synthesis context in prompt if available
|
||||
const intelSection = geoContext ? `\n\n${geoContext}` : '';
|
||||
|
||||
// Current date context for LLM (models may have outdated knowledge)
|
||||
const isTechVariant = variant === 'tech';
|
||||
const dateContext = `Current date: ${new Date().toISOString().split('T')[0]}.${isTechVariant ? '' : ' Donald Trump is the current US President (second term, inaugurated Jan 2025).'}`;
|
||||
|
||||
// Language instruction
|
||||
const langInstruction = lang && lang !== 'en' ? `\nIMPORTANT: Output the summary in ${lang.toUpperCase()} language.` : '';
|
||||
|
||||
if (mode === 'brief') {
|
||||
if (isTechVariant) {
|
||||
// Tech variant: focus on startups, AI, funding, product launches
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Summarize the key tech/startup development in 2-3 sentences.
|
||||
Rules:
|
||||
- Focus ONLY on technology, startups, AI, funding, product launches, or developer news
|
||||
- IGNORE political news, trade policy, tariffs, government actions unless directly about tech regulation
|
||||
- Lead with the company/product/technology name
|
||||
- Start directly: "OpenAI announced...", "A new $50M Series B...", "GitHub released..."
|
||||
- No bullet points, no meta-commentary${langInstruction}`;
|
||||
} else {
|
||||
// Full variant: geopolitical focus
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Summarize the key development in 2-3 sentences.
|
||||
Rules:
|
||||
- Lead with WHAT happened and WHERE - be specific
|
||||
- NEVER start with "Breaking news", "Good evening", "Tonight", or TV-style openings
|
||||
- Start directly with the subject: "Iran's regime...", "The US Treasury...", "Protests in..."
|
||||
- CRITICAL FOCAL POINTS are the main actors - mention them by name
|
||||
- If focal points show news + signals convergence, that's the lead
|
||||
- No bullet points, no meta-commentary${langInstruction}`;
|
||||
}
|
||||
userPrompt = `Summarize the top story:\n${headlineText}${intelSection}`;
|
||||
} else if (mode === 'analysis') {
|
||||
if (isTechVariant) {
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Analyze the tech/startup trend in 2-3 sentences.
|
||||
Rules:
|
||||
- Focus ONLY on technology implications: funding trends, AI developments, market shifts, product strategy
|
||||
- IGNORE political implications, trade wars, government unless directly about tech policy
|
||||
- Lead with the insight for tech industry
|
||||
- Connect to startup ecosystem, VC trends, or technical implications`;
|
||||
} else {
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Provide analysis in 2-3 sentences. Be direct and specific.
|
||||
Rules:
|
||||
- Lead with the insight - what's significant and why
|
||||
- NEVER start with "Breaking news", "Tonight", "The key/dominant narrative is"
|
||||
- Start with substance: "Iran faces...", "The escalation in...", "Multiple signals suggest..."
|
||||
- CRITICAL FOCAL POINTS are your main actors - explain WHY they matter
|
||||
- If focal points show news-signal correlation, flag as escalation
|
||||
- Connect dots, be specific about implications`;
|
||||
}
|
||||
userPrompt = isTechVariant
|
||||
? `What's the key tech trend or development?\n${headlineText}${intelSection}`
|
||||
: `What's the key pattern or risk?\n${headlineText}${intelSection}`;
|
||||
} else if (mode === 'translate') {
|
||||
const targetLang = variant; // In translate mode, variant param holds the target language code
|
||||
systemPrompt = `You are a professional news translator. Translate the following news headlines/summaries into ${targetLang}.
|
||||
Rules:
|
||||
- Maintain the original tone and journalistic style.
|
||||
- Do NOT add any conversational filler.
|
||||
- Output ONLY the translated text.`;
|
||||
userPrompt = `Translate to ${targetLang}:\n${headlines[0]}`;
|
||||
} else {
|
||||
systemPrompt = isTechVariant
|
||||
? `${dateContext}\n\nSynthesize tech news in 2 sentences. Focus on startups, AI, funding, products. Ignore politics unless directly about tech regulation.${langInstruction}`
|
||||
: `${dateContext}\n\nSynthesize in 2 sentences max. Lead with substance. NEVER start with "Breaking news" or "Tonight" - just state the insight directly. CRITICAL focal points with news-signal convergence are significant.${langInstruction}`;
|
||||
userPrompt = `Key takeaway:\n${headlineText}${intelSection}`;
|
||||
}
|
||||
|
||||
const response = await fetch(OPENROUTER_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
'HTTP-Referer': 'https://worldmonitor.app',
|
||||
'X-Title': 'WorldMonitor',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
temperature: 0.3,
|
||||
max_tokens: 150,
|
||||
top_p: 0.9,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.error('[OpenRouter] API error:', response.status, errorText);
|
||||
|
||||
// Return fallback signal for rate limiting
|
||||
if (response.status === 429) {
|
||||
return new Response(JSON.stringify({ error: 'Rate limited', fallback: true }), {
|
||||
status: 429,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({ error: 'OpenRouter API error', fallback: true }), {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const summary = data.choices?.[0]?.message?.content?.trim();
|
||||
|
||||
if (!summary) {
|
||||
return new Response(JSON.stringify({ error: 'Empty response', fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Store in cache (shared with Groq endpoint)
|
||||
await setCachedJson(cacheKey, {
|
||||
summary,
|
||||
model: MODEL,
|
||||
timestamp: Date.now(),
|
||||
}, CACHE_TTL_SECONDS);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
summary,
|
||||
model: MODEL,
|
||||
provider: 'openrouter',
|
||||
cached: false,
|
||||
tokens: data.usage?.total_tokens || 0,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
|
||||
},
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('[OpenRouter] Error:', error);
|
||||
return new Response(JSON.stringify({ error: error.message, fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
+72
-55
@@ -1,73 +1,90 @@
|
||||
// OpenSky Network API proxy - v3
|
||||
// Note: OpenSky seems to block some cloud provider IPs
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
function getRelayBaseUrl() {
|
||||
const relayUrl = process.env.WS_RELAY_URL;
|
||||
if (!relayUrl) return null;
|
||||
return relayUrl.replace('wss://', 'https://').replace('ws://', 'http://').replace(/\/$/, '');
|
||||
}
|
||||
|
||||
function getRelayHeaders(baseHeaders = {}) {
|
||||
const headers = { ...baseHeaders };
|
||||
const relaySecret = process.env.RELAY_SHARED_SECRET || '';
|
||||
if (relaySecret) {
|
||||
const relayHeader = (process.env.RELAY_AUTH_HEADER || 'x-relay-key').toLowerCase();
|
||||
headers[relayHeader] = relaySecret;
|
||||
headers.Authorization = `Bearer ${relaySecret}`;
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
return headers;
|
||||
}
|
||||
|
||||
// Build OpenSky API URL with bounding box params
|
||||
const params = new URLSearchParams();
|
||||
['lamin', 'lomin', 'lamax', 'lomax'].forEach(key => {
|
||||
const val = url.searchParams.get(key);
|
||||
if (val) params.set(key, val);
|
||||
});
|
||||
async function fetchWithTimeout(url, options, timeoutMs = 15000) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
const openskyUrl = `https://opensky-network.org/api/states/all${params.toString() ? '?' + params.toString() : ''}`;
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const relayBaseUrl = getRelayBaseUrl();
|
||||
if (!relayBaseUrl) {
|
||||
return new Response(JSON.stringify({ error: 'WS_RELAY_URL is not configured' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Try fetching with different headers to avoid blocks
|
||||
const response = await fetch(openskyUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
'Accept-Encoding': 'gzip, deflate, br',
|
||||
'Connection': 'keep-alive',
|
||||
},
|
||||
const requestUrl = new URL(req.url);
|
||||
const relayUrl = `${relayBaseUrl}/opensky${requestUrl.search || ''}`;
|
||||
const response = await fetchWithTimeout(relayUrl, {
|
||||
headers: getRelayHeaders({ Accept: 'application/json' }),
|
||||
});
|
||||
|
||||
if (response.status === 429) {
|
||||
return Response.json({ error: 'Rate limited', time: Date.now(), states: null }, {
|
||||
status: 429,
|
||||
headers: cors,
|
||||
});
|
||||
}
|
||||
const body = await response.text();
|
||||
const headers = {
|
||||
'Content-Type': response.headers.get('content-type') || 'application/json',
|
||||
'Cache-Control': response.headers.get('cache-control') || 'no-cache',
|
||||
...corsHeaders,
|
||||
};
|
||||
const xCache = response.headers.get('x-cache');
|
||||
if (xCache) headers['X-Cache'] = xCache;
|
||||
|
||||
// Check if response is OK
|
||||
if (!response.ok) {
|
||||
const text = await response.text();
|
||||
return Response.json({
|
||||
error: `OpenSky HTTP ${response.status}: ${text.substring(0, 200)}`,
|
||||
time: Date.now(),
|
||||
states: null
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: cors,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
return Response.json(data, {
|
||||
return new Response(body, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
},
|
||||
headers,
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${error.name} - ${error.message}`,
|
||||
time: Date.now(),
|
||||
states: null
|
||||
}, {
|
||||
status: 500,
|
||||
headers: cors,
|
||||
const isTimeout = error?.name === 'AbortError';
|
||||
return new Response(JSON.stringify({
|
||||
error: isTimeout ? 'Relay timeout' : 'Relay request failed',
|
||||
details: error?.message || String(error),
|
||||
}), {
|
||||
status: isTimeout ? 504 : 502,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const response = await fetch('https://www.pizzint.watch/api/dashboard-data', {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'WorldMonitor/1.0',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Upstream returned ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch PizzINT data', details: error.message }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,46 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const pairs = url.searchParams.get('pairs') || 'usa_russia,russia_ukraine,usa_china,china_taiwan,usa_iran,usa_venezuela';
|
||||
const dateStart = url.searchParams.get('dateStart');
|
||||
const dateEnd = url.searchParams.get('dateEnd');
|
||||
const method = url.searchParams.get('method') || 'gpr';
|
||||
|
||||
let targetUrl = `https://www.pizzint.watch/api/gdelt/batch?pairs=${encodeURIComponent(pairs)}&method=${method}`;
|
||||
if (dateStart) targetUrl += `&dateStart=${dateStart}`;
|
||||
if (dateEnd) targetUrl += `&dateEnd=${dateEnd}`;
|
||||
|
||||
try {
|
||||
const response = await fetch(targetUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'WorldMonitor/1.0',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Upstream returned ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch GDELT data', details: error.message }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
+69
-89
@@ -2,107 +2,87 @@ import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const GAMMA_BASE = 'https://gamma-api.polymarket.com';
|
||||
|
||||
const ALLOWED_ORDER = ['volume', 'liquidity', 'startDate', 'endDate', 'spread'];
|
||||
const MAX_LIMIT = 100;
|
||||
const MIN_LIMIT = 1;
|
||||
|
||||
function validateBoolean(val, defaultVal) {
|
||||
if (val === 'true' || val === 'false') return val;
|
||||
return defaultVal;
|
||||
function getRelayBaseUrl() {
|
||||
const relayUrl = process.env.WS_RELAY_URL;
|
||||
if (!relayUrl) return null;
|
||||
return relayUrl.replace('wss://', 'https://').replace('ws://', 'http://').replace(/\/$/, '');
|
||||
}
|
||||
|
||||
function validateLimit(val) {
|
||||
const num = parseInt(val, 10);
|
||||
if (isNaN(num)) return 50;
|
||||
return Math.max(MIN_LIMIT, Math.min(MAX_LIMIT, num));
|
||||
function getRelayHeaders(baseHeaders = {}) {
|
||||
const headers = { ...baseHeaders };
|
||||
const relaySecret = process.env.RELAY_SHARED_SECRET || '';
|
||||
if (relaySecret) {
|
||||
const relayHeader = (process.env.RELAY_AUTH_HEADER || 'x-relay-key').toLowerCase();
|
||||
headers[relayHeader] = relaySecret;
|
||||
headers.Authorization = `Bearer ${relaySecret}`;
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
function validateOrder(val) {
|
||||
return ALLOWED_ORDER.includes(val) ? val : 'volume';
|
||||
}
|
||||
|
||||
function sanitizeTagSlug(val) {
|
||||
if (!val) return null;
|
||||
return val.replace(/[^a-z0-9-]/gi, '').slice(0, 100) || null;
|
||||
}
|
||||
|
||||
async function tryFetch(url, timeoutMs = 8000) {
|
||||
async function fetchWithTimeout(url, options, timeoutMs = 15000) {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timer);
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}`);
|
||||
}
|
||||
return await response.text();
|
||||
} catch (err) {
|
||||
clearTimeout(timer);
|
||||
throw err;
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
function buildUrl(base, endpoint, params) {
|
||||
if (endpoint === 'events') {
|
||||
return `${base}/events?${params}`;
|
||||
}
|
||||
return `${base}/markets?${params}`;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const endpoint = url.searchParams.get('endpoint') || 'markets';
|
||||
|
||||
const closed = validateBoolean(url.searchParams.get('closed'), 'false');
|
||||
const order = validateOrder(url.searchParams.get('order'));
|
||||
const ascending = validateBoolean(url.searchParams.get('ascending'), 'false');
|
||||
const limit = validateLimit(url.searchParams.get('limit'));
|
||||
|
||||
const params = new URLSearchParams({
|
||||
closed,
|
||||
order,
|
||||
ascending,
|
||||
limit: String(limit),
|
||||
});
|
||||
|
||||
if (endpoint === 'events') {
|
||||
const tag = sanitizeTagSlug(url.searchParams.get('tag'));
|
||||
if (tag) params.set('tag_slug', tag);
|
||||
}
|
||||
|
||||
// Gamma API is behind Cloudflare which blocks server-side TLS connections
|
||||
// (JA3 fingerprint detection). Only browser-originated requests succeed.
|
||||
// We still try in case Cloudflare policy changes, but gracefully return empty on failure.
|
||||
try {
|
||||
const data = await tryFetch(buildUrl(GAMMA_BASE, endpoint, params));
|
||||
return new Response(data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=120, s-maxage=120, stale-while-revalidate=60',
|
||||
'X-Polymarket-Source': 'gamma',
|
||||
},
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
} catch (err) {
|
||||
// Expected: Cloudflare blocks non-browser TLS connections
|
||||
return new Response(JSON.stringify([]), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'X-Polymarket-Error': err.message,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
}
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const relayBaseUrl = getRelayBaseUrl();
|
||||
if (!relayBaseUrl) {
|
||||
return new Response(JSON.stringify({ error: 'WS_RELAY_URL is not configured' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const requestUrl = new URL(req.url);
|
||||
const relayUrl = `${relayBaseUrl}/polymarket${requestUrl.search || ''}`;
|
||||
const response = await fetchWithTimeout(relayUrl, {
|
||||
headers: getRelayHeaders({ Accept: 'application/json' }),
|
||||
}, 15000);
|
||||
|
||||
const body = await response.text();
|
||||
const headers = {
|
||||
'Content-Type': response.headers.get('content-type') || 'application/json',
|
||||
'Cache-Control': response.headers.get('cache-control') || 'no-cache',
|
||||
...corsHeaders,
|
||||
};
|
||||
|
||||
return new Response(body, {
|
||||
status: response.status,
|
||||
headers,
|
||||
});
|
||||
} catch (error) {
|
||||
const isTimeout = error?.name === 'AbortError';
|
||||
return new Response(JSON.stringify({
|
||||
error: isTimeout ? 'Relay timeout' : 'Relay request failed',
|
||||
details: error?.message || String(error),
|
||||
}), {
|
||||
status: isTimeout ? 504 : 502,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { ConvexHttpClient } from 'convex/browser';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const EMAIL_RE = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
|
||||
const MAX_EMAIL_LENGTH = 320;
|
||||
|
||||
const rateLimitMap = new Map();
|
||||
const RATE_LIMIT = 5;
|
||||
const RATE_WINDOW_MS = 60 * 60 * 1000;
|
||||
|
||||
function isRateLimited(ip) {
|
||||
const now = Date.now();
|
||||
const entry = rateLimitMap.get(ip);
|
||||
if (!entry || now - entry.windowStart > RATE_WINDOW_MS) {
|
||||
rateLimitMap.set(ip, { windowStart: now, count: 1 });
|
||||
return false;
|
||||
}
|
||||
entry.count += 1;
|
||||
return entry.count > RATE_LIMIT;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const cors = getCorsHeaders(req, 'POST, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
|
||||
if (req.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
const ip = req.headers.get('x-forwarded-for')?.split(',')[0]?.trim() || 'unknown';
|
||||
if (isRateLimited(ip)) {
|
||||
return new Response(JSON.stringify({ error: 'Too many requests' }), {
|
||||
status: 429,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
let body;
|
||||
try {
|
||||
body = await req.json();
|
||||
} catch {
|
||||
return new Response(JSON.stringify({ error: 'Invalid JSON' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
const { email, source, appVersion } = body;
|
||||
if (!email || typeof email !== 'string' || email.length > MAX_EMAIL_LENGTH || !EMAIL_RE.test(email)) {
|
||||
return new Response(JSON.stringify({ error: 'Invalid email address' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
const convexUrl = process.env.CONVEX_URL;
|
||||
if (!convexUrl) {
|
||||
return new Response(JSON.stringify({ error: 'Registration service unavailable' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const client = new ConvexHttpClient(convexUrl);
|
||||
const result = await client.mutation('registerInterest:register', {
|
||||
email,
|
||||
source: source || 'unknown',
|
||||
appVersion: appVersion || 'unknown',
|
||||
});
|
||||
return new Response(JSON.stringify(result), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[register-interest] Convex error:', err);
|
||||
return new Response(JSON.stringify({ error: 'Registration failed' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,355 +0,0 @@
|
||||
/**
|
||||
* Risk Scores API - Cached CII and Strategic Risk computation
|
||||
* Eliminates 15-minute "learning mode" for users by pre-computing scores
|
||||
* Uses Upstash Redis for cross-user caching (10-minute TTL)
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const CACHE_TTL_SECONDS = 600; // 10 minutes
|
||||
const STALE_CACHE_TTL_SECONDS = 3600; // 1 hour - serve stale when API fails
|
||||
const CACHE_KEY = 'risk:scores:v2';
|
||||
const STALE_CACHE_KEY = 'risk:scores:stale:v2';
|
||||
|
||||
// Tier 1 countries for CII
|
||||
const TIER1_COUNTRIES = {
|
||||
US: 'United States', RU: 'Russia', CN: 'China', UA: 'Ukraine', IR: 'Iran',
|
||||
IL: 'Israel', TW: 'Taiwan', KP: 'North Korea', SA: 'Saudi Arabia', TR: 'Turkey',
|
||||
PL: 'Poland', DE: 'Germany', FR: 'France', GB: 'United Kingdom', IN: 'India',
|
||||
PK: 'Pakistan', SY: 'Syria', YE: 'Yemen', MM: 'Myanmar', VE: 'Venezuela',
|
||||
};
|
||||
|
||||
// Baseline geopolitical risk (0-50)
|
||||
const BASELINE_RISK = {
|
||||
US: 5, RU: 35, CN: 25, UA: 50, IR: 40, IL: 45, TW: 30, KP: 45,
|
||||
SA: 20, TR: 25, PL: 10, DE: 5, FR: 10, GB: 5, IN: 20, PK: 35,
|
||||
SY: 50, YE: 50, MM: 45, VE: 40,
|
||||
};
|
||||
|
||||
// Event significance multipliers
|
||||
const EVENT_MULTIPLIER = {
|
||||
US: 0.3, RU: 2.0, CN: 2.5, UA: 0.8, IR: 2.0, IL: 0.7, TW: 1.5, KP: 3.0,
|
||||
SA: 2.0, TR: 1.2, PL: 0.8, DE: 0.5, FR: 0.6, GB: 0.5, IN: 0.8, PK: 1.5,
|
||||
SY: 0.7, YE: 0.7, MM: 1.8, VE: 1.8,
|
||||
};
|
||||
|
||||
// Country keywords for matching
|
||||
const COUNTRY_KEYWORDS = {
|
||||
US: ['united states', 'usa', 'america', 'washington', 'biden', 'trump', 'pentagon'],
|
||||
RU: ['russia', 'moscow', 'kremlin', 'putin'],
|
||||
CN: ['china', 'beijing', 'xi jinping', 'prc'],
|
||||
UA: ['ukraine', 'kyiv', 'zelensky', 'donbas'],
|
||||
IR: ['iran', 'tehran', 'khamenei', 'irgc'],
|
||||
IL: ['israel', 'tel aviv', 'netanyahu', 'idf', 'gaza'],
|
||||
TW: ['taiwan', 'taipei'],
|
||||
KP: ['north korea', 'pyongyang', 'kim jong'],
|
||||
SA: ['saudi arabia', 'riyadh'],
|
||||
TR: ['turkey', 'ankara', 'erdogan'],
|
||||
PL: ['poland', 'warsaw'],
|
||||
DE: ['germany', 'berlin'],
|
||||
FR: ['france', 'paris', 'macron'],
|
||||
GB: ['britain', 'uk', 'london'],
|
||||
IN: ['india', 'delhi', 'modi'],
|
||||
PK: ['pakistan', 'islamabad'],
|
||||
SY: ['syria', 'damascus'],
|
||||
YE: ['yemen', 'sanaa', 'houthi'],
|
||||
MM: ['myanmar', 'burma'],
|
||||
VE: ['venezuela', 'caracas', 'maduro'],
|
||||
};
|
||||
|
||||
function normalizeCountryName(text) {
|
||||
const lower = text.toLowerCase();
|
||||
for (const [code, keywords] of Object.entries(COUNTRY_KEYWORDS)) {
|
||||
if (keywords.some(kw => lower.includes(kw))) return code;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function getScoreLevel(score) {
|
||||
if (score >= 70) return 'critical';
|
||||
if (score >= 55) return 'high';
|
||||
if (score >= 40) return 'elevated';
|
||||
if (score >= 25) return 'normal';
|
||||
return 'low';
|
||||
}
|
||||
|
||||
async function fetchACLEDProtests() {
|
||||
try {
|
||||
// Fetch recent protests from ACLED (last 7 days)
|
||||
const endDate = new Date().toISOString().split('T')[0];
|
||||
const startDate = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
|
||||
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 15000); // 15s timeout
|
||||
|
||||
// ACLED API now requires authentication - new endpoint as of Jan 2026
|
||||
const token = process.env.ACLED_ACCESS_TOKEN;
|
||||
const headers = { 'Accept': 'application/json' };
|
||||
if (token) {
|
||||
headers['Authorization'] = `Bearer ${token}`;
|
||||
}
|
||||
|
||||
// Updated endpoint: acleddata.com/api/ instead of api.acleddata.com
|
||||
const response = await fetch(
|
||||
`https://acleddata.com/api/acled/read?_format=json&event_type=Protests&event_type=Riots&event_date=${startDate}|${endDate}&event_date_where=BETWEEN&limit=500`,
|
||||
{
|
||||
headers,
|
||||
signal: controller.signal,
|
||||
}
|
||||
);
|
||||
|
||||
clearTimeout(timeoutId);
|
||||
|
||||
if (!response.ok) {
|
||||
const text = await response.text().catch(() => '');
|
||||
console.warn('[RiskScores] ACLED fetch failed:', response.status, text.slice(0, 200));
|
||||
// Check for auth errors specifically
|
||||
if (response.status === 401 || response.status === 403) {
|
||||
throw new Error('ACLED API requires valid authentication token');
|
||||
}
|
||||
throw new Error(`ACLED API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
// Check for API-level error in response
|
||||
if (data.message) {
|
||||
console.warn('[RiskScores] ACLED API returned message:', data.message);
|
||||
throw new Error(data.message);
|
||||
}
|
||||
if (data.error || data.success === false) {
|
||||
console.warn('[RiskScores] ACLED API returned error:', data.error || 'unknown');
|
||||
throw new Error(data.error || 'ACLED API error');
|
||||
}
|
||||
|
||||
return data.data || [];
|
||||
} catch (error) {
|
||||
console.warn('[RiskScores] ACLED error:', error.message);
|
||||
throw error; // Re-throw to trigger stale cache fallback
|
||||
}
|
||||
}
|
||||
|
||||
function computeCIIScores(protests) {
|
||||
const countryEvents = new Map();
|
||||
|
||||
// Count events per country
|
||||
for (const event of protests) {
|
||||
const country = event.country;
|
||||
const code = normalizeCountryName(country);
|
||||
if (code && TIER1_COUNTRIES[code]) {
|
||||
const count = countryEvents.get(code) || { protests: 0, riots: 0 };
|
||||
if (event.event_type === 'Riots') {
|
||||
count.riots++;
|
||||
} else {
|
||||
count.protests++;
|
||||
}
|
||||
countryEvents.set(code, count);
|
||||
}
|
||||
}
|
||||
|
||||
// Compute scores for all Tier 1 countries
|
||||
const scores = [];
|
||||
const now = new Date();
|
||||
|
||||
for (const [code, name] of Object.entries(TIER1_COUNTRIES)) {
|
||||
const events = countryEvents.get(code) || { protests: 0, riots: 0 };
|
||||
const baseline = BASELINE_RISK[code] || 20;
|
||||
const multiplier = EVENT_MULTIPLIER[code] || 1.0;
|
||||
|
||||
// Unrest component: protests + riots (riots weighted 2x)
|
||||
const unrestRaw = (events.protests + events.riots * 2) * multiplier;
|
||||
const unrest = Math.min(100, Math.round(unrestRaw * 2));
|
||||
|
||||
// Security component: baseline + riot contribution
|
||||
const security = Math.min(100, baseline + events.riots * multiplier * 5);
|
||||
|
||||
// Information component: based on event count (proxy for news coverage)
|
||||
const totalEvents = events.protests + events.riots;
|
||||
const information = Math.min(100, totalEvents * multiplier * 3);
|
||||
|
||||
// Composite score: weighted average + baseline
|
||||
const composite = Math.min(100, Math.round(
|
||||
baseline +
|
||||
(unrest * 0.4 + security * 0.35 + information * 0.25) * 0.5
|
||||
));
|
||||
|
||||
scores.push({
|
||||
code,
|
||||
name,
|
||||
score: composite,
|
||||
level: getScoreLevel(composite),
|
||||
trend: 'stable', // Would need historical data for real trend
|
||||
change24h: 0,
|
||||
components: { unrest, security, information },
|
||||
lastUpdated: now.toISOString(),
|
||||
});
|
||||
}
|
||||
|
||||
// Sort by score descending
|
||||
scores.sort((a, b) => b.score - a.score);
|
||||
return scores;
|
||||
}
|
||||
|
||||
function computeStrategicRisk(ciiScores) {
|
||||
// Top 5 CII scores weighted average
|
||||
const top5 = ciiScores.slice(0, 5);
|
||||
const weights = top5.map((_, i) => 1 - (i * 0.15)); // [1.0, 0.85, 0.70, 0.55, 0.40]
|
||||
const totalWeight = weights.reduce((sum, w) => sum + w, 0); // 3.5
|
||||
const weightedSum = top5.reduce((sum, s, i) => sum + s.score * weights[i], 0);
|
||||
const ciiComponent = weightedSum / totalWeight;
|
||||
|
||||
// Overall strategic risk
|
||||
const overallScore = Math.round(ciiComponent * 0.7 + 15); // 30% baseline
|
||||
|
||||
return {
|
||||
score: Math.min(100, overallScore),
|
||||
level: getScoreLevel(overallScore),
|
||||
trend: 'stable',
|
||||
lastUpdated: new Date().toISOString(),
|
||||
contributors: top5.map(s => ({
|
||||
country: s.name,
|
||||
code: s.code,
|
||||
score: s.score,
|
||||
level: s.level,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'GET, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (!process.env.ACLED_ACCESS_TOKEN) {
|
||||
const baselineScores = computeCIIScores([]);
|
||||
const baselineStrategic = computeStrategicRisk(baselineScores);
|
||||
return new Response(JSON.stringify({
|
||||
cii: baselineScores,
|
||||
strategicRisk: baselineStrategic,
|
||||
protestCount: 0,
|
||||
computedAt: new Date().toISOString(),
|
||||
baseline: true,
|
||||
error: 'ACLED token not configured - showing baseline risk assessments',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Check cache first
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (cached && typeof cached === 'object') {
|
||||
console.log('[RiskScores] Cache hit');
|
||||
return new Response(JSON.stringify({
|
||||
...cached,
|
||||
cached: true,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Fetch ACLED protests
|
||||
console.log('[RiskScores] Computing scores...');
|
||||
const protests = await fetchACLEDProtests();
|
||||
|
||||
// Compute CII scores
|
||||
const ciiScores = computeCIIScores(protests);
|
||||
|
||||
// Compute strategic risk
|
||||
const strategicRisk = computeStrategicRisk(ciiScores);
|
||||
|
||||
const result = {
|
||||
cii: ciiScores,
|
||||
strategicRisk,
|
||||
protestCount: protests.length,
|
||||
computedAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
// Cache (both regular and stale backup)
|
||||
await Promise.all([
|
||||
setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS),
|
||||
setCachedJson(STALE_CACHE_KEY, result, STALE_CACHE_TTL_SECONDS),
|
||||
]);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
...result,
|
||||
cached: false,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('[RiskScores] Error:', error);
|
||||
|
||||
// Try to return stale cached data
|
||||
const stale = await getCachedJson(STALE_CACHE_KEY);
|
||||
if (stale && typeof stale === 'object') {
|
||||
console.log('[RiskScores] Returning stale cache due to error');
|
||||
return new Response(JSON.stringify({
|
||||
...stale,
|
||||
cached: true,
|
||||
stale: true,
|
||||
error: 'Using cached data - ACLED temporarily unavailable',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Final fallback: return baseline scores without unrest data
|
||||
console.log('[RiskScores] Returning baseline scores (no ACLED data)');
|
||||
const baselineScores = computeCIIScores([]); // Empty protests = baseline only
|
||||
const baselineStrategic = computeStrategicRisk(baselineScores);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
cii: baselineScores,
|
||||
strategicRisk: baselineStrategic,
|
||||
protestCount: 0,
|
||||
computedAt: new Date().toISOString(),
|
||||
baseline: true,
|
||||
error: 'ACLED unavailable - showing baseline risk assessments',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
+158
-35
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
@@ -14,6 +15,35 @@ async function fetchWithTimeout(url, options, timeoutMs = 15000) {
|
||||
}
|
||||
}
|
||||
|
||||
function getRelayBaseUrl() {
|
||||
const relayUrl = process.env.WS_RELAY_URL || '';
|
||||
if (!relayUrl) return '';
|
||||
return relayUrl.replace('wss://', 'https://').replace('ws://', 'http://').replace(/\/$/, '');
|
||||
}
|
||||
|
||||
function getRelayHeaders(baseHeaders = {}) {
|
||||
const headers = { ...baseHeaders };
|
||||
const relaySecret = process.env.RELAY_SHARED_SECRET || '';
|
||||
if (relaySecret) {
|
||||
const relayHeader = (process.env.RELAY_AUTH_HEADER || 'x-relay-key').toLowerCase();
|
||||
headers[relayHeader] = relaySecret;
|
||||
headers.Authorization = `Bearer ${relaySecret}`;
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
async function fetchViaRailway(feedUrl, timeoutMs) {
|
||||
const relayBaseUrl = getRelayBaseUrl();
|
||||
if (!relayBaseUrl) return null;
|
||||
const relayUrl = `${relayBaseUrl}/rss?url=${encodeURIComponent(feedUrl)}`;
|
||||
return fetchWithTimeout(relayUrl, {
|
||||
headers: getRelayHeaders({
|
||||
'Accept': 'application/rss+xml, application/xml, text/xml, */*',
|
||||
'User-Agent': 'WorldMonitor-RSS-Proxy/1.0',
|
||||
}),
|
||||
}, timeoutMs);
|
||||
}
|
||||
|
||||
// Allowed RSS feed domains for security
|
||||
const ALLOWED_DOMAINS = [
|
||||
'feeds.bbci.co.uk',
|
||||
@@ -21,6 +51,7 @@ const ALLOWED_DOMAINS = [
|
||||
'feeds.npr.org',
|
||||
'news.google.com',
|
||||
'www.aljazeera.com',
|
||||
'www.aljazeera.net',
|
||||
'rss.cnn.com',
|
||||
'hnrss.org',
|
||||
'feeds.arstechnica.com',
|
||||
@@ -63,6 +94,11 @@ const ALLOWED_DOMAINS = [
|
||||
'www.brookings.edu',
|
||||
'layoffs.fyi',
|
||||
'www.defensenews.com',
|
||||
'www.militarytimes.com',
|
||||
'taskandpurpose.com',
|
||||
'news.usni.org',
|
||||
'www.oryxspioenkop.com',
|
||||
'www.gov.uk',
|
||||
'www.foreignaffairs.com',
|
||||
'www.atlanticcouncil.org',
|
||||
// Tech variant domains
|
||||
@@ -70,6 +106,7 @@ const ALLOWED_DOMAINS = [
|
||||
'www.techmeme.com',
|
||||
'www.darkreading.com',
|
||||
'www.schneier.com',
|
||||
'www.ransomware.live',
|
||||
'rss.politico.com',
|
||||
'www.anandtech.com',
|
||||
'www.tomshardware.com',
|
||||
@@ -133,11 +170,30 @@ const ALLOWED_DOMAINS = [
|
||||
// International News Sources
|
||||
'www.france24.com',
|
||||
'www.euronews.com',
|
||||
'de.euronews.com',
|
||||
'es.euronews.com',
|
||||
'fr.euronews.com',
|
||||
'it.euronews.com',
|
||||
'pt.euronews.com',
|
||||
'ru.euronews.com',
|
||||
'www.lemonde.fr',
|
||||
'rss.dw.com',
|
||||
'www.bild.de',
|
||||
'www.africanews.com',
|
||||
'fr.africanews.com',
|
||||
// Nigeria
|
||||
'www.premiumtimesng.com',
|
||||
'www.vanguardngr.com',
|
||||
'www.channelstv.com',
|
||||
'dailytrust.com',
|
||||
'www.thisdaylive.com',
|
||||
// Greek
|
||||
'www.naftemporiki.gr',
|
||||
'www.in.gr',
|
||||
'www.iefimerida.gr',
|
||||
'www.lasillavacia.com',
|
||||
'www.channelnewsasia.com',
|
||||
'japantoday.com',
|
||||
'www.thehindu.com',
|
||||
// International Organizations
|
||||
'news.un.org',
|
||||
@@ -171,12 +227,74 @@ const ALLOWED_DOMAINS = [
|
||||
'www.fao.org',
|
||||
'worldbank.org',
|
||||
'www.imf.org',
|
||||
// International news (various languages)
|
||||
'www.bbc.com',
|
||||
'www.spiegel.de',
|
||||
'www.tagesschau.de',
|
||||
'newsfeed.zeit.de',
|
||||
'feeds.elpais.com',
|
||||
'e00-elmundo.uecdn.es',
|
||||
'www.repubblica.it',
|
||||
'www.ansa.it',
|
||||
'xml2.corriereobjects.it',
|
||||
'feeds.nos.nl',
|
||||
'www.nrc.nl',
|
||||
'www.telegraaf.nl',
|
||||
'www.dn.se',
|
||||
'www.svd.se',
|
||||
'www.svt.se',
|
||||
'www.asahi.com',
|
||||
'www.clarin.com',
|
||||
'oglobo.globo.com',
|
||||
'feeds.folha.uol.com.br',
|
||||
'www.eltiempo.com',
|
||||
'www.eluniversal.com.mx',
|
||||
'www.jeuneafrique.com',
|
||||
'www.lorientlejour.com',
|
||||
// Regional locale feeds (tr, pl, ru, th, vi, pt)
|
||||
'www.hurriyet.com.tr',
|
||||
'tvn24.pl',
|
||||
'www.polsatnews.pl',
|
||||
'www.rp.pl',
|
||||
'meduza.io',
|
||||
'novayagazeta.eu',
|
||||
'www.bangkokpost.com',
|
||||
'vnexpress.net',
|
||||
'www.abc.net.au',
|
||||
'islandtimes.org',
|
||||
'www.brasilparalelo.com.br',
|
||||
// Mexico & LatAm Security
|
||||
'mexiconewsdaily.com',
|
||||
'animalpolitico.com',
|
||||
'www.proceso.com.mx',
|
||||
'www.milenio.com',
|
||||
'insightcrime.org',
|
||||
// Additional
|
||||
'news.ycombinator.com',
|
||||
// Finance variant
|
||||
'seekingalpha.com',
|
||||
'www.coindesk.com',
|
||||
'cointelegraph.com',
|
||||
// Happy variant — positive news sources
|
||||
'www.goodnewsnetwork.org',
|
||||
'www.positive.news',
|
||||
'reasonstobecheerful.world',
|
||||
'www.optimistdaily.com',
|
||||
'www.upworthy.com',
|
||||
'www.dailygood.org',
|
||||
'www.goodgoodgood.co',
|
||||
'www.good.is',
|
||||
'www.sunnyskyz.com',
|
||||
'thebetterindia.com',
|
||||
'singularityhub.com',
|
||||
'humanprogress.org',
|
||||
'greatergood.berkeley.edu',
|
||||
'www.onlygoodnewsdaily.com',
|
||||
'www.sciencedaily.com',
|
||||
'feeds.nature.com',
|
||||
'www.nature.com',
|
||||
'www.livescience.com',
|
||||
'www.newscientist.com',
|
||||
];
|
||||
|
||||
export default async function handler(req) {
|
||||
@@ -200,8 +318,11 @@ export default async function handler(req) {
|
||||
try {
|
||||
const parsedUrl = new URL(feedUrl);
|
||||
|
||||
// Security: Check if domain is allowed
|
||||
if (!ALLOWED_DOMAINS.includes(parsedUrl.hostname)) {
|
||||
// Security: Check if domain is allowed (normalize www prefix)
|
||||
const hostname = parsedUrl.hostname;
|
||||
const bare = hostname.replace(/^www\./, '');
|
||||
const withWww = hostname.startsWith('www.') ? hostname : `www.${hostname}`;
|
||||
if (!ALLOWED_DOMAINS.includes(hostname) && !ALLOWED_DOMAINS.includes(bare) && !ALLOWED_DOMAINS.includes(withWww)) {
|
||||
return new Response(JSON.stringify({ error: 'Domain not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
@@ -212,57 +333,59 @@ export default async function handler(req) {
|
||||
const isGoogleNews = feedUrl.includes('news.google.com');
|
||||
const timeout = isGoogleNews ? 20000 : 12000;
|
||||
|
||||
const response = await fetchWithTimeout(feedUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'Accept': 'application/rss+xml, application/xml, text/xml, */*',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
},
|
||||
redirect: 'manual',
|
||||
}, timeout);
|
||||
const fetchDirect = async () => {
|
||||
const response = await fetchWithTimeout(feedUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'Accept': 'application/rss+xml, application/xml, text/xml, */*',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
},
|
||||
redirect: 'manual',
|
||||
}, timeout);
|
||||
|
||||
if (response.status >= 300 && response.status < 400) {
|
||||
const location = response.headers.get('location');
|
||||
if (location) {
|
||||
try {
|
||||
if (response.status >= 300 && response.status < 400) {
|
||||
const location = response.headers.get('location');
|
||||
if (location) {
|
||||
const redirectUrl = new URL(location, feedUrl);
|
||||
if (!ALLOWED_DOMAINS.includes(redirectUrl.hostname)) {
|
||||
return new Response(JSON.stringify({ error: 'Redirect to disallowed domain' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
throw new Error('Redirect to disallowed domain');
|
||||
}
|
||||
const redirectResponse = await fetchWithTimeout(redirectUrl.href, {
|
||||
return fetchWithTimeout(redirectUrl.href, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'Accept': 'application/rss+xml, application/xml, text/xml, */*',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
},
|
||||
}, timeout);
|
||||
const data = await redirectResponse.text();
|
||||
return new Response(data, {
|
||||
status: redirectResponse.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/xml',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
} catch {
|
||||
return new Response(JSON.stringify({ error: 'Invalid redirect' }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return response;
|
||||
};
|
||||
|
||||
let response;
|
||||
let usedRelay = false;
|
||||
try {
|
||||
response = await fetchDirect();
|
||||
} catch (directError) {
|
||||
response = await fetchViaRailway(feedUrl, timeout);
|
||||
usedRelay = !!response;
|
||||
if (!response) throw directError;
|
||||
}
|
||||
|
||||
if (!response.ok && !usedRelay) {
|
||||
const relayResponse = await fetchViaRailway(feedUrl, timeout);
|
||||
if (relayResponse && relayResponse.ok) {
|
||||
response = relayResponse;
|
||||
}
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/xml',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'Content-Type': response.headers.get('content-type') || 'application/xml',
|
||||
'Cache-Control': response.headers.get('cache-control') || 'public, max-age=600, s-maxage=600, stale-while-revalidate=300',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,296 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
// Major tech services and their status page endpoints
|
||||
// Most use Statuspage.io which has a standard /api/v2/status.json endpoint
|
||||
const SERVICES = [
|
||||
// Cloud Providers
|
||||
{ id: 'aws', name: 'AWS', statusPage: 'https://health.aws.amazon.com/health/status', customParser: 'aws', category: 'cloud' },
|
||||
{ id: 'azure', name: 'Azure', statusPage: 'https://azure.status.microsoft/en-us/status/feed/', customParser: 'rss', category: 'cloud' },
|
||||
{ id: 'gcp', name: 'Google Cloud', statusPage: 'https://status.cloud.google.com/incidents.json', customParser: 'gcp', category: 'cloud' },
|
||||
{ id: 'cloudflare', name: 'Cloudflare', statusPage: 'https://www.cloudflarestatus.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'vercel', name: 'Vercel', statusPage: 'https://www.vercel-status.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'netlify', name: 'Netlify', statusPage: 'https://www.netlifystatus.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'digitalocean', name: 'DigitalOcean', statusPage: 'https://status.digitalocean.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'render', name: 'Render', statusPage: 'https://status.render.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'railway', name: 'Railway', statusPage: 'https://railway.instatus.com/summary.json', customParser: 'instatus', category: 'cloud' },
|
||||
|
||||
// Developer Tools
|
||||
{ id: 'github', name: 'GitHub', statusPage: 'https://www.githubstatus.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'gitlab', name: 'GitLab', statusPage: 'https://status.gitlab.com/1.0/status/5b36dc6502d06804c08349f7', customParser: 'statusio', category: 'dev' },
|
||||
{ id: 'npm', name: 'npm', statusPage: 'https://status.npmjs.org/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'docker', name: 'Docker Hub', statusPage: 'https://www.dockerstatus.com/1.0/status/533c6539221ae15e3f000031', customParser: 'statusio', category: 'dev' },
|
||||
{ id: 'bitbucket', name: 'Bitbucket', statusPage: 'https://bitbucket.status.atlassian.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'circleci', name: 'CircleCI', statusPage: 'https://status.circleci.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'jira', name: 'Jira', statusPage: 'https://jira-software.status.atlassian.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'confluence', name: 'Confluence', statusPage: 'https://confluence.status.atlassian.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'linear', name: 'Linear', statusPage: 'https://linearstatus.com/api/v2/status.json', customParser: 'incidentio', category: 'dev' },
|
||||
|
||||
// Communication
|
||||
{ id: 'slack', name: 'Slack', statusPage: 'https://slack-status.com/api/v2.0.0/current', customParser: 'slack', category: 'comm' },
|
||||
{ id: 'discord', name: 'Discord', statusPage: 'https://discordstatus.com/api/v2/status.json', category: 'comm' },
|
||||
{ id: 'zoom', name: 'Zoom', statusPage: 'https://www.zoomstatus.com/api/v2/status.json', category: 'comm' },
|
||||
{ id: 'notion', name: 'Notion', statusPage: 'https://www.notion-status.com/api/v2/status.json', category: 'comm' },
|
||||
|
||||
// AI Services (incident.io powered)
|
||||
{ id: 'openai', name: 'OpenAI', statusPage: 'https://status.openai.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
|
||||
{ id: 'anthropic', name: 'Anthropic', statusPage: 'https://status.claude.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
|
||||
{ id: 'replicate', name: 'Replicate', statusPage: 'https://www.replicatestatus.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
|
||||
|
||||
// SaaS
|
||||
{ id: 'stripe', name: 'Stripe', statusPage: 'https://status.stripe.com/current', customParser: 'stripe', category: 'saas' },
|
||||
{ id: 'twilio', name: 'Twilio', statusPage: 'https://status.twilio.com/api/v2/status.json', category: 'saas' },
|
||||
{ id: 'datadog', name: 'Datadog', statusPage: 'https://status.datadoghq.com/api/v2/status.json', category: 'saas' },
|
||||
{ id: 'sentry', name: 'Sentry', statusPage: 'https://status.sentry.io/api/v2/status.json', category: 'saas' },
|
||||
{ id: 'supabase', name: 'Supabase', statusPage: 'https://status.supabase.com/api/v2/status.json', category: 'saas' },
|
||||
];
|
||||
|
||||
// Statuspage.io API returns status like: none, minor, major, critical
|
||||
function normalizeStatus(indicator) {
|
||||
if (!indicator) return 'unknown';
|
||||
const val = indicator.toLowerCase();
|
||||
// Check for operational indicators
|
||||
if (val === 'none' || val === 'operational' || val.includes('all systems operational')) {
|
||||
return 'operational';
|
||||
}
|
||||
// Check for degraded indicators
|
||||
if (val === 'minor' || val === 'degraded_performance' || val === 'partial_outage' || val.includes('degraded')) {
|
||||
return 'degraded';
|
||||
}
|
||||
// Check for outage indicators
|
||||
if (val === 'major' || val === 'major_outage' || val === 'critical' || val.includes('outage')) {
|
||||
return 'outage';
|
||||
}
|
||||
return 'unknown';
|
||||
}
|
||||
|
||||
async function checkStatusPage(service) {
|
||||
if (!service.statusPage) {
|
||||
return { ...service, status: 'unknown', description: 'No API available' };
|
||||
}
|
||||
|
||||
try {
|
||||
// Use browser-like headers to avoid being blocked
|
||||
const headers = {
|
||||
'Accept': service.customParser === 'rss' ? 'application/xml, text/xml' : 'application/json, text/plain, */*',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
'Cache-Control': 'no-cache',
|
||||
};
|
||||
// Don't send User-Agent for incident.io - they may block bots
|
||||
if (service.customParser !== 'incidentio') {
|
||||
headers['User-Agent'] = 'Mozilla/5.0 (compatible; WorldMonitor/1.0)';
|
||||
}
|
||||
|
||||
const response = await fetch(service.statusPage, {
|
||||
headers,
|
||||
signal: AbortSignal.timeout(10000),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return { ...service, status: 'unknown', description: `HTTP ${response.status}` };
|
||||
}
|
||||
|
||||
// Handle custom parsers
|
||||
if (service.customParser === 'gcp') {
|
||||
const data = await response.json();
|
||||
// GCP incidents.json returns array of incidents
|
||||
const activeIncidents = Array.isArray(data) ? data.filter(i =>
|
||||
i.end === undefined || new Date(i.end) > new Date()
|
||||
) : [];
|
||||
if (activeIncidents.length === 0) {
|
||||
return { ...service, status: 'operational', description: 'All services operational' };
|
||||
}
|
||||
const severity = activeIncidents.some(i => i.severity === 'high') ? 'outage' : 'degraded';
|
||||
return { ...service, status: severity, description: `${activeIncidents.length} active incident(s)` };
|
||||
}
|
||||
|
||||
if (service.customParser === 'aws') {
|
||||
// AWS status page is complex HTML - assume operational if reachable
|
||||
return { ...service, status: 'operational', description: 'Status page reachable' };
|
||||
}
|
||||
|
||||
if (service.customParser === 'rss') {
|
||||
// Azure RSS feed - check if there are recent items (incidents)
|
||||
const text = await response.text();
|
||||
const hasRecentIncident = text.includes('<item>') &&
|
||||
(text.includes('degradation') || text.includes('outage') || text.includes('incident'));
|
||||
return {
|
||||
...service,
|
||||
status: hasRecentIncident ? 'degraded' : 'operational',
|
||||
description: hasRecentIncident ? 'Recent incidents reported' : 'No recent incidents'
|
||||
};
|
||||
}
|
||||
|
||||
if (service.customParser === 'instatus') {
|
||||
// Instatus format (Railway, etc.)
|
||||
const data = await response.json();
|
||||
const pageStatus = data.page?.status;
|
||||
if (pageStatus === 'UP') {
|
||||
return { ...service, status: 'operational', description: 'All systems operational' };
|
||||
} else if (pageStatus === 'HASISSUES') {
|
||||
return { ...service, status: 'degraded', description: 'Some issues reported' };
|
||||
} else {
|
||||
return { ...service, status: 'unknown', description: pageStatus || 'Unknown' };
|
||||
}
|
||||
}
|
||||
|
||||
if (service.customParser === 'statusio') {
|
||||
// Status.io format (GitLab, Docker Hub)
|
||||
const data = await response.json();
|
||||
const overall = data.result?.status_overall;
|
||||
const statusCode = overall?.status_code;
|
||||
if (statusCode === 100) {
|
||||
return { ...service, status: 'operational', description: overall.status || 'All systems operational' };
|
||||
} else if (statusCode >= 300 && statusCode < 500) {
|
||||
return { ...service, status: 'degraded', description: overall.status || 'Degraded performance' };
|
||||
} else if (statusCode >= 500) {
|
||||
return { ...service, status: 'outage', description: overall.status || 'Service disruption' };
|
||||
}
|
||||
return { ...service, status: 'unknown', description: overall?.status || 'Unknown status' };
|
||||
}
|
||||
|
||||
if (service.customParser === 'slack') {
|
||||
// Slack custom API format
|
||||
const data = await response.json();
|
||||
if (data.status === 'ok') {
|
||||
return { ...service, status: 'operational', description: 'All systems operational' };
|
||||
} else if (data.status === 'active' || data.active_incidents?.length > 0) {
|
||||
const count = data.active_incidents?.length || 1;
|
||||
return { ...service, status: 'degraded', description: `${count} active incident(s)` };
|
||||
}
|
||||
return { ...service, status: 'unknown', description: data.status || 'Unknown' };
|
||||
}
|
||||
|
||||
if (service.customParser === 'stripe') {
|
||||
// Stripe custom API format at /current
|
||||
const data = await response.json();
|
||||
if (data.largestatus === 'up') {
|
||||
return { ...service, status: 'operational', description: data.message || 'All systems operational' };
|
||||
} else if (data.largestatus === 'degraded') {
|
||||
return { ...service, status: 'degraded', description: data.message || 'Degraded performance' };
|
||||
} else if (data.largestatus === 'down') {
|
||||
return { ...service, status: 'outage', description: data.message || 'Service disruption' };
|
||||
}
|
||||
return { ...service, status: 'unknown', description: data.message || 'Unknown' };
|
||||
}
|
||||
|
||||
if (service.customParser === 'incidentio') {
|
||||
// incident.io status pages (OpenAI, Linear, Replicate, Anthropic)
|
||||
const text = await response.text();
|
||||
// Check for HTML response (blocked)
|
||||
if (text.startsWith('<!') || text.startsWith('<html')) {
|
||||
// Try parsing HTML for status - incident.io pages have status in HTML
|
||||
const operationalMatch = text.match(/All Systems Operational|fully operational|no issues/i);
|
||||
if (operationalMatch) {
|
||||
return { ...service, status: 'operational', description: 'All systems operational' };
|
||||
}
|
||||
const degradedMatch = text.match(/degraded|partial outage|experiencing issues/i);
|
||||
if (degradedMatch) {
|
||||
return { ...service, status: 'degraded', description: 'Some issues reported' };
|
||||
}
|
||||
return { ...service, status: 'unknown', description: 'Could not parse status' };
|
||||
}
|
||||
// Parse JSON response
|
||||
try {
|
||||
const data = JSON.parse(text);
|
||||
const indicator = data.status?.indicator || '';
|
||||
const description = data.status?.description || '';
|
||||
if (indicator === 'none' || description.toLowerCase().includes('operational')) {
|
||||
return { ...service, status: 'operational', description: description || 'All systems operational' };
|
||||
} else if (indicator === 'minor' || indicator === 'maintenance') {
|
||||
return { ...service, status: 'degraded', description: description || 'Minor issues' };
|
||||
} else if (indicator === 'major' || indicator === 'critical') {
|
||||
return { ...service, status: 'outage', description: description || 'Major outage' };
|
||||
}
|
||||
return { ...service, status: 'operational', description: description || 'Status OK' };
|
||||
} catch {
|
||||
return { ...service, status: 'unknown', description: 'Invalid response' };
|
||||
}
|
||||
}
|
||||
|
||||
const text = await response.text();
|
||||
|
||||
// Check if we got HTML instead of JSON (blocked/redirected)
|
||||
if (text.startsWith('<!') || text.startsWith('<html')) {
|
||||
return { ...service, status: 'unknown', description: 'Blocked by service' };
|
||||
}
|
||||
|
||||
let data;
|
||||
try {
|
||||
data = JSON.parse(text);
|
||||
} catch {
|
||||
return { ...service, status: 'unknown', description: 'Invalid JSON response' };
|
||||
}
|
||||
|
||||
// Handle different API formats
|
||||
let status, description;
|
||||
|
||||
if (data.status?.indicator !== undefined) {
|
||||
// Standard Statuspage.io format
|
||||
status = normalizeStatus(data.status.indicator);
|
||||
description = data.status.description || '';
|
||||
} else if (data.status?.status) {
|
||||
// Slack format
|
||||
status = data.status.status === 'ok' ? 'operational' : 'degraded';
|
||||
description = data.status.description || '';
|
||||
} else if (data.page && data.status) {
|
||||
// Alternative Statuspage format - check if status object exists
|
||||
status = normalizeStatus(data.status.indicator || data.status.description);
|
||||
description = data.status.description || 'Status available';
|
||||
} else {
|
||||
status = 'unknown';
|
||||
description = 'Unknown format';
|
||||
}
|
||||
|
||||
return { ...service, status, description };
|
||||
} catch (error) {
|
||||
return { ...service, status: 'unknown', description: error.message || 'Request failed' };
|
||||
}
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const category = url.searchParams.get('category'); // cloud, dev, comm, ai, saas, or all
|
||||
|
||||
let servicesToCheck = SERVICES;
|
||||
if (category && category !== 'all') {
|
||||
servicesToCheck = SERVICES.filter(s => s.category === category);
|
||||
}
|
||||
|
||||
// Check all services in parallel
|
||||
const results = await Promise.all(servicesToCheck.map(checkStatusPage));
|
||||
|
||||
// Sort by status (outages first, then degraded, then operational)
|
||||
const statusOrder = { outage: 0, degraded: 1, unknown: 2, operational: 3 };
|
||||
results.sort((a, b) => statusOrder[a.status] - statusOrder[b.status]);
|
||||
|
||||
const summary = {
|
||||
operational: results.filter(r => r.status === 'operational').length,
|
||||
degraded: results.filter(r => r.status === 'degraded').length,
|
||||
outage: results.filter(r => r.status === 'outage').length,
|
||||
unknown: results.filter(r => r.status === 'unknown').length,
|
||||
};
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
success: true,
|
||||
timestamp: new Date().toISOString(),
|
||||
summary,
|
||||
services: results.map(r => ({
|
||||
id: r.id,
|
||||
name: r.name,
|
||||
category: r.category,
|
||||
status: r.status,
|
||||
description: r.description,
|
||||
})),
|
||||
}), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30', // 1 min cache
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,130 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_TTL = 120;
|
||||
let cachedResponse = null;
|
||||
let cacheTimestamp = 0;
|
||||
|
||||
const DEFAULT_COINS = 'tether,usd-coin,dai,first-digital-usd,ethena-usde';
|
||||
|
||||
function buildFallbackResult() {
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
summary: {
|
||||
totalMarketCap: 0,
|
||||
totalVolume24h: 0,
|
||||
coinCount: 0,
|
||||
depeggedCount: 0,
|
||||
healthStatus: 'UNAVAILABLE',
|
||||
},
|
||||
stablecoins: [],
|
||||
unavailable: true,
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: cors });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
|
||||
return new Response(JSON.stringify(cachedResponse), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=300` },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const rawCoins = url.searchParams.get('coins') || DEFAULT_COINS;
|
||||
const coins = rawCoins.split(',').filter(c => /^[a-z0-9-]+$/.test(c)).join(',') || DEFAULT_COINS;
|
||||
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const id = setTimeout(() => controller.abort(), 10000);
|
||||
|
||||
const apiUrl = `https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&ids=${coins}&order=market_cap_desc&sparkline=false&price_change_percentage=7d`;
|
||||
const res = await fetch(apiUrl, {
|
||||
signal: controller.signal,
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
clearTimeout(id);
|
||||
|
||||
if (res.status === 429) {
|
||||
if (cachedResponse) {
|
||||
return new Response(JSON.stringify(cachedResponse), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
|
||||
});
|
||||
}
|
||||
return new Response(JSON.stringify({ error: 'Rate limited', timestamp: new Date().toISOString() }), {
|
||||
status: 429,
|
||||
headers: { ...cors, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (!res.ok) throw new Error(`CoinGecko HTTP ${res.status}`);
|
||||
|
||||
const data = await res.json();
|
||||
|
||||
const stablecoins = data.map(coin => {
|
||||
const price = coin.current_price || 0;
|
||||
const deviation = Math.abs(price - 1.0);
|
||||
let pegStatus;
|
||||
if (deviation <= 0.005) pegStatus = 'ON PEG';
|
||||
else if (deviation <= 0.01) pegStatus = 'SLIGHT DEPEG';
|
||||
else pegStatus = 'DEPEGGED';
|
||||
|
||||
return {
|
||||
id: coin.id,
|
||||
symbol: (coin.symbol || '').toUpperCase(),
|
||||
name: coin.name,
|
||||
price,
|
||||
deviation: +(deviation * 100).toFixed(3),
|
||||
pegStatus,
|
||||
marketCap: coin.market_cap || 0,
|
||||
volume24h: coin.total_volume || 0,
|
||||
change24h: coin.price_change_percentage_24h || 0,
|
||||
change7d: coin.price_change_percentage_7d_in_currency || 0,
|
||||
image: coin.image,
|
||||
};
|
||||
});
|
||||
|
||||
const totalMarketCap = stablecoins.reduce((sum, c) => sum + c.marketCap, 0);
|
||||
const totalVolume24h = stablecoins.reduce((sum, c) => sum + c.volume24h, 0);
|
||||
const depeggedCount = stablecoins.filter(c => c.pegStatus === 'DEPEGGED').length;
|
||||
|
||||
const result = {
|
||||
timestamp: new Date().toISOString(),
|
||||
summary: {
|
||||
totalMarketCap,
|
||||
totalVolume24h,
|
||||
coinCount: stablecoins.length,
|
||||
depeggedCount,
|
||||
healthStatus: depeggedCount === 0 ? 'HEALTHY' : depeggedCount === 1 ? 'CAUTION' : 'WARNING',
|
||||
},
|
||||
stablecoins,
|
||||
};
|
||||
|
||||
cachedResponse = result;
|
||||
cacheTimestamp = now;
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=300` },
|
||||
});
|
||||
} catch (err) {
|
||||
const fallback = cachedResponse || buildFallbackResult();
|
||||
cachedResponse = fallback;
|
||||
cacheTimestamp = now;
|
||||
return new Response(JSON.stringify(fallback), {
|
||||
status: 200,
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
/**
|
||||
* Stock Market Index Endpoint
|
||||
* Fetches weekly % change for a country's primary stock index via Yahoo Finance
|
||||
* Redis cached (1h TTL)
|
||||
*/
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const CACHE_TTL_SECONDS = 3600; // 1 hour
|
||||
const CACHE_VERSION = 'stock-v1';
|
||||
|
||||
const COUNTRY_INDEX = {
|
||||
US: { symbol: '^GSPC', name: 'S&P 500' },
|
||||
GB: { symbol: '^FTSE', name: 'FTSE 100' },
|
||||
DE: { symbol: '^GDAXI', name: 'DAX' },
|
||||
FR: { symbol: '^FCHI', name: 'CAC 40' },
|
||||
JP: { symbol: '^N225', name: 'Nikkei 225' },
|
||||
CN: { symbol: '000001.SS', name: 'SSE Composite' },
|
||||
HK: { symbol: '^HSI', name: 'Hang Seng' },
|
||||
IN: { symbol: '^BSESN', name: 'BSE Sensex' },
|
||||
KR: { symbol: '^KS11', name: 'KOSPI' },
|
||||
TW: { symbol: '^TWII', name: 'TAIEX' },
|
||||
AU: { symbol: '^AXJO', name: 'ASX 200' },
|
||||
BR: { symbol: '^BVSP', name: 'Bovespa' },
|
||||
CA: { symbol: '^GSPTSE', name: 'TSX Composite' },
|
||||
MX: { symbol: '^MXX', name: 'IPC Mexico' },
|
||||
AR: { symbol: '^MERV', name: 'MERVAL' },
|
||||
RU: { symbol: 'IMOEX.ME', name: 'MOEX' },
|
||||
ZA: { symbol: '^J203.JO', name: 'JSE All Share' },
|
||||
SA: { symbol: '^TASI.SR', name: 'Tadawul' },
|
||||
AE: { symbol: 'DFMGI.AE', name: 'DFM General' },
|
||||
IL: { symbol: '^TA125.TA', name: 'TA-125' },
|
||||
TR: { symbol: 'XU100.IS', name: 'BIST 100' },
|
||||
PL: { symbol: '^WIG20', name: 'WIG 20' },
|
||||
NL: { symbol: '^AEX', name: 'AEX' },
|
||||
CH: { symbol: '^SSMI', name: 'SMI' },
|
||||
ES: { symbol: '^IBEX', name: 'IBEX 35' },
|
||||
IT: { symbol: 'FTSEMIB.MI', name: 'FTSE MIB' },
|
||||
SE: { symbol: '^OMX', name: 'OMX Stockholm 30' },
|
||||
NO: { symbol: '^OSEAX', name: 'Oslo All Share' },
|
||||
SG: { symbol: '^STI', name: 'STI' },
|
||||
TH: { symbol: '^SET.BK', name: 'SET' },
|
||||
MY: { symbol: '^KLSE', name: 'KLCI' },
|
||||
ID: { symbol: '^JKSE', name: 'Jakarta Composite' },
|
||||
PH: { symbol: 'PSEI.PS', name: 'PSEi' },
|
||||
NZ: { symbol: '^NZ50', name: 'NZX 50' },
|
||||
EG: { symbol: '^EGX30.CA', name: 'EGX 30' },
|
||||
CL: { symbol: '^IPSA', name: 'IPSA' },
|
||||
PE: { symbol: '^SPBLPGPT', name: 'S&P Lima' },
|
||||
AT: { symbol: '^ATX', name: 'ATX' },
|
||||
BE: { symbol: '^BFX', name: 'BEL 20' },
|
||||
FI: { symbol: '^OMXH25', name: 'OMX Helsinki 25' },
|
||||
DK: { symbol: '^OMXC25', name: 'OMX Copenhagen 25' },
|
||||
IE: { symbol: '^ISEQ', name: 'ISEQ Overall' },
|
||||
PT: { symbol: '^PSI20', name: 'PSI 20' },
|
||||
CZ: { symbol: '^PX', name: 'PX Prague' },
|
||||
HU: { symbol: '^BUX', name: 'BUX' },
|
||||
};
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (request.method === 'OPTIONS') return new Response(null, { status: 204, headers: cors });
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
if (request.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const code = (url.searchParams.get('code') || '').toUpperCase();
|
||||
|
||||
if (!code) {
|
||||
return new Response(JSON.stringify({ error: 'code parameter required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const index = COUNTRY_INDEX[code];
|
||||
if (!index) {
|
||||
return new Response(JSON.stringify({ error: 'No stock index for country', code, available: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const cacheKey = `${CACHE_VERSION}:${code}`;
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.indexName) {
|
||||
return new Response(JSON.stringify({ ...cached, cached: true }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const encodedSymbol = encodeURIComponent(index.symbol);
|
||||
// Use 1mo range to handle markets with different trading weeks (e.g. Sun-Thu Middle East)
|
||||
const yahooUrl = `https://query1.finance.yahoo.com/v8/finance/chart/${encodedSymbol}?range=1mo&interval=1d`;
|
||||
|
||||
const res = await fetch(yahooUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)',
|
||||
},
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
console.error('[StockIndex] Yahoo error:', res.status, index.symbol);
|
||||
return new Response(JSON.stringify({ error: 'Upstream error', available: false }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
const result = data?.chart?.result?.[0];
|
||||
if (!result) {
|
||||
return new Response(JSON.stringify({ error: 'No data', available: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const allCloses = result.indicators?.quote?.[0]?.close?.filter(v => v != null);
|
||||
if (!allCloses || allCloses.length < 2) {
|
||||
return new Response(JSON.stringify({ error: 'Insufficient data', available: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
// Take last ~5 trading days worth of data
|
||||
const closes = allCloses.slice(-6);
|
||||
const latest = closes[closes.length - 1];
|
||||
const oldest = closes[0];
|
||||
const weekChange = ((latest - oldest) / oldest) * 100;
|
||||
const meta = result.meta || {};
|
||||
|
||||
const payload = {
|
||||
available: true,
|
||||
code,
|
||||
symbol: index.symbol,
|
||||
indexName: index.name,
|
||||
price: latest.toFixed(2),
|
||||
weekChangePercent: weekChange.toFixed(2),
|
||||
currency: meta.currency || 'USD',
|
||||
fetchedAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
await setCachedJson(cacheKey, payload, CACHE_TTL_SECONDS);
|
||||
|
||||
return new Response(JSON.stringify(payload), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[StockIndex] Error:', err);
|
||||
return new Response(JSON.stringify({ error: 'Internal error', available: false }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
/**
|
||||
* Story Page for Social Crawlers
|
||||
* Returns HTML with proper og:image and twitter:card meta tags.
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
// Telegram feed proxy (web)
|
||||
// Fetches Telegram Early Signals from the Railway relay (stateful MTProto lives there).
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
async function fetchWithTimeout(url, options, timeoutMs = 25000) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
|
||||
let relay = process.env.WS_RELAY_URL;
|
||||
if (!relay) {
|
||||
return new Response(JSON.stringify({ error: 'WS_RELAY_URL not configured' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
// Guard: WS_RELAY_URL should be HTTP(S) for server-side fetches.
|
||||
// If someone accidentally sets a ws:// or wss:// URL, normalize it.
|
||||
if (relay.startsWith('wss://')) relay = relay.replace('wss://', 'https://');
|
||||
if (relay.startsWith('ws://')) relay = relay.replace('ws://', 'http://');
|
||||
|
||||
const url = new URL(req.url);
|
||||
const limit = Math.max(1, Math.min(200, parseInt(url.searchParams.get('limit') || '50', 10) || 50));
|
||||
const topic = (url.searchParams.get('topic') || '').trim();
|
||||
const channel = (url.searchParams.get('channel') || '').trim();
|
||||
|
||||
const relayUrl = new URL('/telegram/feed', relay);
|
||||
relayUrl.searchParams.set('limit', String(limit));
|
||||
if (topic) relayUrl.searchParams.set('topic', topic);
|
||||
if (channel) relayUrl.searchParams.set('channel', channel);
|
||||
|
||||
try {
|
||||
const res = await fetchWithTimeout(relayUrl.toString(), {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
}, 25000);
|
||||
|
||||
const text = await res.text();
|
||||
return new Response(text, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
'Content-Type': res.headers.get('content-type') || 'application/json',
|
||||
// Short cache. Telegram is near-real-time.
|
||||
'Cache-Control': 'public, max-age=10',
|
||||
...cors,
|
||||
},
|
||||
});
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
const isAbort = err && (err.name === 'AbortError' || /aborted/i.test(msg));
|
||||
return new Response(JSON.stringify({
|
||||
error: isAbort ? 'Telegram relay request timed out' : 'Telegram relay fetch failed',
|
||||
}), {
|
||||
status: isAbort ? 504 : 502,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,176 +0,0 @@
|
||||
/**
|
||||
* Temporal Baseline Anomaly Detection API
|
||||
* Stores and queries activity baselines using Welford's online algorithm
|
||||
* Backed by Upstash Redis for cross-user persistence
|
||||
*
|
||||
* GET ?type=military_flights®ion=global&count=47 — check anomaly
|
||||
* POST { updates: [{ type, region, count }] } — batch update baselines
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson, mget } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const BASELINE_TTL = 7776000; // 90 days in seconds
|
||||
const MIN_SAMPLES = 10;
|
||||
const Z_THRESHOLD_LOW = 1.5;
|
||||
const Z_THRESHOLD_MEDIUM = 2.0;
|
||||
const Z_THRESHOLD_HIGH = 3.0;
|
||||
|
||||
const VALID_TYPES = ['military_flights', 'vessels', 'protests', 'news', 'ais_gaps', 'satellite_fires'];
|
||||
|
||||
function makeKey(type, region, weekday, month) {
|
||||
return `baseline:${type}:${region}:${weekday}:${month}`;
|
||||
}
|
||||
|
||||
function getSeverity(zScore) {
|
||||
if (zScore >= Z_THRESHOLD_HIGH) return 'critical';
|
||||
if (zScore >= Z_THRESHOLD_MEDIUM) return 'high';
|
||||
if (zScore >= Z_THRESHOLD_LOW) return 'medium';
|
||||
return 'normal';
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'GET, POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
if (request.method === 'GET') {
|
||||
return await handleGet(request);
|
||||
} else if (request.method === 'POST') {
|
||||
return await handlePost(request);
|
||||
}
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[TemporalBaseline] Error:', err);
|
||||
return new Response(JSON.stringify({ error: 'Internal error' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async function handleGet(request) {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const type = searchParams.get('type');
|
||||
const region = searchParams.get('region') || 'global';
|
||||
const count = parseFloat(searchParams.get('count'));
|
||||
|
||||
if (!type || !VALID_TYPES.includes(type) || isNaN(count)) {
|
||||
return json({ error: 'Missing or invalid params: type, count required' }, 400);
|
||||
}
|
||||
|
||||
const now = new Date();
|
||||
const weekday = now.getUTCDay();
|
||||
const month = now.getUTCMonth() + 1;
|
||||
const key = makeKey(type, region, weekday, month);
|
||||
|
||||
const baseline = await getCachedJson(key);
|
||||
|
||||
if (!baseline || baseline.sampleCount < MIN_SAMPLES) {
|
||||
return json({
|
||||
anomaly: null,
|
||||
learning: true,
|
||||
sampleCount: baseline?.sampleCount || 0,
|
||||
samplesNeeded: MIN_SAMPLES,
|
||||
});
|
||||
}
|
||||
|
||||
const variance = Math.max(0, baseline.m2 / (baseline.sampleCount - 1));
|
||||
const stdDev = Math.sqrt(variance);
|
||||
const zScore = stdDev > 0 ? Math.abs((count - baseline.mean) / stdDev) : 0;
|
||||
const severity = getSeverity(zScore);
|
||||
const multiplier = baseline.mean > 0
|
||||
? Math.round((count / baseline.mean) * 100) / 100
|
||||
: count > 0 ? 999 : 1;
|
||||
|
||||
return json({
|
||||
anomaly: zScore >= Z_THRESHOLD_LOW ? {
|
||||
zScore: Math.round(zScore * 100) / 100,
|
||||
severity,
|
||||
multiplier,
|
||||
} : null,
|
||||
baseline: {
|
||||
mean: Math.round(baseline.mean * 100) / 100,
|
||||
stdDev: Math.round(stdDev * 100) / 100,
|
||||
sampleCount: baseline.sampleCount,
|
||||
},
|
||||
learning: false,
|
||||
});
|
||||
}
|
||||
|
||||
async function handlePost(request) {
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return json({ error: 'Payload too large' }, 413);
|
||||
}
|
||||
|
||||
const body = await request.json();
|
||||
const updates = body?.updates;
|
||||
|
||||
if (!Array.isArray(updates) || updates.length === 0) {
|
||||
return json({ error: 'Body must have updates array' }, 400);
|
||||
}
|
||||
|
||||
const batch = updates.slice(0, 20);
|
||||
const now = new Date();
|
||||
const weekday = now.getUTCDay();
|
||||
const month = now.getUTCMonth() + 1;
|
||||
|
||||
const keys = batch.map(u => makeKey(u.type, u.region || 'global', weekday, month));
|
||||
const existing = await mget(...keys);
|
||||
|
||||
const writes = [];
|
||||
|
||||
for (let i = 0; i < batch.length; i++) {
|
||||
const { type, region = 'global', count } = batch[i];
|
||||
if (!VALID_TYPES.includes(type) || typeof count !== 'number' || isNaN(count)) continue;
|
||||
|
||||
const prev = existing[i] || { mean: 0, m2: 0, sampleCount: 0 };
|
||||
|
||||
const n = prev.sampleCount + 1;
|
||||
const delta = count - prev.mean;
|
||||
const newMean = prev.mean + delta / n;
|
||||
const delta2 = count - newMean;
|
||||
const newM2 = prev.m2 + delta * delta2;
|
||||
|
||||
writes.push(setCachedJson(keys[i], {
|
||||
mean: newMean,
|
||||
m2: newM2,
|
||||
sampleCount: n,
|
||||
lastUpdated: now.toISOString(),
|
||||
}, BASELINE_TTL));
|
||||
}
|
||||
|
||||
if (writes.length > 0) {
|
||||
await Promise.all(writes);
|
||||
}
|
||||
|
||||
return json({ updated: writes.length });
|
||||
}
|
||||
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,611 +0,0 @@
|
||||
/**
|
||||
* Theater Posture API - Aggregates military aircraft by theater
|
||||
* Caches results in Upstash Redis for cross-user efficiency
|
||||
* TTL: 5 minutes (matches OpenSky refresh rate)
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const CACHE_TTL_SECONDS = 300; // 5 minutes
|
||||
const STALE_CACHE_TTL_SECONDS = 86400; // 24 hours - serve stale data when API is down
|
||||
const BACKUP_CACHE_TTL_SECONDS = 604800; // 7 days - last resort backup
|
||||
const CACHE_KEY = 'theater-posture:v4';
|
||||
const STALE_CACHE_KEY = 'theater-posture:stale:v4';
|
||||
const BACKUP_CACHE_KEY = 'theater-posture:backup:v4';
|
||||
|
||||
// Theater definitions (matches client-side POSTURE_THEATERS)
|
||||
const POSTURE_THEATERS = [
|
||||
{
|
||||
id: 'iran-theater',
|
||||
name: 'Iran Theater',
|
||||
shortName: 'IRAN',
|
||||
targetNation: 'Iran',
|
||||
bounds: { north: 42, south: 20, east: 65, west: 30 },
|
||||
thresholds: { elevated: 8, critical: 20 },
|
||||
strikeIndicators: { minTankers: 2, minAwacs: 1, minFighters: 5 },
|
||||
},
|
||||
{
|
||||
id: 'taiwan-theater',
|
||||
name: 'Taiwan Strait',
|
||||
shortName: 'TAIWAN',
|
||||
targetNation: 'Taiwan',
|
||||
bounds: { north: 30, south: 18, east: 130, west: 115 },
|
||||
thresholds: { elevated: 6, critical: 15 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 4 },
|
||||
},
|
||||
{
|
||||
id: 'baltic-theater',
|
||||
name: 'Baltic Theater',
|
||||
shortName: 'BALTIC',
|
||||
targetNation: null,
|
||||
bounds: { north: 65, south: 52, east: 32, west: 10 },
|
||||
thresholds: { elevated: 5, critical: 12 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'blacksea-theater',
|
||||
name: 'Black Sea',
|
||||
shortName: 'BLACK SEA',
|
||||
targetNation: null,
|
||||
bounds: { north: 48, south: 40, east: 42, west: 26 },
|
||||
thresholds: { elevated: 4, critical: 10 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'korea-theater',
|
||||
name: 'Korean Peninsula',
|
||||
shortName: 'KOREA',
|
||||
targetNation: 'North Korea',
|
||||
bounds: { north: 43, south: 33, east: 132, west: 124 },
|
||||
thresholds: { elevated: 5, critical: 12 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'south-china-sea',
|
||||
name: 'South China Sea',
|
||||
shortName: 'SCS',
|
||||
targetNation: null,
|
||||
bounds: { north: 25, south: 5, east: 121, west: 105 },
|
||||
thresholds: { elevated: 6, critical: 15 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 4 },
|
||||
},
|
||||
{
|
||||
id: 'east-med-theater',
|
||||
name: 'Eastern Mediterranean',
|
||||
shortName: 'E.MED',
|
||||
targetNation: null,
|
||||
bounds: { north: 37, south: 33, east: 37, west: 25 },
|
||||
thresholds: { elevated: 4, critical: 10 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'israel-gaza-theater',
|
||||
name: 'Israel/Gaza',
|
||||
shortName: 'GAZA',
|
||||
targetNation: 'Gaza',
|
||||
bounds: { north: 33, south: 29, east: 36, west: 33 },
|
||||
thresholds: { elevated: 3, critical: 8 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'yemen-redsea-theater',
|
||||
name: 'Yemen/Red Sea',
|
||||
shortName: 'RED SEA',
|
||||
targetNation: 'Yemen',
|
||||
bounds: { north: 22, south: 11, east: 54, west: 32 },
|
||||
thresholds: { elevated: 4, critical: 10 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
];
|
||||
|
||||
// Military hex database from ADS-B Exchange (updated daily at adsbexchange.com)
|
||||
// Contains ~20k verified military aircraft hex IDs
|
||||
import { MILITARY_HEX_LIST } from './data/military-hex-db.js';
|
||||
|
||||
// Create Set for O(1) lookup
|
||||
const MILITARY_HEX_SET = new Set(MILITARY_HEX_LIST.map(h => h.toLowerCase()));
|
||||
console.log(`[TheaterPosture] Loaded ${MILITARY_HEX_SET.size} military hex IDs from ADS-B Exchange`);
|
||||
|
||||
// Check if ICAO hex is in military database
|
||||
function isMilitaryHex(hexId) {
|
||||
if (!hexId) return false;
|
||||
// Handle both string and number, remove ~ prefix if present
|
||||
const cleanHex = String(hexId).replace(/^~/, '').toLowerCase();
|
||||
return MILITARY_HEX_SET.has(cleanHex);
|
||||
}
|
||||
|
||||
// Military callsign prefixes for identification
|
||||
const MILITARY_PREFIXES = [
|
||||
// US Military
|
||||
'RCH', 'REACH', 'MOOSE', 'EVAC', 'DUSTOFF', 'PEDRO', // Transport/medevac
|
||||
'DUKE', 'HAVOC', 'KNIFE', 'WARHAWK', 'VIPER', 'RAGE', 'FURY', // Fighters
|
||||
'SHELL', 'TEXACO', 'ARCO', 'ESSO', 'PETRO', // Tankers
|
||||
'SENTRY', 'AWACS', 'MAGIC', 'DISCO', 'DARKSTAR', // AWACS/ISR
|
||||
'COBRA', 'PYTHON', 'RAPTOR', 'EAGLE', 'HAWK', 'TALON', // Various
|
||||
'BOXER', 'OMNI', 'TOPCAT', 'SKULL', 'REAPER', 'HUNTER', // More callsigns
|
||||
'ARMY', 'NAVY', 'USAF', 'USMC', 'USCG', // Service prefixes
|
||||
'AE', 'CNV', 'PAT', 'SAM', 'EXEC', // Special missions
|
||||
'OPS', 'CTF', 'TF', // Operations/Task Force
|
||||
// NATO
|
||||
'NATO', 'GAF', 'RRF', 'RAF', 'FAF', 'IAF', 'RNLAF', 'BAF', 'DAF', 'HAF', 'PAF',
|
||||
'SWORD', 'LANCE', 'ARROW', 'SPARTAN', // NATO tactical
|
||||
// Middle East (avoid UAE - conflicts with Emirates airline)
|
||||
'RSAF', 'EMIRI', 'UAEAF', 'KAF', 'QAF', 'BAHAF', 'OMAAF', // Gulf states
|
||||
'IRIAF', 'IRG', 'IRGC', // Iran (IAF already in NATO section covers Israel)
|
||||
'TAF', 'TUAF', // Turkey
|
||||
// Russia
|
||||
'RSD', 'RF', 'RFF', 'VKS',
|
||||
// China (NOTE: CCA is Air China airline, not military)
|
||||
'CHN', 'PLAAF', 'PLA',
|
||||
];
|
||||
|
||||
// Airline ICAO codes to exclude from military detection (Set for O(1) lookup)
|
||||
const AIRLINE_CODES = new Set([
|
||||
// Middle East
|
||||
'SVA', 'QTR', 'THY', 'UAE', 'ETD', 'GFA', 'MEA', 'RJA', 'KAC', 'ELY',
|
||||
'IAW', 'IRA', 'MSR', 'SYR', 'PGT', 'AXB', 'FDB', 'KNE', 'FAD', 'ADY', 'OMA',
|
||||
'ABQ', 'ABY', 'NIA', 'FJA', 'SWR', 'HZA', 'OMS', 'EGF', 'NOS', 'SXD',
|
||||
// Europe
|
||||
'BAW', 'AFR', 'DLH', 'KLM', 'AUA', 'SAS', 'FIN', 'LOT', 'AZA', 'TAP', 'IBE',
|
||||
'VLG', 'RYR', 'EZY', 'WZZ', 'NOZ', 'BEL', 'AEE', 'ROT',
|
||||
// Asia
|
||||
'AIC', 'CPA', 'SIA', 'MAS', 'THA', 'VNM', 'JAL', 'ANA', 'KAL', 'AAR', 'EVA',
|
||||
'CAL', 'CCA', 'CES', 'CSN', 'HDA', 'CHH', 'CXA', 'GIA', 'PAL', 'SLK',
|
||||
// Americas
|
||||
'AAL', 'DAL', 'UAL', 'SWA', 'JBU', 'FFT', 'ASA', 'NKS', 'WJA', 'ACA',
|
||||
// Cargo
|
||||
'FDX', 'UPS', 'GTI', 'ABW', 'CLX', 'MPH',
|
||||
// Generic
|
||||
'AIR', 'SKY', 'JET',
|
||||
]);
|
||||
|
||||
// Aircraft type detection from callsign patterns
|
||||
function detectAircraftType(callsign) {
|
||||
if (!callsign) return 'unknown';
|
||||
const cs = callsign.toUpperCase().trim();
|
||||
|
||||
// Tankers
|
||||
if (/^(SHELL|TEXACO|ARCO|ESSO|PETRO)/.test(cs)) return 'tanker';
|
||||
if (/^(KC|STRAT)/.test(cs)) return 'tanker';
|
||||
|
||||
// AWACS
|
||||
if (/^(SENTRY|AWACS|MAGIC|DISCO|DARKSTAR)/.test(cs)) return 'awacs';
|
||||
if (/^(E3|E8|E6)/.test(cs)) return 'awacs';
|
||||
|
||||
// Transport
|
||||
if (/^(RCH|REACH|MOOSE|EVAC|DUSTOFF)/.test(cs)) return 'transport';
|
||||
if (/^(C17|C5|C130|C40)/.test(cs)) return 'transport';
|
||||
|
||||
// Reconnaissance
|
||||
if (/^(HOMER|OLIVE|JAKE|PSEUDO|GORDO)/.test(cs)) return 'reconnaissance';
|
||||
if (/^(RC|U2|SR)/.test(cs)) return 'reconnaissance';
|
||||
|
||||
// Drones/UAVs
|
||||
if (/^(RQ|MQ|REAPER|PREDATOR|GLOBAL)/.test(cs)) return 'drone';
|
||||
|
||||
// Bombers
|
||||
if (/^(DEATH|BONE|DOOM)/.test(cs)) return 'bomber';
|
||||
if (/^(B52|B1|B2)/.test(cs)) return 'bomber';
|
||||
|
||||
// Default to unknown for unrecognized military aircraft
|
||||
return 'unknown';
|
||||
}
|
||||
|
||||
// Check if callsign is military
|
||||
function isMilitaryCallsign(callsign) {
|
||||
if (!callsign) return false;
|
||||
const cs = callsign.toUpperCase().trim();
|
||||
|
||||
// Check prefixes
|
||||
for (const prefix of MILITARY_PREFIXES) {
|
||||
if (cs.startsWith(prefix)) return true;
|
||||
}
|
||||
|
||||
// Check patterns - tactical callsigns (word + small number)
|
||||
// DUKE01, VIPER12, RAGE1 but NOT airline codes like PGT5873, IAW9011
|
||||
if (/^[A-Z]{4,}\d{1,3}$/.test(cs)) return true;
|
||||
|
||||
// Short tactical: 3 letters + 1-2 digits (but exclude common airlines)
|
||||
// This catches OPS4, CTF01, TF12 but blocks SVA12, QTR76, etc.
|
||||
if (/^[A-Z]{3}\d{1,2}$/.test(cs)) {
|
||||
const prefix = cs.slice(0, 3);
|
||||
if (!AIRLINE_CODES.has(prefix)) return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
// Fetch military flights from OpenSky
|
||||
async function fetchMilitaryFlights() {
|
||||
const isSidecar = (process.env.LOCAL_API_MODE || '').includes('sidecar');
|
||||
// Desktop sidecar: fetch directly from OpenSky (single user, no rate limit concern)
|
||||
// Cloud: use Railway relay to avoid OpenSky rate limits across many users
|
||||
const baseUrl = isSidecar
|
||||
? 'https://opensky-network.org/api/states/all'
|
||||
: (process.env.WS_RELAY_URL ? process.env.WS_RELAY_URL + '/opensky' : null);
|
||||
|
||||
if (!baseUrl) return [];
|
||||
|
||||
// Fetch global data with 20s timeout (Edge has 25s limit)
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 20000);
|
||||
|
||||
try {
|
||||
console.log('[TheaterPosture] Fetching from:', baseUrl);
|
||||
const response = await fetch(baseUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'Mozilla/5.0 WorldMonitor/1.0',
|
||||
},
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`OpenSky API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
if (!data.states) return [];
|
||||
|
||||
// Filter and transform to military flights
|
||||
const flights = [];
|
||||
for (const state of data.states) {
|
||||
const [icao24, callsign, , , , lon, lat, altitude, onGround, velocity, heading] = state;
|
||||
|
||||
// Skip if no position
|
||||
if (lat == null || lon == null) continue;
|
||||
|
||||
// Skip if on ground
|
||||
if (onGround) continue;
|
||||
|
||||
// Check if military (by callsign OR hex range)
|
||||
const isMilitary = isMilitaryCallsign(callsign) || isMilitaryHex(icao24);
|
||||
if (!isMilitary) continue;
|
||||
|
||||
flights.push({
|
||||
id: icao24,
|
||||
callsign: callsign?.trim() || '',
|
||||
lat,
|
||||
lon,
|
||||
altitude: altitude || 0,
|
||||
heading: heading || 0,
|
||||
speed: velocity || 0,
|
||||
aircraftType: detectAircraftType(callsign),
|
||||
operator: 'unknown',
|
||||
militaryHex: isMilitaryHex(icao24),
|
||||
});
|
||||
}
|
||||
|
||||
return flights;
|
||||
} catch (err) {
|
||||
if (err.name === 'AbortError') {
|
||||
throw new Error('OpenSky API timeout - try again');
|
||||
}
|
||||
throw err;
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch military flights from Wingbits (fallback when OpenSky fails)
|
||||
async function fetchMilitaryFlightsFromWingbits() {
|
||||
const apiKey = process.env.WINGBITS_API_KEY;
|
||||
if (!apiKey) {
|
||||
console.log('[TheaterPosture] Wingbits not configured, skipping fallback');
|
||||
return null;
|
||||
}
|
||||
|
||||
console.log('[TheaterPosture] Trying Wingbits fallback...');
|
||||
|
||||
// Build batch request for all theaters
|
||||
const areas = POSTURE_THEATERS.map(theater => ({
|
||||
alias: theater.id,
|
||||
by: 'box',
|
||||
la: (theater.bounds.north + theater.bounds.south) / 2,
|
||||
lo: (theater.bounds.east + theater.bounds.west) / 2,
|
||||
w: Math.abs(theater.bounds.east - theater.bounds.west) * 60, // degrees to nm
|
||||
h: Math.abs(theater.bounds.north - theater.bounds.south) * 60,
|
||||
unit: 'nm',
|
||||
}));
|
||||
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 15000);
|
||||
|
||||
try {
|
||||
const response = await fetch('https://customer-api.wingbits.com/v1/flights', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify(areas),
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
console.warn('[TheaterPosture] Wingbits API error:', response.status);
|
||||
return null;
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
console.log('[TheaterPosture] Wingbits returned', data.length, 'theater results');
|
||||
|
||||
// Transform Wingbits data to our format
|
||||
// Wingbits uses short field names: h=icao24, f=flight, la=lat, lo=lon, ab=alt, th=heading, gs=speed
|
||||
const flights = [];
|
||||
const seenIds = new Set();
|
||||
|
||||
for (const areaResult of data) {
|
||||
// Batch response: each area result has flights in various possible formats
|
||||
const areaFlights = areaResult.flights || areaResult.data || areaResult || [];
|
||||
const flightList = Array.isArray(areaFlights) ? areaFlights : [];
|
||||
|
||||
for (const f of flightList) {
|
||||
// Get icao24 - Wingbits uses 'h' for hex ID
|
||||
const icao24 = f.h || f.icao24 || f.id;
|
||||
if (!icao24) continue;
|
||||
|
||||
// Skip duplicates (aircraft may appear in multiple theaters)
|
||||
if (seenIds.has(icao24)) continue;
|
||||
seenIds.add(icao24);
|
||||
|
||||
// Get callsign - Wingbits uses 'f' for flight
|
||||
const callsign = f.f || f.callsign || f.flight || '';
|
||||
|
||||
// Skip if not military (by callsign OR hex range)
|
||||
const isMilitary = isMilitaryCallsign(callsign) || isMilitaryHex(icao24);
|
||||
if (!isMilitary) continue;
|
||||
|
||||
flights.push({
|
||||
id: icao24,
|
||||
callsign: callsign.trim(),
|
||||
lat: f.la || f.latitude || f.lat,
|
||||
lon: f.lo || f.longitude || f.lon || f.lng,
|
||||
altitude: f.ab || f.altitude || f.alt || 0,
|
||||
heading: f.th || f.heading || f.track || 0,
|
||||
speed: f.gs || f.groundSpeed || f.speed || f.velocity || 0,
|
||||
aircraftType: detectAircraftType(callsign),
|
||||
operator: f.operator || 'unknown',
|
||||
source: 'wingbits',
|
||||
militaryHex: isMilitaryHex(icao24),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
console.log('[TheaterPosture] Wingbits: found', flights.length, 'military flights');
|
||||
return flights;
|
||||
} catch (err) {
|
||||
console.error('[TheaterPosture] Wingbits fetch error:', err.message);
|
||||
return null;
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
}
|
||||
}
|
||||
|
||||
// Calculate theater postures
|
||||
function calculatePostures(flights) {
|
||||
const summaries = [];
|
||||
|
||||
for (const theater of POSTURE_THEATERS) {
|
||||
// Filter flights within theater bounds
|
||||
const theaterFlights = flights.filter(f =>
|
||||
f.lat >= theater.bounds.south &&
|
||||
f.lat <= theater.bounds.north &&
|
||||
f.lon >= theater.bounds.west &&
|
||||
f.lon <= theater.bounds.east
|
||||
);
|
||||
|
||||
// Count by type
|
||||
const byType = {
|
||||
fighters: theaterFlights.filter(f => f.aircraftType === 'fighter').length,
|
||||
tankers: theaterFlights.filter(f => f.aircraftType === 'tanker').length,
|
||||
awacs: theaterFlights.filter(f => f.aircraftType === 'awacs').length,
|
||||
reconnaissance: theaterFlights.filter(f => f.aircraftType === 'reconnaissance').length,
|
||||
transport: theaterFlights.filter(f => f.aircraftType === 'transport').length,
|
||||
bombers: theaterFlights.filter(f => f.aircraftType === 'bomber').length,
|
||||
drones: theaterFlights.filter(f => f.aircraftType === 'drone').length,
|
||||
unknown: theaterFlights.filter(f => f.aircraftType === 'unknown').length,
|
||||
};
|
||||
|
||||
const total = Object.values(byType).reduce((a, b) => a + b, 0);
|
||||
|
||||
// Determine posture level
|
||||
const postureLevel = total >= theater.thresholds.critical ? 'critical' :
|
||||
total >= theater.thresholds.elevated ? 'elevated' : 'normal';
|
||||
|
||||
// Check strike capability
|
||||
const strikeCapable =
|
||||
byType.tankers >= theater.strikeIndicators.minTankers &&
|
||||
byType.awacs >= theater.strikeIndicators.minAwacs &&
|
||||
byType.fighters >= theater.strikeIndicators.minFighters;
|
||||
|
||||
// Build summary string
|
||||
const parts = [];
|
||||
if (byType.fighters > 0) parts.push(`${byType.fighters} fighters`);
|
||||
if (byType.tankers > 0) parts.push(`${byType.tankers} tankers`);
|
||||
if (byType.awacs > 0) parts.push(`${byType.awacs} AWACS`);
|
||||
if (byType.reconnaissance > 0) parts.push(`${byType.reconnaissance} recon`);
|
||||
if (byType.bombers > 0) parts.push(`${byType.bombers} bombers`);
|
||||
if (byType.transport > 0) parts.push(`${byType.transport} transport`);
|
||||
if (byType.drones > 0) parts.push(`${byType.drones} drones`);
|
||||
if (byType.unknown > 0) parts.push(`${byType.unknown} other`);
|
||||
const summary = parts.join(', ') || 'No military aircraft';
|
||||
|
||||
// Build headline
|
||||
const headline = postureLevel === 'critical'
|
||||
? `Critical military buildup - ${theater.name}`
|
||||
: postureLevel === 'elevated'
|
||||
? `Elevated military activity - ${theater.name}`
|
||||
: `Normal activity - ${theater.name}`;
|
||||
|
||||
// Build byOperator map for aircraft
|
||||
const byOperator = {};
|
||||
for (const f of theaterFlights) {
|
||||
const op = f.operator || 'unknown';
|
||||
byOperator[op] = (byOperator[op] || 0) + 1;
|
||||
}
|
||||
|
||||
summaries.push({
|
||||
theaterId: theater.id,
|
||||
theaterName: theater.name,
|
||||
shortName: theater.shortName,
|
||||
targetNation: theater.targetNation,
|
||||
// Aircraft
|
||||
fighters: byType.fighters,
|
||||
tankers: byType.tankers,
|
||||
awacs: byType.awacs,
|
||||
reconnaissance: byType.reconnaissance,
|
||||
transport: byType.transport,
|
||||
bombers: byType.bombers,
|
||||
drones: byType.drones,
|
||||
unknown: byType.unknown,
|
||||
totalAircraft: total,
|
||||
// Vessels (populated client-side)
|
||||
destroyers: 0,
|
||||
frigates: 0,
|
||||
carriers: 0,
|
||||
submarines: 0,
|
||||
patrol: 0,
|
||||
auxiliaryVessels: 0,
|
||||
totalVessels: 0,
|
||||
// By operator (aircraft + vessels added client-side)
|
||||
byOperator,
|
||||
// Metadata
|
||||
postureLevel,
|
||||
strikeCapable,
|
||||
trend: 'stable',
|
||||
changePercent: 0,
|
||||
summary,
|
||||
headline,
|
||||
centerLat: (theater.bounds.north + theater.bounds.south) / 2,
|
||||
centerLon: (theater.bounds.east + theater.bounds.west) / 2,
|
||||
bounds: theater.bounds,
|
||||
});
|
||||
}
|
||||
|
||||
return summaries;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
try {
|
||||
// Try to get from cache first
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (cached) {
|
||||
console.log('[TheaterPosture] Cache hit');
|
||||
return Response.json({
|
||||
...cached,
|
||||
cached: true,
|
||||
}, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Fetch and calculate - try OpenSky first, then Wingbits fallback
|
||||
console.log('[TheaterPosture] Fetching fresh data...');
|
||||
let flights;
|
||||
let source = 'opensky';
|
||||
|
||||
try {
|
||||
flights = await fetchMilitaryFlights();
|
||||
} catch (openskyError) {
|
||||
console.warn('[TheaterPosture] OpenSky failed:', openskyError.message);
|
||||
console.log('[TheaterPosture] Trying Wingbits fallback...');
|
||||
|
||||
flights = await fetchMilitaryFlightsFromWingbits();
|
||||
if (flights && flights.length > 0) {
|
||||
source = 'wingbits';
|
||||
console.log('[TheaterPosture] Wingbits fallback succeeded:', flights.length, 'flights');
|
||||
} else {
|
||||
// Both failed, re-throw OpenSky error to trigger cache fallback
|
||||
throw openskyError;
|
||||
}
|
||||
}
|
||||
|
||||
const postures = calculatePostures(flights);
|
||||
|
||||
const result = {
|
||||
postures,
|
||||
totalFlights: flights.length,
|
||||
timestamp: new Date().toISOString(),
|
||||
cached: false,
|
||||
source, // 'opensky' or 'wingbits'
|
||||
};
|
||||
|
||||
// Cache the result (regular, stale, and long-term backup)
|
||||
await Promise.all([
|
||||
setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS),
|
||||
setCachedJson(STALE_CACHE_KEY, result, STALE_CACHE_TTL_SECONDS),
|
||||
setCachedJson(BACKUP_CACHE_KEY, result, BACKUP_CACHE_TTL_SECONDS),
|
||||
]);
|
||||
|
||||
return Response.json(result, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.warn('[TheaterPosture] Error:', error.message);
|
||||
|
||||
// Try to return cached data when API fails (stale first, then backup)
|
||||
const stale = await getCachedJson(STALE_CACHE_KEY);
|
||||
if (stale) {
|
||||
console.log('[TheaterPosture] Returning stale cached data (24h) due to API error');
|
||||
return Response.json({
|
||||
...stale,
|
||||
cached: true,
|
||||
stale: true,
|
||||
error: 'Using cached data - live feed temporarily unavailable',
|
||||
}, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const backup = await getCachedJson(BACKUP_CACHE_KEY);
|
||||
if (backup) {
|
||||
console.log('[TheaterPosture] Returning backup cached data (7d) due to API error');
|
||||
return Response.json({
|
||||
...backup,
|
||||
cached: true,
|
||||
stale: true,
|
||||
error: 'Using backup data - live feed temporarily unavailable',
|
||||
}, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// No cached data available - return error
|
||||
return Response.json({
|
||||
error: error.message,
|
||||
postures: [],
|
||||
timestamp: new Date().toISOString(),
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,237 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'ucdp:gedevents:v2';
|
||||
const CACHE_TTL_SECONDS = 6 * 60 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const UCDP_PAGE_SIZE = 1000;
|
||||
const MAX_PAGES = 12;
|
||||
const TRAILING_WINDOW_MS = 365 * 24 * 60 * 60 * 1000;
|
||||
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: 15,
|
||||
windowMs: 60 * 1000,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(data && typeof data === 'object' && Array.isArray(data.data));
|
||||
}
|
||||
|
||||
const VIOLENCE_TYPE_MAP = {
|
||||
1: 'state-based',
|
||||
2: 'non-state',
|
||||
3: 'one-sided',
|
||||
};
|
||||
|
||||
function parseDateMs(value) {
|
||||
if (!value) return NaN;
|
||||
return Date.parse(String(value));
|
||||
}
|
||||
|
||||
function getMaxDateMs(events) {
|
||||
let maxMs = NaN;
|
||||
for (const event of events) {
|
||||
const ms = parseDateMs(event?.date_start);
|
||||
if (!Number.isFinite(ms)) continue;
|
||||
if (!Number.isFinite(maxMs) || ms > maxMs) {
|
||||
maxMs = ms;
|
||||
}
|
||||
}
|
||||
return maxMs;
|
||||
}
|
||||
|
||||
function buildVersionCandidates() {
|
||||
const year = new Date().getFullYear() - 2000;
|
||||
return Array.from(new Set([
|
||||
`${year}.1`,
|
||||
`${year - 1}.1`,
|
||||
'25.1',
|
||||
'24.1',
|
||||
]));
|
||||
}
|
||||
|
||||
async function fetchGedPage(version, page) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), 8000);
|
||||
try {
|
||||
const response = await fetch(
|
||||
`https://ucdpapi.pcr.uu.se/api/gedevents/${version}?pagesize=${UCDP_PAGE_SIZE}&page=${page}`,
|
||||
{ headers: { Accept: 'application/json' }, signal: controller.signal }
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`UCDP GED API error (${version}, page ${page}): ${response.status}`);
|
||||
}
|
||||
return response.json();
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
async function discoverGedVersion() {
|
||||
const candidates = buildVersionCandidates();
|
||||
for (const version of candidates) {
|
||||
try {
|
||||
const page0 = await fetchGedPage(version, 0);
|
||||
if (Array.isArray(page0?.Result)) {
|
||||
return { version, page0 };
|
||||
}
|
||||
} catch {
|
||||
// Try the next version candidate.
|
||||
}
|
||||
}
|
||||
throw new Error('Unable to fetch UCDP GED metadata from known API versions');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed', data: [] }, {
|
||||
status: 405, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed', data: [] }, {
|
||||
status: 403, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited', data: [] }, {
|
||||
status: 429,
|
||||
headers: { ...corsHeaders, 'Retry-After': '60' },
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/ucdp-events', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/ucdp-events', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const { version, page0 } = await discoverGedVersion();
|
||||
const totalPages = Math.max(1, Number(page0?.TotalPages) || 1);
|
||||
const newestPage = totalPages - 1;
|
||||
|
||||
let allEvents = [];
|
||||
let latestDatasetMs = NaN;
|
||||
|
||||
for (let offset = 0; offset < MAX_PAGES && (newestPage - offset) >= 0; offset++) {
|
||||
const page = newestPage - offset;
|
||||
const rawData = page === 0 ? page0 : await fetchGedPage(version, page);
|
||||
const events = Array.isArray(rawData?.Result) ? rawData.Result : [];
|
||||
allEvents = allEvents.concat(events);
|
||||
|
||||
const pageMaxMs = getMaxDateMs(events);
|
||||
if (!Number.isFinite(latestDatasetMs) && Number.isFinite(pageMaxMs)) {
|
||||
latestDatasetMs = pageMaxMs;
|
||||
}
|
||||
|
||||
// Pages are ordered oldest->newest; once we are fully outside trailing window, stop.
|
||||
if (Number.isFinite(latestDatasetMs) && Number.isFinite(pageMaxMs)) {
|
||||
const cutoffMs = latestDatasetMs - TRAILING_WINDOW_MS;
|
||||
if (pageMaxMs < cutoffMs) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const sanitized = allEvents
|
||||
.filter((event) => {
|
||||
if (!Number.isFinite(latestDatasetMs)) return true;
|
||||
const eventMs = parseDateMs(event?.date_start);
|
||||
if (!Number.isFinite(eventMs)) return false;
|
||||
return eventMs >= (latestDatasetMs - TRAILING_WINDOW_MS);
|
||||
})
|
||||
.map(e => ({
|
||||
id: String(e.id || ''),
|
||||
date_start: e.date_start || '',
|
||||
date_end: e.date_end || '',
|
||||
latitude: Number(e.latitude) || 0,
|
||||
longitude: Number(e.longitude) || 0,
|
||||
country: e.country || '',
|
||||
side_a: (e.side_a || '').substring(0, 200),
|
||||
side_b: (e.side_b || '').substring(0, 200),
|
||||
deaths_best: Number(e.best) || 0,
|
||||
deaths_low: Number(e.low) || 0,
|
||||
deaths_high: Number(e.high) || 0,
|
||||
type_of_violence: VIOLENCE_TYPE_MAP[e.type_of_violence] || 'state-based',
|
||||
source_original: (e.source_original || '').substring(0, 300),
|
||||
}))
|
||||
.sort((a, b) => {
|
||||
const bMs = parseDateMs(b.date_start);
|
||||
const aMs = parseDateMs(a.date_start);
|
||||
return (Number.isFinite(bMs) ? bMs : 0) - (Number.isFinite(aMs) ? aMs : 0);
|
||||
});
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: sanitized.length,
|
||||
data: sanitized,
|
||||
version,
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/ucdp-events', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/ucdp-events', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/ucdp-events', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, data: [] }, {
|
||||
status: 500, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
-150
@@ -1,150 +0,0 @@
|
||||
// UCDP (Uppsala Conflict Data Program) proxy
|
||||
// Returns conflict classification per country with intensity levels
|
||||
// No auth required - public API
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_KEY = 'ucdp:country-conflicts:v2';
|
||||
const CACHE_TTL_SECONDS = 24 * 60 * 60; // 24 hours (annual data)
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const RESPONSE_CACHE_CONTROL = 'public, max-age=3600';
|
||||
|
||||
// In-memory fallback when Redis is unavailable.
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(
|
||||
data &&
|
||||
typeof data === 'object' &&
|
||||
Array.isArray(data.conflicts)
|
||||
);
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/ucdp', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/ucdp', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Fetch all pages of conflicts
|
||||
let allConflicts = [];
|
||||
let page = 0;
|
||||
let totalPages = 1;
|
||||
|
||||
while (page < totalPages) {
|
||||
const response = await fetch(`https://ucdpapi.pcr.uu.se/api/ucdpprioconflict/24.1?pagesize=100&page=${page}`, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`UCDP API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const rawData = await response.json();
|
||||
totalPages = rawData.TotalPages || 1;
|
||||
const conflicts = rawData.Result || [];
|
||||
allConflicts = allConflicts.concat(conflicts);
|
||||
page++;
|
||||
}
|
||||
|
||||
// Fields are snake_case: conflict_id, location, side_a, side_b, year, intensity_level, type_of_conflict
|
||||
const countryConflicts = {};
|
||||
for (const c of allConflicts) {
|
||||
const name = c.location || '';
|
||||
const year = parseInt(c.year, 10) || 0;
|
||||
const intensity = parseInt(c.intensity_level, 10) || 0;
|
||||
|
||||
const entry = {
|
||||
conflictId: parseInt(c.conflict_id, 10) || 0,
|
||||
conflictName: c.side_b || '',
|
||||
location: name,
|
||||
year,
|
||||
intensityLevel: intensity,
|
||||
typeOfConflict: parseInt(c.type_of_conflict, 10) || 0,
|
||||
startDate: c.start_date,
|
||||
startDate2: c.start_date2,
|
||||
sideA: c.side_a,
|
||||
sideB: c.side_b,
|
||||
region: c.region,
|
||||
};
|
||||
|
||||
// Keep most recent / highest intensity per location
|
||||
if (!countryConflicts[name] || year > countryConflicts[name].year ||
|
||||
(year === countryConflicts[name].year && intensity > countryConflicts[name].intensityLevel)) {
|
||||
countryConflicts[name] = entry;
|
||||
}
|
||||
}
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: Object.keys(countryConflicts).length,
|
||||
conflicts: Object.values(countryConflicts),
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/ucdp', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'MISS',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/ucdp', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120',
|
||||
'X-Cache': 'STALE',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/ucdp', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, conflicts: [] }, {
|
||||
status: 500,
|
||||
headers: { ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,270 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'unhcr:population:v2';
|
||||
const CACHE_TTL_SECONDS = 24 * 60 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: 20,
|
||||
windowMs: 60 * 1000,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(data && typeof data === 'object' && Array.isArray(data.countries));
|
||||
}
|
||||
|
||||
const COUNTRY_CENTROIDS = {
|
||||
AFG: [33.9, 67.7], SYR: [35.0, 38.0], UKR: [48.4, 31.2], SDN: [15.5, 32.5],
|
||||
SSD: [6.9, 31.3], SOM: [5.2, 46.2], COD: [-4.0, 21.8], MMR: [19.8, 96.7],
|
||||
YEM: [15.6, 48.5], ETH: [9.1, 40.5], VEN: [6.4, -66.6], IRQ: [33.2, 43.7],
|
||||
COL: [4.6, -74.1], NGA: [9.1, 7.5], PSE: [31.9, 35.2], TUR: [39.9, 32.9],
|
||||
DEU: [51.2, 10.4], PAK: [30.4, 69.3], UGA: [1.4, 32.3], BGD: [23.7, 90.4],
|
||||
KEN: [0.0, 38.0], TCD: [15.5, 19.0], JOR: [31.0, 36.0], LBN: [33.9, 35.5],
|
||||
EGY: [26.8, 30.8], IRN: [32.4, 53.7], TZA: [-6.4, 34.9], RWA: [-1.9, 29.9],
|
||||
CMR: [7.4, 12.4], MLI: [17.6, -4.0], BFA: [12.3, -1.6], NER: [17.6, 8.1],
|
||||
CAF: [6.6, 20.9], MOZ: [-18.7, 35.5], USA: [37.1, -95.7], FRA: [46.2, 2.2],
|
||||
GBR: [55.4, -3.4], IND: [20.6, 79.0], CHN: [35.9, 104.2], RUS: [61.5, 105.3],
|
||||
};
|
||||
|
||||
async function fetchUnhcrYearItems(year) {
|
||||
const limit = 10000;
|
||||
const maxPageGuard = 25;
|
||||
const items = [];
|
||||
|
||||
for (let page = 1; page <= maxPageGuard; page++) {
|
||||
const response = await fetch(
|
||||
`https://api.unhcr.org/population/v1/population/?year=${year}&limit=${limit}&page=${page}`,
|
||||
{ headers: { Accept: 'application/json' } }
|
||||
);
|
||||
|
||||
if (!response.ok) return null;
|
||||
|
||||
const data = await response.json();
|
||||
const pageItems = Array.isArray(data.items) ? data.items : [];
|
||||
if (pageItems.length === 0) break;
|
||||
items.push(...pageItems);
|
||||
|
||||
const maxPages = Number(data.maxPages);
|
||||
if (Number.isFinite(maxPages) && maxPages > 0) {
|
||||
if (page >= maxPages) break;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (pageItems.length < limit) break;
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited' }, {
|
||||
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/unhcr-population', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/unhcr-population', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const currentYear = new Date().getFullYear();
|
||||
let rawItems = [];
|
||||
let dataYearUsed = null;
|
||||
|
||||
for (let year = currentYear; year >= currentYear - 2; year--) {
|
||||
const yearItems = await fetchUnhcrYearItems(year);
|
||||
if (!yearItems) {
|
||||
continue;
|
||||
}
|
||||
rawItems = yearItems;
|
||||
if (rawItems.length > 0) {
|
||||
dataYearUsed = year;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
const byOrigin = {};
|
||||
const byAsylum = {};
|
||||
const flowMap = {};
|
||||
let totalRefugees = 0, totalAsylumSeekers = 0, totalIdps = 0, totalStateless = 0;
|
||||
|
||||
for (const item of rawItems) {
|
||||
const originCode = item.coo_iso || '';
|
||||
const asylumCode = item.coa_iso || '';
|
||||
const refugees = Number(item.refugees) || 0;
|
||||
const asylumSeekers = Number(item.asylum_seekers) || 0;
|
||||
const idps = Number(item.idps) || 0;
|
||||
const stateless = Number(item.stateless) || 0;
|
||||
|
||||
totalRefugees += refugees;
|
||||
totalAsylumSeekers += asylumSeekers;
|
||||
totalIdps += idps;
|
||||
totalStateless += stateless;
|
||||
|
||||
if (originCode) {
|
||||
if (!byOrigin[originCode]) byOrigin[originCode] = { refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, name: item.coo_name || originCode };
|
||||
byOrigin[originCode].refugees += refugees;
|
||||
byOrigin[originCode].asylumSeekers += asylumSeekers;
|
||||
byOrigin[originCode].idps += idps;
|
||||
byOrigin[originCode].stateless += stateless;
|
||||
}
|
||||
|
||||
if (asylumCode) {
|
||||
if (!byAsylum[asylumCode]) byAsylum[asylumCode] = { refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, name: item.coa_name || asylumCode };
|
||||
byAsylum[asylumCode].refugees += refugees;
|
||||
byAsylum[asylumCode].asylumSeekers += asylumSeekers;
|
||||
}
|
||||
|
||||
if (originCode && asylumCode && refugees > 0) {
|
||||
const flowKey = `${originCode}->${asylumCode}`;
|
||||
if (!flowMap[flowKey]) {
|
||||
flowMap[flowKey] = {
|
||||
originCode, originName: item.coo_name || originCode,
|
||||
asylumCode, asylumName: item.coa_name || asylumCode,
|
||||
refugees: 0,
|
||||
};
|
||||
}
|
||||
flowMap[flowKey].refugees += refugees;
|
||||
}
|
||||
}
|
||||
|
||||
const countries = {};
|
||||
for (const [code, data] of Object.entries(byOrigin)) {
|
||||
const centroid = COUNTRY_CENTROIDS[code];
|
||||
countries[code] = {
|
||||
code, name: data.name,
|
||||
refugees: data.refugees, asylumSeekers: data.asylumSeekers,
|
||||
idps: data.idps, stateless: data.stateless,
|
||||
totalDisplaced: data.refugees + data.asylumSeekers + data.idps + data.stateless,
|
||||
hostRefugees: 0,
|
||||
hostAsylumSeekers: 0,
|
||||
hostTotal: 0,
|
||||
lat: centroid?.[0], lon: centroid?.[1],
|
||||
};
|
||||
}
|
||||
for (const [code, data] of Object.entries(byAsylum)) {
|
||||
const hostRefugees = data.refugees;
|
||||
const hostAsylumSeekers = data.asylumSeekers;
|
||||
const hostTotal = hostRefugees + hostAsylumSeekers;
|
||||
if (!countries[code]) {
|
||||
const centroid = COUNTRY_CENTROIDS[code];
|
||||
countries[code] = {
|
||||
code, name: data.name,
|
||||
refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, totalDisplaced: 0,
|
||||
hostRefugees,
|
||||
hostAsylumSeekers,
|
||||
hostTotal,
|
||||
lat: centroid?.[0], lon: centroid?.[1],
|
||||
};
|
||||
} else {
|
||||
countries[code].hostRefugees = hostRefugees;
|
||||
countries[code].hostAsylumSeekers = hostAsylumSeekers;
|
||||
countries[code].hostTotal = hostTotal;
|
||||
}
|
||||
}
|
||||
|
||||
const topFlows = Object.values(flowMap)
|
||||
.sort((a, b) => b.refugees - a.refugees)
|
||||
.slice(0, 50)
|
||||
.map(f => {
|
||||
const oC = COUNTRY_CENTROIDS[f.originCode];
|
||||
const aC = COUNTRY_CENTROIDS[f.asylumCode];
|
||||
return {
|
||||
...f,
|
||||
originLat: oC?.[0], originLon: oC?.[1],
|
||||
asylumLat: aC?.[0], asylumLon: aC?.[1],
|
||||
};
|
||||
});
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
year: dataYearUsed ?? currentYear,
|
||||
globalTotals: {
|
||||
refugees: totalRefugees,
|
||||
asylumSeekers: totalAsylumSeekers,
|
||||
idps: totalIdps,
|
||||
stateless: totalStateless,
|
||||
total: totalRefugees + totalAsylumSeekers + totalIdps + totalStateless,
|
||||
},
|
||||
countries: Object.values(countries).sort((a, b) => {
|
||||
const aSize = Math.max(a.totalDisplaced || 0, a.hostTotal || 0);
|
||||
const bSize = Math.max(b.totalDisplaced || 0, b.hostTotal || 0);
|
||||
return bSize - aSize;
|
||||
}),
|
||||
topFlows,
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/unhcr-population', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/unhcr-population', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/unhcr-population', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, countries: [], topFlows: [] }, {
|
||||
status: 500, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const RELEASES_URL = 'https://api.github.com/repos/koala73/worldmonitor/releases/latest';
|
||||
|
||||
@@ -1,304 +0,0 @@
|
||||
// Wingbits API proxy - keeps API key server-side
|
||||
// Note: Edge runtime is stateless - caching happens client-side and via HTTP Cache-Control
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const url = new URL(req.url);
|
||||
const path = url.pathname.replace('/api/wingbits', '');
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, POST, OPTIONS');
|
||||
|
||||
// Handle CORS preflight
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
return new Response(null, {
|
||||
status: 204,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Get API key from server-side env
|
||||
const apiKey = process.env.WINGBITS_API_KEY;
|
||||
|
||||
if (!apiKey) {
|
||||
return Response.json({
|
||||
error: 'Wingbits not configured',
|
||||
configured: false
|
||||
}, {
|
||||
status: 200,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Route: GET /details/:icao24 - Aircraft details
|
||||
const detailsMatch = path.match(/^\/details\/([a-fA-F0-9]+)$/);
|
||||
if (detailsMatch) {
|
||||
const icao24 = detailsMatch[1].toLowerCase();
|
||||
|
||||
try {
|
||||
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return Response.json({
|
||||
error: `Wingbits API error: ${response.status}`,
|
||||
icao24,
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
return Response.json(data, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', // 24h - aircraft details rarely change
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${error.message}`,
|
||||
icao24,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Route: POST /details/batch - Batch lookup multiple aircraft (parallel)
|
||||
if (path === '/details/batch' && req.method === 'POST') {
|
||||
try {
|
||||
const body = await req.json();
|
||||
const icao24List = body.icao24s || [];
|
||||
|
||||
if (!Array.isArray(icao24List) || icao24List.length === 0) {
|
||||
return Response.json({ error: 'icao24s array required' }, {
|
||||
status: 400,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Limit batch size to avoid overwhelming the API
|
||||
const limitedList = icao24List.slice(0, 20).map(id => id.toLowerCase());
|
||||
const results = {};
|
||||
|
||||
// Fetch all in parallel
|
||||
const fetchPromises = limitedList.map(async (icao24) => {
|
||||
try {
|
||||
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
const data = await response.json();
|
||||
return { icao24, data };
|
||||
}
|
||||
} catch {
|
||||
// Skip failed lookups
|
||||
}
|
||||
return null;
|
||||
});
|
||||
|
||||
const fetchResults = await Promise.all(fetchPromises);
|
||||
|
||||
for (const result of fetchResults) {
|
||||
if (result) {
|
||||
results[result.icao24] = result.data;
|
||||
}
|
||||
}
|
||||
|
||||
return Response.json({
|
||||
results,
|
||||
fetched: Object.keys(results).length,
|
||||
requested: limitedList.length,
|
||||
}, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({
|
||||
error: `Batch lookup failed: ${error.message}`,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Route: GET /flights - Get live flight positions in a geographic area
|
||||
// Query params: la (lat), lo (lon), w (width), h (height), unit (km|nm)
|
||||
if (path === '/flights' && req.method === 'GET') {
|
||||
try {
|
||||
const params = new URLSearchParams(url.search);
|
||||
const la = params.get('la') || params.get('lat');
|
||||
const lo = params.get('lo') || params.get('lon');
|
||||
const w = params.get('w') || params.get('width') || '500';
|
||||
const h = params.get('h') || params.get('height') || '500';
|
||||
const unit = params.get('unit') || 'nm';
|
||||
|
||||
if (!la || !lo) {
|
||||
return Response.json({ error: 'lat (la) and lon (lo) required' }, {
|
||||
status: 400,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const wingbitsUrl = `https://customer-api.wingbits.com/v1/flights?by=box&la=${la}&lo=${lo}&w=${w}&h=${h}&unit=${unit}`;
|
||||
console.log('[Wingbits] Fetching flights:', wingbitsUrl);
|
||||
|
||||
const response = await fetch(wingbitsUrl, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.error('[Wingbits] API error:', response.status, errorText);
|
||||
return Response.json({
|
||||
error: `Wingbits API error: ${response.status}`,
|
||||
details: errorText,
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
console.log('[Wingbits] Got', Array.isArray(data) ? data.length : 0, 'flights');
|
||||
|
||||
return Response.json(data, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15', // 30 seconds - live data
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[Wingbits] Flights fetch error:', error);
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${error.message}`,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Route: POST /flights/batch - Get flights for multiple areas (for theater posture)
|
||||
if (path === '/flights/batch' && req.method === 'POST') {
|
||||
try {
|
||||
const body = await req.json();
|
||||
const areas = body.areas || [];
|
||||
|
||||
if (!Array.isArray(areas) || areas.length === 0) {
|
||||
return Response.json({ error: 'areas array required' }, {
|
||||
status: 400,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Wingbits batch endpoint format
|
||||
const wingbitsAreas = areas.map(area => ({
|
||||
alias: area.id || area.alias,
|
||||
by: 'box',
|
||||
la: (area.north + area.south) / 2,
|
||||
lo: (area.east + area.west) / 2,
|
||||
w: Math.abs(area.east - area.west) * 60, // degrees to nautical miles (approx)
|
||||
h: Math.abs(area.north - area.south) * 60,
|
||||
unit: 'nm',
|
||||
}));
|
||||
|
||||
const response = await fetch('https://customer-api.wingbits.com/v1/flights', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify(wingbitsAreas),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
return Response.json({
|
||||
error: `Wingbits API error: ${response.status}`,
|
||||
details: errorText,
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
console.log('[Wingbits] Batch got', data.length, 'area results');
|
||||
|
||||
return Response.json(data, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[Wingbits] Batch flights error:', error);
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${error.message}`,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Route: GET /health - Check Wingbits status
|
||||
if (path === '/health' || path === '') {
|
||||
try {
|
||||
const response = await fetch('https://customer-api.wingbits.com/health', {
|
||||
headers: { 'x-api-key': apiKey },
|
||||
});
|
||||
const data = await response.json();
|
||||
return Response.json({
|
||||
...data,
|
||||
configured: true,
|
||||
}, {
|
||||
headers: corsHeaders,
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({
|
||||
error: error.message,
|
||||
configured: true,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return Response.json({ error: 'Not found' }, {
|
||||
status: 404,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
@@ -1,57 +0,0 @@
|
||||
// Wingbits single aircraft details
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req, { params }) {
|
||||
const icao24 = params.icao24?.toLowerCase();
|
||||
const apiKey = process.env.WINGBITS_API_KEY;
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (!apiKey) {
|
||||
return Response.json({ error: 'Wingbits not configured', configured: false }, {
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (!icao24 || !/^[a-f0-9]+$/i.test(icao24)) {
|
||||
return Response.json({ error: 'Invalid icao24' }, { status: 400, headers: corsHeaders });
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return Response.json({ error: `Wingbits API error: ${response.status}`, icao24 }, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
return Response.json(data, {
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600' },
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({ error: error.message, icao24 }, { status: 500, headers: corsHeaders });
|
||||
}
|
||||
}
|
||||
@@ -1,74 +0,0 @@
|
||||
// Wingbits batch aircraft details
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const apiKey = process.env.WINGBITS_API_KEY;
|
||||
|
||||
const corsHeaders = getCorsHeaders(req, 'POST, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'POST') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (!apiKey) {
|
||||
return Response.json({ error: 'Wingbits not configured', configured: false }, {
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const body = await req.json();
|
||||
const icao24List = body.icao24s || [];
|
||||
|
||||
if (!Array.isArray(icao24List) || icao24List.length === 0) {
|
||||
return Response.json({ error: 'icao24s array required' }, { status: 400, headers: corsHeaders });
|
||||
}
|
||||
|
||||
// Limit batch size
|
||||
const limitedList = icao24List.slice(0, 20).map(id => id.toLowerCase());
|
||||
const results = {};
|
||||
|
||||
// Fetch all in parallel
|
||||
const fetchPromises = limitedList.map(async (icao24) => {
|
||||
try {
|
||||
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
if (response.ok) {
|
||||
return { icao24, data: await response.json() };
|
||||
}
|
||||
} catch {
|
||||
// Skip failed lookups
|
||||
}
|
||||
return null;
|
||||
});
|
||||
|
||||
const fetchResults = await Promise.all(fetchPromises);
|
||||
for (const result of fetchResults) {
|
||||
if (result) results[result.icao24] = result.data;
|
||||
}
|
||||
|
||||
return Response.json({
|
||||
results,
|
||||
fetched: Object.keys(results).length,
|
||||
requested: limitedList.length,
|
||||
}, { headers: { 'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', ...corsHeaders } });
|
||||
} catch (error) {
|
||||
return Response.json({ error: error.message }, { status: 500, headers: corsHeaders });
|
||||
}
|
||||
}
|
||||
@@ -1,145 +0,0 @@
|
||||
// World Bank API proxy (Web API handler for Edge + sidecar compatibility)
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const TECH_INDICATORS = {
|
||||
'IT.NET.USER.ZS': 'Internet Users (% of population)',
|
||||
'IT.CEL.SETS.P2': 'Mobile Subscriptions (per 100 people)',
|
||||
'IT.NET.BBND.P2': 'Fixed Broadband Subscriptions (per 100 people)',
|
||||
'IT.NET.SECR.P6': 'Secure Internet Servers (per million people)',
|
||||
'GB.XPD.RSDV.GD.ZS': 'R&D Expenditure (% of GDP)',
|
||||
'IP.PAT.RESD': 'Patent Applications (residents)',
|
||||
'IP.PAT.NRES': 'Patent Applications (non-residents)',
|
||||
'IP.TMK.TOTL': 'Trademark Applications',
|
||||
'TX.VAL.TECH.MF.ZS': 'High-Tech Exports (% of manufactured exports)',
|
||||
'BX.GSR.CCIS.ZS': 'ICT Service Exports (% of service exports)',
|
||||
'TM.VAL.ICTG.ZS.UN': 'ICT Goods Imports (% of total goods imports)',
|
||||
'SE.TER.ENRR': 'Tertiary Education Enrollment (%)',
|
||||
'SE.XPD.TOTL.GD.ZS': 'Education Expenditure (% of GDP)',
|
||||
'NY.GDP.MKTP.KD.ZG': 'GDP Growth (annual %)',
|
||||
'NY.GDP.PCAP.CD': 'GDP per Capita (current US$)',
|
||||
'NE.EXP.GNFS.ZS': 'Exports of Goods & Services (% of GDP)',
|
||||
};
|
||||
|
||||
const TECH_COUNTRIES = [
|
||||
'USA', 'CHN', 'JPN', 'DEU', 'KOR', 'GBR', 'IND', 'ISR', 'SGP', 'TWN',
|
||||
'FRA', 'CAN', 'SWE', 'NLD', 'CHE', 'FIN', 'IRL', 'AUS', 'BRA', 'IDN',
|
||||
'ARE', 'SAU', 'QAT', 'BHR', 'EGY', 'TUR',
|
||||
'MYS', 'THA', 'VNM', 'PHL',
|
||||
'ESP', 'ITA', 'POL', 'CZE', 'DNK', 'NOR', 'AUT', 'BEL', 'PRT', 'EST',
|
||||
'MEX', 'ARG', 'CHL', 'COL',
|
||||
'ZAF', 'NGA', 'KEN',
|
||||
];
|
||||
|
||||
export default async function handler(request) {
|
||||
const CORS = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: CORS });
|
||||
}
|
||||
|
||||
function json(data, status = 200, extra = {}) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { 'Content-Type': 'application/json', ...CORS, ...extra },
|
||||
});
|
||||
}
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: CORS });
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const indicator = url.searchParams.get('indicator');
|
||||
const country = url.searchParams.get('country');
|
||||
const countries = url.searchParams.get('countries');
|
||||
const years = url.searchParams.get('years') || '5';
|
||||
const action = url.searchParams.get('action');
|
||||
|
||||
if (action === 'indicators') {
|
||||
return json({ indicators: TECH_INDICATORS, defaultCountries: TECH_COUNTRIES }, 200, { 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600' });
|
||||
}
|
||||
|
||||
if (!indicator) {
|
||||
return json({ error: 'Missing indicator parameter', availableIndicators: Object.keys(TECH_INDICATORS) }, 400);
|
||||
}
|
||||
|
||||
try {
|
||||
let countryList = country || countries || TECH_COUNTRIES.join(';');
|
||||
if (countries) {
|
||||
countryList = countries.split(',').join(';');
|
||||
}
|
||||
|
||||
const currentYear = new Date().getFullYear();
|
||||
const startYear = currentYear - parseInt(years);
|
||||
|
||||
const wbUrl = `https://api.worldbank.org/v2/country/${countryList}/indicator/${indicator}?format=json&date=${startYear}:${currentYear}&per_page=1000`;
|
||||
|
||||
const response = await fetch(wbUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0; +https://worldmonitor.app)',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`World Bank API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (!data || !Array.isArray(data) || data.length < 2 || !data[1]) {
|
||||
return json({
|
||||
indicator,
|
||||
indicatorName: TECH_INDICATORS[indicator] || indicator,
|
||||
metadata: { page: 1, pages: 1, total: 0 },
|
||||
byCountry: {},
|
||||
latestByCountry: {},
|
||||
timeSeries: [],
|
||||
}, 200, { 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' });
|
||||
}
|
||||
|
||||
const [metadata, records] = data;
|
||||
|
||||
const transformed = {
|
||||
indicator,
|
||||
indicatorName: TECH_INDICATORS[indicator] || (records[0]?.indicator?.value || indicator),
|
||||
metadata: { page: metadata.page, pages: metadata.pages, total: metadata.total },
|
||||
byCountry: {},
|
||||
latestByCountry: {},
|
||||
timeSeries: [],
|
||||
};
|
||||
|
||||
for (const record of records || []) {
|
||||
const countryCode = record.countryiso3code || record.country?.id;
|
||||
const countryName = record.country?.value;
|
||||
const year = record.date;
|
||||
const value = record.value;
|
||||
|
||||
if (!countryCode || value === null) continue;
|
||||
|
||||
if (!transformed.byCountry[countryCode]) {
|
||||
transformed.byCountry[countryCode] = { code: countryCode, name: countryName, values: [] };
|
||||
}
|
||||
transformed.byCountry[countryCode].values.push({ year, value });
|
||||
|
||||
if (!transformed.latestByCountry[countryCode] || year > transformed.latestByCountry[countryCode].year) {
|
||||
transformed.latestByCountry[countryCode] = { code: countryCode, name: countryName, year, value };
|
||||
}
|
||||
|
||||
transformed.timeSeries.push({ countryCode, countryName, year, value });
|
||||
}
|
||||
|
||||
for (const c of Object.values(transformed.byCountry)) {
|
||||
c.values.sort((a, b) => a.year - b.year);
|
||||
}
|
||||
|
||||
transformed.timeSeries.sort((a, b) => b.year - a.year || a.countryCode.localeCompare(b.countryCode));
|
||||
|
||||
return json(transformed, 200, { 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' });
|
||||
} catch (error) {
|
||||
return json({ error: error.message, indicator }, 500);
|
||||
}
|
||||
}
|
||||
@@ -1,171 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const COUNTRIES_CACHE_KEY = 'worldpop:countries:v1';
|
||||
const COUNTRIES_TTL_SECONDS = 7 * 24 * 60 * 60;
|
||||
const COUNTRIES_TTL_MS = COUNTRIES_TTL_SECONDS * 1000;
|
||||
const EXPOSURE_TTL_SECONDS = 24 * 60 * 60;
|
||||
|
||||
let countriesFallback = { data: null, timestamp: 0 };
|
||||
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: 30,
|
||||
windowMs: 60 * 1000,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
const PRIORITY_COUNTRIES = {
|
||||
UKR: { name: 'Ukraine', pop: 37000000, area: 603550 },
|
||||
RUS: { name: 'Russia', pop: 144100000, area: 17098242 },
|
||||
ISR: { name: 'Israel', pop: 9800000, area: 22072 },
|
||||
PSE: { name: 'Palestine', pop: 5400000, area: 6020 },
|
||||
SYR: { name: 'Syria', pop: 22100000, area: 185180 },
|
||||
IRN: { name: 'Iran', pop: 88600000, area: 1648195 },
|
||||
TWN: { name: 'Taiwan', pop: 23600000, area: 36193 },
|
||||
ETH: { name: 'Ethiopia', pop: 126500000, area: 1104300 },
|
||||
SDN: { name: 'Sudan', pop: 48100000, area: 1861484 },
|
||||
SSD: { name: 'South Sudan', pop: 11400000, area: 619745 },
|
||||
SOM: { name: 'Somalia', pop: 18100000, area: 637657 },
|
||||
YEM: { name: 'Yemen', pop: 34400000, area: 527968 },
|
||||
AFG: { name: 'Afghanistan', pop: 42200000, area: 652230 },
|
||||
PAK: { name: 'Pakistan', pop: 240500000, area: 881913 },
|
||||
IND: { name: 'India', pop: 1428600000, area: 3287263 },
|
||||
MMR: { name: 'Myanmar', pop: 54200000, area: 676578 },
|
||||
COD: { name: 'DR Congo', pop: 102300000, area: 2344858 },
|
||||
NGA: { name: 'Nigeria', pop: 223800000, area: 923768 },
|
||||
MLI: { name: 'Mali', pop: 22600000, area: 1240192 },
|
||||
BFA: { name: 'Burkina Faso', pop: 22700000, area: 274200 },
|
||||
};
|
||||
|
||||
function isValidCountries(data) {
|
||||
return Boolean(data && typeof data === 'object' && Array.isArray(data.countries));
|
||||
}
|
||||
|
||||
async function handleCountries(corsHeaders, now) {
|
||||
const cached = await getCachedJson(COUNTRIES_CACHE_KEY);
|
||||
if (isValidCountries(cached)) {
|
||||
recordCacheTelemetry('/api/worldpop-exposure?countries', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'REDIS-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidCountries(countriesFallback.data) && now - countriesFallback.timestamp < COUNTRIES_TTL_MS) {
|
||||
recordCacheTelemetry('/api/worldpop-exposure?countries', 'MEMORY-HIT');
|
||||
return Response.json(countriesFallback.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'MEMORY-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
const countries = Object.entries(PRIORITY_COUNTRIES).map(([code, info]) => ({
|
||||
code,
|
||||
name: info.name,
|
||||
population: info.pop,
|
||||
densityPerKm2: Math.round(info.pop / info.area),
|
||||
}));
|
||||
|
||||
const result = { success: true, countries, cached_at: new Date().toISOString() };
|
||||
countriesFallback = { data: result, timestamp: now };
|
||||
void setCachedJson(COUNTRIES_CACHE_KEY, result, COUNTRIES_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/worldpop-exposure?countries', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'MISS' },
|
||||
});
|
||||
}
|
||||
|
||||
function handleExposure(corsHeaders, lat, lon, radius) {
|
||||
let bestMatch = null;
|
||||
let bestDist = Infinity;
|
||||
|
||||
const CENTROIDS = {
|
||||
UKR: [48.4, 31.2], RUS: [61.5, 105.3], ISR: [31.0, 34.8], PSE: [31.9, 35.2],
|
||||
SYR: [35.0, 38.0], IRN: [32.4, 53.7], TWN: [23.7, 121.0], ETH: [9.1, 40.5],
|
||||
SDN: [15.5, 32.5], SSD: [6.9, 31.3], SOM: [5.2, 46.2], YEM: [15.6, 48.5],
|
||||
AFG: [33.9, 67.7], PAK: [30.4, 69.3], IND: [20.6, 79.0], MMR: [19.8, 96.7],
|
||||
COD: [-4.0, 21.8], NGA: [9.1, 7.5], MLI: [17.6, -4.0], BFA: [12.3, -1.6],
|
||||
};
|
||||
|
||||
for (const [code, [cLat, cLon]] of Object.entries(CENTROIDS)) {
|
||||
const dist = Math.sqrt(Math.pow(lat - cLat, 2) + Math.pow(lon - cLon, 2));
|
||||
if (dist < bestDist) {
|
||||
bestDist = dist;
|
||||
bestMatch = code;
|
||||
}
|
||||
}
|
||||
|
||||
const info = PRIORITY_COUNTRIES[bestMatch] || { pop: 50000000, area: 500000 };
|
||||
const density = info.pop / info.area;
|
||||
const areaKm2 = Math.PI * radius * radius;
|
||||
const exposed = Math.round(density * areaKm2);
|
||||
|
||||
return Response.json({
|
||||
success: true,
|
||||
exposedPopulation: exposed,
|
||||
exposureRadiusKm: radius,
|
||||
nearestCountry: bestMatch,
|
||||
densityPerKm2: Math.round(density),
|
||||
}, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited' }, {
|
||||
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const mode = url.searchParams.get('mode') || 'countries';
|
||||
|
||||
if (mode === 'exposure') {
|
||||
const lat = Number(url.searchParams.get('lat'));
|
||||
const lon = Number(url.searchParams.get('lon'));
|
||||
const radius = Number(url.searchParams.get('radius')) || 50;
|
||||
|
||||
if (isNaN(lat) || isNaN(lon)) {
|
||||
return Response.json({ error: 'lat and lon required' }, { status: 400, headers: corsHeaders });
|
||||
}
|
||||
|
||||
return handleExposure(corsHeaders, lat, lon, radius);
|
||||
}
|
||||
|
||||
return handleCountries(corsHeaders, Date.now());
|
||||
}
|
||||
Vendored
-54
@@ -1,54 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const SYMBOL_PATTERN = /^[A-Za-z0-9.^=\-]+$/;
|
||||
const MAX_SYMBOL_LENGTH = 20;
|
||||
|
||||
function validateSymbol(symbol) {
|
||||
if (!symbol) return null;
|
||||
const trimmed = symbol.trim().toUpperCase();
|
||||
if (trimmed.length > MAX_SYMBOL_LENGTH) return null;
|
||||
if (!SYMBOL_PATTERN.test(trimmed)) return null;
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const symbol = validateSymbol(url.searchParams.get('symbol'));
|
||||
|
||||
if (!symbol) {
|
||||
return new Response(JSON.stringify({ error: 'Invalid or missing symbol parameter' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const yahooUrl = `https://query1.finance.yahoo.com/v8/finance/chart/${encodeURIComponent(symbol)}`;
|
||||
const response = await fetch(yahooUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
|
||||
},
|
||||
});
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
+48
-12
@@ -19,17 +19,31 @@ const ALLOWED_ORIGINS = [
|
||||
/^tauri:\/\/localhost$/,
|
||||
];
|
||||
|
||||
function sanitizeOrigin(raw) {
|
||||
if (!raw) return 'https://worldmonitor.app';
|
||||
const ALLOWED_PARENT_ORIGINS = [
|
||||
...ALLOWED_ORIGINS,
|
||||
/^https?:\/\/tauri\.localhost$/,
|
||||
/^https?:\/\/[a-z0-9-]+\.tauri\.localhost$/,
|
||||
];
|
||||
|
||||
function sanitizeAllowedOrigin(raw, fallback, allowList = ALLOWED_ORIGINS) {
|
||||
if (!raw) return fallback;
|
||||
try {
|
||||
const parsed = new URL(raw);
|
||||
if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:' && parsed.protocol !== 'tauri:') {
|
||||
return 'https://worldmonitor.app';
|
||||
if (!['https:', 'http:', 'tauri:'].includes(parsed.protocol)) {
|
||||
return fallback;
|
||||
}
|
||||
const origin = parsed.origin !== 'null' ? parsed.origin : raw;
|
||||
if (ALLOWED_ORIGINS.some(p => p.test(origin))) return origin;
|
||||
if (allowList.some(p => p.test(origin))) return origin;
|
||||
} catch { /* invalid URL */ }
|
||||
return 'https://worldmonitor.app';
|
||||
return fallback;
|
||||
}
|
||||
|
||||
function sanitizeOrigin(raw) {
|
||||
return sanitizeAllowedOrigin(raw, 'https://worldmonitor.app', ALLOWED_ORIGINS);
|
||||
}
|
||||
|
||||
function sanitizeParentOrigin(raw, fallback) {
|
||||
return sanitizeAllowedOrigin(raw, fallback, ALLOWED_PARENT_ORIGINS);
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
@@ -45,10 +59,12 @@ export default async function handler(request) {
|
||||
|
||||
const autoplay = parseFlag(url.searchParams.get('autoplay'), '1');
|
||||
const mute = parseFlag(url.searchParams.get('mute'), '1');
|
||||
const vq = ['small', 'medium', 'large', 'hd720', 'hd1080'].includes(url.searchParams.get('vq') || '') ? url.searchParams.get('vq') : '';
|
||||
|
||||
const origin = sanitizeOrigin(url.searchParams.get('origin'));
|
||||
const parentOrigin = sanitizeParentOrigin(url.searchParams.get('parentOrigin'), origin);
|
||||
|
||||
const embedSrc = new URL(`https://www.youtube-nocookie.com/embed/${videoId}`);
|
||||
const embedSrc = new URL(`https://www.youtube.com/embed/${videoId}`);
|
||||
embedSrc.searchParams.set('autoplay', autoplay);
|
||||
embedSrc.searchParams.set('mute', mute);
|
||||
embedSrc.searchParams.set('playsinline', '1');
|
||||
@@ -79,21 +95,39 @@ export default async function handler(request) {
|
||||
var tag=document.createElement('script');
|
||||
tag.src='https://www.youtube.com/iframe_api';
|
||||
document.head.appendChild(tag);
|
||||
var player,overlay=document.getElementById('play-overlay'),started=false;
|
||||
var player,overlay=document.getElementById('play-overlay'),started=false,muteSyncIntervalId,parentOrigin=${JSON.stringify(parentOrigin)},allowedOrigin=${JSON.stringify(parentOrigin)};
|
||||
function hideOverlay(){overlay.classList.add('hidden')}
|
||||
function readMuted(){
|
||||
if(!player)return null;
|
||||
if(typeof player.isMuted==='function')return player.isMuted();
|
||||
if(typeof player.getVolume==='function')return player.getVolume()===0;
|
||||
return null;
|
||||
}
|
||||
function stopMuteSync(){if(muteSyncIntervalId){clearInterval(muteSyncIntervalId);muteSyncIntervalId=null}}
|
||||
function startMuteSync(){
|
||||
if(muteSyncIntervalId)return;
|
||||
var lastMuted=readMuted();
|
||||
if(lastMuted!==null)window.parent.postMessage({type:'yt-mute-state',muted:lastMuted},parentOrigin);
|
||||
muteSyncIntervalId=setInterval(function(){
|
||||
var m=readMuted();
|
||||
if(m!==null&&m!==lastMuted){lastMuted=m;window.parent.postMessage({type:'yt-mute-state',muted:m},parentOrigin)}
|
||||
},500);
|
||||
}
|
||||
function onYouTubeIframeAPIReady(){
|
||||
player=new YT.Player('player',{
|
||||
videoId:'${videoId}',
|
||||
host:'https://www.youtube-nocookie.com',
|
||||
host:'https://www.youtube.com',
|
||||
playerVars:{autoplay:${autoplay},mute:${mute},playsinline:1,rel:0,controls:1,modestbranding:1,enablejsapi:1,origin:${JSON.stringify(origin)},widget_referrer:${JSON.stringify(origin)}},
|
||||
events:{
|
||||
onReady:function(){
|
||||
window.parent.postMessage({type:'yt-ready'},'*');
|
||||
window.parent.postMessage({type:'yt-ready'},parentOrigin);
|
||||
${vq ? `if(player.setPlaybackQuality)player.setPlaybackQuality('${vq}');` : ''}
|
||||
if(${autoplay}===1){player.playVideo()}
|
||||
startMuteSync();
|
||||
},
|
||||
onError:function(e){window.parent.postMessage({type:'yt-error',code:e.data},'*')},
|
||||
onError:function(e){stopMuteSync();window.parent.postMessage({type:'yt-error',code:e.data},parentOrigin)},
|
||||
onStateChange:function(e){
|
||||
window.parent.postMessage({type:'yt-state',state:e.data},'*');
|
||||
window.parent.postMessage({type:'yt-state',state:e.data},parentOrigin);
|
||||
if(e.data===1||e.data===3){hideOverlay();started=true}
|
||||
}
|
||||
}
|
||||
@@ -104,6 +138,7 @@ export default async function handler(request) {
|
||||
});
|
||||
setTimeout(function(){if(!started)overlay.classList.remove('hidden')},3000);
|
||||
window.addEventListener('message',function(e){
|
||||
if(allowedOrigin!=='*'&&e.origin!==allowedOrigin)return;
|
||||
if(!player||!player.getPlayerState)return;
|
||||
var m=e.data;if(!m||!m.type)return;
|
||||
switch(m.type){
|
||||
@@ -112,6 +147,7 @@ export default async function handler(request) {
|
||||
case'mute':player.mute();break;
|
||||
case'unmute':player.unMute();break;
|
||||
case'loadVideo':if(m.videoId)player.loadVideoById(m.videoId);break;
|
||||
case'setQuality':if(m.quality&&player.setPlaybackQuality)player.setPlaybackQuality(m.quality);break;
|
||||
}
|
||||
});
|
||||
</script>
|
||||
|
||||
@@ -21,7 +21,7 @@ test('returns embeddable html for valid video id', async () => {
|
||||
|
||||
const html = await response.text();
|
||||
assert.equal(html.includes("videoId:'iEpJwprxDdk'"), true);
|
||||
assert.equal(html.includes("host:'https://www.youtube-nocookie.com'"), true);
|
||||
assert.equal(html.includes("host:'https://www.youtube.com'"), true);
|
||||
assert.equal(html.includes('autoplay:0'), true);
|
||||
assert.equal(html.includes('mute:1'), true);
|
||||
assert.equal(html.includes('origin:"https://worldmonitor.app"'), true);
|
||||
@@ -33,3 +33,18 @@ test('accepts custom origin parameter', async () => {
|
||||
const html = await response.text();
|
||||
assert.equal(html.includes('origin:"http://127.0.0.1:46123"'), true);
|
||||
});
|
||||
|
||||
test('uses dedicated parentOrigin for iframe postMessage target', async () => {
|
||||
const response = await handler(makeRequest('?videoId=iEpJwprxDdk&origin=https://worldmonitor.app&parentOrigin=https://tauri.localhost'));
|
||||
const html = await response.text();
|
||||
assert.match(html, /playerVars:\{[^}]*origin:"https:\/\/worldmonitor\.app"/);
|
||||
assert.match(html, /parentOrigin="https:\/\/tauri\.localhost"/);
|
||||
assert.match(html, /if\(allowedOrigin!==['"]\*['"]&&e\.origin!==allowedOrigin\)return/);
|
||||
});
|
||||
|
||||
test('does not accept wildcard parentOrigin query parameter', async () => {
|
||||
const response = await handler(makeRequest('?videoId=iEpJwprxDdk&origin=https://worldmonitor.app&parentOrigin=*'));
|
||||
const html = await response.text();
|
||||
assert.equal(html.includes('parentOrigin="*"'), false);
|
||||
assert.match(html, /parentOrigin="https:\/\/worldmonitor\.app"/);
|
||||
});
|
||||
|
||||
+15
-14
@@ -36,7 +36,7 @@ export default async function handler(request) {
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return new Response(JSON.stringify({ videoId: null }), {
|
||||
return new Response(JSON.stringify({ videoId: null, channelExists: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
@@ -44,22 +44,23 @@ export default async function handler(request) {
|
||||
|
||||
const html = await response.text();
|
||||
|
||||
// Extract video ID from the page
|
||||
const videoIdMatch = html.match(/"videoId":"([a-zA-Z0-9_-]{11})"/);
|
||||
const isLiveMatch = html.match(/"isLive":\s*true/);
|
||||
// Channel exists if the page contains canonical channel metadata
|
||||
const channelExists = html.includes('"channelId"') || html.includes('og:url');
|
||||
|
||||
if (videoIdMatch && isLiveMatch) {
|
||||
return new Response(JSON.stringify({ videoId: videoIdMatch[1], isLive: true }), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', // Cache for 5 minutes
|
||||
},
|
||||
});
|
||||
// Scope both fields to the same videoDetails block so we don't
|
||||
// combine a videoId from one object with isLive from another.
|
||||
let videoId = null;
|
||||
const detailsIdx = html.indexOf('"videoDetails"');
|
||||
if (detailsIdx !== -1) {
|
||||
const block = html.substring(detailsIdx, detailsIdx + 5000);
|
||||
const vidMatch = block.match(/"videoId":"([a-zA-Z0-9_-]{11})"/);
|
||||
const liveMatch = block.match(/"isLive"\s*:\s*true/);
|
||||
if (vidMatch && liveMatch) {
|
||||
videoId = vidMatch[1];
|
||||
}
|
||||
}
|
||||
|
||||
// Return null if no live stream found
|
||||
return new Response(JSON.stringify({ videoId: null, isLive: false }), {
|
||||
return new Response(JSON.stringify({ videoId, isLive: videoId !== null, channelExists }), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
import { mutation } from "./_generated/server";
|
||||
import { v } from "convex/values";
|
||||
|
||||
export const register = mutation({
|
||||
args: {
|
||||
email: v.string(),
|
||||
source: v.optional(v.string()),
|
||||
appVersion: v.optional(v.string()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const normalizedEmail = args.email.trim().toLowerCase();
|
||||
|
||||
const existing = await ctx.db
|
||||
.query("registrations")
|
||||
.withIndex("by_normalized_email", (q) => q.eq("normalizedEmail", normalizedEmail))
|
||||
.first();
|
||||
|
||||
if (existing) {
|
||||
return { status: "already_registered" as const };
|
||||
}
|
||||
|
||||
await ctx.db.insert("registrations", {
|
||||
email: args.email.trim(),
|
||||
normalizedEmail,
|
||||
registeredAt: Date.now(),
|
||||
source: args.source ?? "unknown",
|
||||
appVersion: args.appVersion ?? "unknown",
|
||||
});
|
||||
|
||||
return { status: "registered" as const };
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,12 @@
|
||||
import { defineSchema, defineTable } from "convex/server";
|
||||
import { v } from "convex/values";
|
||||
|
||||
export default defineSchema({
|
||||
registrations: defineTable({
|
||||
email: v.string(),
|
||||
normalizedEmail: v.string(),
|
||||
registeredAt: v.number(),
|
||||
source: v.optional(v.string()),
|
||||
appVersion: v.optional(v.string()),
|
||||
}).index("by_normalized_email", ["normalizedEmail"]),
|
||||
});
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ESNext",
|
||||
"lib": ["ES2021"],
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"strict": true,
|
||||
"skipLibCheck": true,
|
||||
"allowJs": true,
|
||||
"outDir": "./_generated"
|
||||
},
|
||||
"include": ["./**/*.ts"],
|
||||
"exclude": ["./_generated"]
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
{
|
||||
"version": 1,
|
||||
"updatedAt": "2026-02-23T12:19:41Z",
|
||||
"note": "Product-managed curated list. Not user-configurable.",
|
||||
"channels": {
|
||||
"full": [
|
||||
{
|
||||
"handle": "VahidOnline",
|
||||
"label": "Vahid Online",
|
||||
"topic": "politics",
|
||||
"tier": 1,
|
||||
"enabled": true,
|
||||
"region": "iran",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "BNONews",
|
||||
"label": "BNO News",
|
||||
"topic": "breaking",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 25
|
||||
},
|
||||
{
|
||||
"handle": "LiveUAMap",
|
||||
"label": "LiveUAMap",
|
||||
"topic": "breaking",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 25
|
||||
},
|
||||
{
|
||||
"handle": "ClashReport",
|
||||
"label": "Clash Report",
|
||||
"topic": "conflict",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 30
|
||||
},
|
||||
{
|
||||
"handle": "OSINTdefender",
|
||||
"label": "OSINTdefender",
|
||||
"topic": "conflict",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 25
|
||||
},
|
||||
{
|
||||
"handle": "AuroraIntel",
|
||||
"label": "Aurora Intel",
|
||||
"topic": "conflict",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "GeopoliticalCenter",
|
||||
"label": "GeopoliticalCenter",
|
||||
"topic": "geopolitics",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "Osintlatestnews",
|
||||
"label": "OSIntOps News",
|
||||
"topic": "osint",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "air_alert_ua",
|
||||
"label": "Повітряна Тривога",
|
||||
"topic": "alerts",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "ukraine",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "kpszsu",
|
||||
"label": "Air Force of the Armed Forces of Ukraine",
|
||||
"topic": "alerts",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "ukraine",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "war_monitor",
|
||||
"label": "monitor",
|
||||
"topic": "alerts",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "ukraine",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "DeepStateUA",
|
||||
"label": "DeepState",
|
||||
"topic": "conflict",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "ukraine",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "bellingcat",
|
||||
"label": "Bellingcat",
|
||||
"topic": "osint",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 10
|
||||
},
|
||||
{
|
||||
"handle": "nexta_live",
|
||||
"label": "NEXTA Live",
|
||||
"topic": "breaking",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "europe",
|
||||
"maxMessages": 15
|
||||
},
|
||||
{
|
||||
"handle": "nexta_tv",
|
||||
"label": "NEXTA",
|
||||
"topic": "politics",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "europe",
|
||||
"maxMessages": 15
|
||||
}
|
||||
],
|
||||
"tech": [],
|
||||
"finance": []
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
# Nginx API proxy compression baseline for WorldMonitor.
|
||||
# Requires ngx_brotli (or Nginx Plus Brotli module) to be installed.
|
||||
|
||||
# Prefer Brotli for HTTPS clients and keep gzip as fallback.
|
||||
brotli on;
|
||||
brotli_comp_level 5;
|
||||
brotli_min_length 1024;
|
||||
brotli_types application/json application/javascript text/css text/plain application/xml text/xml;
|
||||
|
||||
gzip on;
|
||||
gzip_comp_level 5;
|
||||
gzip_min_length 1024;
|
||||
gzip_vary on;
|
||||
gzip_proxied any;
|
||||
gzip_types application/json application/javascript text/css text/plain application/xml text/xml;
|
||||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name api.worldmonitor.local;
|
||||
|
||||
location /api/ {
|
||||
proxy_pass http://127.0.0.1:8787;
|
||||
proxy_http_version 1.1;
|
||||
|
||||
# Preserve upstream compression behavior and pass through client preferences.
|
||||
proxy_set_header Accept-Encoding $http_accept_encoding;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# If upstream sends pre-compressed content, do not decompress.
|
||||
gunzip off;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,461 @@
|
||||
# Adding API Endpoints
|
||||
|
||||
All JSON API endpoints in WorldMonitor **must** use sebuf. Do not create standalone `api/*.js` files — the legacy pattern is deprecated and being removed.
|
||||
|
||||
This guide walks through adding a new RPC to an existing service and adding an entirely new service.
|
||||
|
||||
> **Important:** After modifying any `.proto` file, you **must** run `make generate` before building or pushing. The generated TypeScript files in `src/generated/` are checked into the repo and must stay in sync with the proto definitions. CI does not run generation yet — this is your responsibility until we add it to the pipeline (see [#200](https://github.com/koala73/worldmonitor/issues/200)).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need **Go 1.21+** and **Node.js 18+** installed. Everything else is installed automatically:
|
||||
|
||||
```bash
|
||||
make install # one-time: installs buf, sebuf plugins, npm deps, proto deps
|
||||
```
|
||||
|
||||
This installs:
|
||||
|
||||
- **buf** — proto linting, dependency management, and code generation orchestrator
|
||||
- **protoc-gen-ts-client** — generates TypeScript client classes (from [sebuf](https://github.com/SebastienMelki/sebuf))
|
||||
- **protoc-gen-ts-server** — generates TypeScript server handler interfaces (from sebuf)
|
||||
- **protoc-gen-openapiv3** — generates OpenAPI v3 specs (from sebuf)
|
||||
- **npm dependencies** — all Node.js packages
|
||||
|
||||
Run code generation from the repo root:
|
||||
|
||||
```bash
|
||||
make generate # regenerate all TypeScript + OpenAPI from protos
|
||||
```
|
||||
|
||||
This produces three outputs per service:
|
||||
|
||||
- `src/generated/client/{domain}/v1/service_client.ts` — typed fetch client for the frontend
|
||||
- `src/generated/server/{domain}/v1/service_server.ts` — handler interface + route factory for the backend
|
||||
- `docs/api/{Domain}Service.openapi.yaml` + `.json` — OpenAPI v3 documentation
|
||||
|
||||
## Adding an RPC to an existing service
|
||||
|
||||
Example: adding `GetEarthquakeDetails` to `SeismologyService`.
|
||||
|
||||
### 1. Define the request/response messages
|
||||
|
||||
Create `proto/worldmonitor/seismology/v1/get_earthquake_details.proto`:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package worldmonitor.seismology.v1;
|
||||
|
||||
import "buf/validate/validate.proto";
|
||||
import "worldmonitor/seismology/v1/earthquake.proto";
|
||||
|
||||
// GetEarthquakeDetailsRequest specifies which earthquake to retrieve.
|
||||
message GetEarthquakeDetailsRequest {
|
||||
// USGS event identifier (e.g., "us7000abcd").
|
||||
string earthquake_id = 1 [
|
||||
(buf.validate.field).required = true,
|
||||
(buf.validate.field).string.min_len = 1,
|
||||
(buf.validate.field).string.max_len = 100
|
||||
];
|
||||
}
|
||||
|
||||
// GetEarthquakeDetailsResponse contains the full earthquake record.
|
||||
message GetEarthquakeDetailsResponse {
|
||||
// The earthquake matching the requested ID.
|
||||
Earthquake earthquake = 1;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Add the RPC to the service definition
|
||||
|
||||
Edit `proto/worldmonitor/seismology/v1/service.proto`:
|
||||
|
||||
```protobuf
|
||||
import "worldmonitor/seismology/v1/get_earthquake_details.proto";
|
||||
|
||||
service SeismologyService {
|
||||
// ... existing RPCs ...
|
||||
|
||||
// GetEarthquakeDetails retrieves a single earthquake by its USGS event ID.
|
||||
rpc GetEarthquakeDetails(GetEarthquakeDetailsRequest) returns (GetEarthquakeDetailsResponse) {
|
||||
option (sebuf.http.config) = {path: "/get-earthquake-details"};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Lint and generate
|
||||
|
||||
```bash
|
||||
make check # lint + generate in one step
|
||||
```
|
||||
|
||||
At this point, `npx tsc --noEmit` will **fail** because the handler doesn't implement the new method yet. This is by design — the compiler enforces the contract.
|
||||
|
||||
### 4. Implement the handler
|
||||
|
||||
Create `server/worldmonitor/seismology/v1/get-earthquake-details.ts`:
|
||||
|
||||
```typescript
|
||||
import type {
|
||||
SeismologyServiceHandler,
|
||||
ServerContext,
|
||||
GetEarthquakeDetailsRequest,
|
||||
GetEarthquakeDetailsResponse,
|
||||
} from '../../../../src/generated/server/worldmonitor/seismology/v1/service_server';
|
||||
|
||||
export const getEarthquakeDetails: SeismologyServiceHandler['getEarthquakeDetails'] = async (
|
||||
_ctx: ServerContext,
|
||||
req: GetEarthquakeDetailsRequest,
|
||||
): Promise<GetEarthquakeDetailsResponse> => {
|
||||
const response = await fetch(
|
||||
`https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/${req.earthquakeId}.geojson`,
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`USGS API error: ${response.status}`);
|
||||
}
|
||||
const f: any = await response.json();
|
||||
return {
|
||||
earthquake: {
|
||||
id: f.id,
|
||||
place: f.properties.place || '',
|
||||
magnitude: f.properties.mag ?? 0,
|
||||
depthKm: f.geometry.coordinates[2] ?? 0,
|
||||
location: {
|
||||
latitude: f.geometry.coordinates[1],
|
||||
longitude: f.geometry.coordinates[0],
|
||||
},
|
||||
occurredAt: f.properties.time,
|
||||
sourceUrl: f.properties.url || '',
|
||||
},
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
### 5. Wire it into the handler re-export
|
||||
|
||||
Edit `server/worldmonitor/seismology/v1/handler.ts`:
|
||||
|
||||
```typescript
|
||||
import type { SeismologyServiceHandler } from '../../../../src/generated/server/worldmonitor/seismology/v1/service_server';
|
||||
|
||||
import { listEarthquakes } from './list-earthquakes';
|
||||
import { getEarthquakeDetails } from './get-earthquake-details';
|
||||
|
||||
export const seismologyHandler: SeismologyServiceHandler = {
|
||||
listEarthquakes,
|
||||
getEarthquakeDetails,
|
||||
};
|
||||
```
|
||||
|
||||
### 6. Verify
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit # should pass with zero errors
|
||||
```
|
||||
|
||||
The route is already live. `createSeismologyServiceRoutes()` picks up the new RPC automatically — no changes needed to `api/[[...path]].ts` or `vite.config.ts`.
|
||||
|
||||
### 7. Check the generated docs
|
||||
|
||||
Open `docs/api/SeismologyService.openapi.yaml` — the new endpoint should appear with all validation constraints from your proto annotations.
|
||||
|
||||
## Adding a new service
|
||||
|
||||
Example: adding a `SanctionsService`.
|
||||
|
||||
### 1. Create the proto directory
|
||||
|
||||
```
|
||||
proto/worldmonitor/sanctions/v1/
|
||||
```
|
||||
|
||||
### 2. Define entity messages
|
||||
|
||||
Create `proto/worldmonitor/sanctions/v1/sanctions_entry.proto`:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package worldmonitor.sanctions.v1;
|
||||
|
||||
import "buf/validate/validate.proto";
|
||||
import "sebuf/http/annotations.proto";
|
||||
|
||||
// SanctionsEntry represents a single entity on a sanctions list.
|
||||
message SanctionsEntry {
|
||||
// Unique identifier.
|
||||
string id = 1 [
|
||||
(buf.validate.field).required = true,
|
||||
(buf.validate.field).string.min_len = 1
|
||||
];
|
||||
// Name of the sanctioned entity or individual.
|
||||
string name = 2;
|
||||
// Issuing authority (e.g., "OFAC", "EU", "UN").
|
||||
string authority = 3;
|
||||
// ISO 3166-1 alpha-2 country code of the target.
|
||||
string country_code = 4;
|
||||
// Date the sanction was imposed, as Unix epoch milliseconds.
|
||||
int64 imposed_at = 5 [(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER];
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Define request/response messages
|
||||
|
||||
Create `proto/worldmonitor/sanctions/v1/list_sanctions.proto`:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package worldmonitor.sanctions.v1;
|
||||
|
||||
import "buf/validate/validate.proto";
|
||||
import "worldmonitor/core/v1/pagination.proto";
|
||||
import "worldmonitor/sanctions/v1/sanctions_entry.proto";
|
||||
|
||||
// ListSanctionsRequest specifies filters for sanctions data.
|
||||
message ListSanctionsRequest {
|
||||
// Filter by issuing authority (e.g., "OFAC"). Empty returns all.
|
||||
string authority = 1;
|
||||
// Filter by country code.
|
||||
string country_code = 2 [(buf.validate.field).string.max_len = 2];
|
||||
// Pagination parameters.
|
||||
worldmonitor.core.v1.PaginationRequest pagination = 3;
|
||||
}
|
||||
|
||||
// ListSanctionsResponse contains the matching sanctions entries.
|
||||
message ListSanctionsResponse {
|
||||
// The list of sanctions entries.
|
||||
repeated SanctionsEntry entries = 1;
|
||||
// Pagination metadata.
|
||||
worldmonitor.core.v1.PaginationResponse pagination = 2;
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Define the service
|
||||
|
||||
Create `proto/worldmonitor/sanctions/v1/service.proto`:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package worldmonitor.sanctions.v1;
|
||||
|
||||
import "sebuf/http/annotations.proto";
|
||||
import "worldmonitor/sanctions/v1/list_sanctions.proto";
|
||||
|
||||
// SanctionsService provides APIs for international sanctions monitoring.
|
||||
service SanctionsService {
|
||||
option (sebuf.http.service_config) = {base_path: "/api/sanctions/v1"};
|
||||
|
||||
// ListSanctions retrieves sanctions entries matching the given filters.
|
||||
rpc ListSanctions(ListSanctionsRequest) returns (ListSanctionsResponse) {
|
||||
option (sebuf.http.config) = {path: "/list-sanctions"};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Generate
|
||||
|
||||
```bash
|
||||
make check # lint + generate in one step
|
||||
```
|
||||
|
||||
### 6. Implement the handler
|
||||
|
||||
Create the handler directory and files:
|
||||
|
||||
```
|
||||
server/worldmonitor/sanctions/v1/
|
||||
├── handler.ts # thin re-export
|
||||
└── list-sanctions.ts # RPC implementation
|
||||
```
|
||||
|
||||
`server/worldmonitor/sanctions/v1/list-sanctions.ts`:
|
||||
```typescript
|
||||
import type {
|
||||
SanctionsServiceHandler,
|
||||
ServerContext,
|
||||
ListSanctionsRequest,
|
||||
ListSanctionsResponse,
|
||||
} from '../../../../src/generated/server/worldmonitor/sanctions/v1/service_server';
|
||||
|
||||
export const listSanctions: SanctionsServiceHandler['listSanctions'] = async (
|
||||
_ctx: ServerContext,
|
||||
req: ListSanctionsRequest,
|
||||
): Promise<ListSanctionsResponse> => {
|
||||
// Your implementation here — fetch from upstream API, transform to proto shape
|
||||
return { entries: [], pagination: undefined };
|
||||
};
|
||||
```
|
||||
|
||||
`server/worldmonitor/sanctions/v1/handler.ts`:
|
||||
```typescript
|
||||
import type { SanctionsServiceHandler } from '../../../../src/generated/server/worldmonitor/sanctions/v1/service_server';
|
||||
|
||||
import { listSanctions } from './list-sanctions';
|
||||
|
||||
export const sanctionsHandler: SanctionsServiceHandler = {
|
||||
listSanctions,
|
||||
};
|
||||
```
|
||||
|
||||
### 7. Register the service in the gateway
|
||||
|
||||
Edit `api/[[...path]].js` — add the import and mount the routes:
|
||||
|
||||
```typescript
|
||||
import { createSanctionsServiceRoutes } from '../src/generated/server/worldmonitor/sanctions/v1/service_server';
|
||||
import { sanctionsHandler } from './server/worldmonitor/sanctions/v1/handler';
|
||||
|
||||
const allRoutes = [
|
||||
// ... existing routes ...
|
||||
...createSanctionsServiceRoutes(sanctionsHandler, serverOptions),
|
||||
];
|
||||
```
|
||||
|
||||
### 8. Register in the Vite dev server
|
||||
|
||||
Edit `vite.config.ts` — add the lazy import and route mount inside the `sebufApiPlugin()` function. Follow the existing pattern (search for any other service to see the exact locations).
|
||||
|
||||
### 9. Create the frontend service wrapper
|
||||
|
||||
Create `src/services/sanctions.ts`:
|
||||
|
||||
```typescript
|
||||
import {
|
||||
SanctionsServiceClient,
|
||||
type SanctionsEntry,
|
||||
type ListSanctionsResponse,
|
||||
} from '@/generated/client/worldmonitor/sanctions/v1/service_client';
|
||||
import { createCircuitBreaker } from '@/utils';
|
||||
|
||||
export type { SanctionsEntry };
|
||||
|
||||
const client = new SanctionsServiceClient('', { fetch: fetch.bind(globalThis) });
|
||||
const breaker = createCircuitBreaker<ListSanctionsResponse>({ name: 'Sanctions' });
|
||||
|
||||
const emptyFallback: ListSanctionsResponse = { entries: [] };
|
||||
|
||||
export async function fetchSanctions(authority?: string): Promise<SanctionsEntry[]> {
|
||||
const response = await breaker.execute(async () => {
|
||||
return client.listSanctions({ authority: authority ?? '', countryCode: '', pagination: undefined });
|
||||
}, emptyFallback);
|
||||
return response.entries;
|
||||
}
|
||||
```
|
||||
|
||||
### 10. Verify
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit # zero errors
|
||||
```
|
||||
|
||||
## Proto conventions
|
||||
|
||||
These conventions are enforced across the codebase. Follow them for consistency.
|
||||
|
||||
### File naming
|
||||
|
||||
- One file per message type: `earthquake.proto`, `sanctions_entry.proto`
|
||||
- One file per RPC pair: `list_earthquakes.proto`, `get_earthquake_details.proto`
|
||||
- Service definition: `service.proto`
|
||||
- Use `snake_case` for file names and field names
|
||||
|
||||
### Time fields
|
||||
|
||||
Always use `int64` with Unix epoch milliseconds. Never use `google.protobuf.Timestamp`.
|
||||
|
||||
Always add the `INT64_ENCODING_NUMBER` annotation so TypeScript gets `number` instead of `string`:
|
||||
|
||||
```protobuf
|
||||
int64 occurred_at = 6 [(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER];
|
||||
```
|
||||
|
||||
### Validation annotations
|
||||
|
||||
Import `buf/validate/validate.proto` and annotate fields at the proto level. These constraints flow through to the generated OpenAPI spec automatically.
|
||||
|
||||
Common patterns:
|
||||
|
||||
```protobuf
|
||||
// Required string with length bounds
|
||||
string id = 1 [
|
||||
(buf.validate.field).required = true,
|
||||
(buf.validate.field).string.min_len = 1,
|
||||
(buf.validate.field).string.max_len = 100
|
||||
];
|
||||
|
||||
// Numeric range (e.g., score 0-100)
|
||||
double risk_score = 2 [
|
||||
(buf.validate.field).double.gte = 0,
|
||||
(buf.validate.field).double.lte = 100
|
||||
];
|
||||
|
||||
// Non-negative value
|
||||
double min_magnitude = 3 [(buf.validate.field).double.gte = 0];
|
||||
|
||||
// Coordinate bounds (prefer using core.v1.GeoCoordinates instead)
|
||||
double latitude = 1 [
|
||||
(buf.validate.field).double.gte = -90,
|
||||
(buf.validate.field).double.lte = 90
|
||||
];
|
||||
```
|
||||
|
||||
### Shared core types
|
||||
|
||||
Reuse these instead of redefining:
|
||||
|
||||
| Type | Import | Use for |
|
||||
|------|--------|---------|
|
||||
| `GeoCoordinates` | `worldmonitor/core/v1/geo.proto` | Any lat/lon location (has built-in -90/90 and -180/180 bounds) |
|
||||
| `BoundingBox` | `worldmonitor/core/v1/geo.proto` | Spatial filtering |
|
||||
| `TimeRange` | `worldmonitor/core/v1/time.proto` | Time-based filtering (has `INT64_ENCODING_NUMBER`) |
|
||||
| `PaginationRequest` | `worldmonitor/core/v1/pagination.proto` | Request pagination (has page_size 1-100 constraint) |
|
||||
| `PaginationResponse` | `worldmonitor/core/v1/pagination.proto` | Response pagination metadata |
|
||||
|
||||
### Comments
|
||||
|
||||
buf lint enforces comments on all messages, fields, services, RPCs, and enum values. Every proto element must have a `//` comment. This is not optional — `buf lint` will fail without them.
|
||||
|
||||
### Route paths
|
||||
|
||||
- Service base path: `/api/{domain}/v1`
|
||||
- RPC path: `/{verb}-{noun}` in kebab-case (e.g., `/list-earthquakes`, `/get-vessel-snapshot`)
|
||||
|
||||
### Handler typing
|
||||
|
||||
Always type the handler function against the generated interface using indexed access:
|
||||
|
||||
```typescript
|
||||
export const listSanctions: SanctionsServiceHandler['listSanctions'] = async (
|
||||
_ctx: ServerContext,
|
||||
req: ListSanctionsRequest,
|
||||
): Promise<ListSanctionsResponse> => {
|
||||
// ...
|
||||
};
|
||||
```
|
||||
|
||||
This ensures the compiler catches any mismatch between your implementation and the proto contract.
|
||||
|
||||
### Client construction
|
||||
|
||||
Always pass `{ fetch: fetch.bind(globalThis) }` when creating clients:
|
||||
|
||||
```typescript
|
||||
const client = new SanctionsServiceClient('', { fetch: fetch.bind(globalThis) });
|
||||
```
|
||||
|
||||
The empty string base URL works because both Vite dev server and Vercel serve the API on the same origin. The `fetch.bind(globalThis)` is required for Tauri compatibility.
|
||||
|
||||
## Generated documentation
|
||||
|
||||
Every time you run `make generate`, OpenAPI v3 specs are generated for each service:
|
||||
|
||||
- `docs/api/{Domain}Service.openapi.yaml` — human-readable YAML
|
||||
- `docs/api/{Domain}Service.openapi.json` — machine-readable JSON
|
||||
|
||||
These specs include:
|
||||
|
||||
- All endpoints with request/response schemas
|
||||
- Validation constraints from `buf.validate` annotations (min/max, required fields, ranges)
|
||||
- Field descriptions from proto comments
|
||||
- Error response schemas (400 validation errors, 500 server errors)
|
||||
|
||||
You do not need to write or maintain OpenAPI specs by hand. They are generated artifacts. If you need to change the API documentation, change the proto and regenerate.
|
||||
@@ -0,0 +1,140 @@
|
||||
# API Key Gating & Registration — Deployment Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Desktop cloud fallback is gated on a `WORLDMONITOR_API_KEY`. Without a valid key, the desktop app operates local-only (sidecar). A registration form collects emails via Convex DB for future key distribution.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Desktop App Cloud (Vercel)
|
||||
┌──────────────────┐ ┌──────────────────────┐
|
||||
│ fetch('/api/...')│ │ api/[domain]/v1/[rpc]│
|
||||
│ │ │ │ │ │
|
||||
│ ┌──────▼───────┐ │ │ ┌──────▼───────┐ │
|
||||
│ │ sidecar try │ │ │ │ validateApiKey│ │
|
||||
│ │ (local-first)│ │ │ │ (origin-aware)│ │
|
||||
│ └──────┬───────┘ │ │ └──────┬───────┘ │
|
||||
│ fail │ │ │ 401 if invalid │
|
||||
│ ┌──────▼───────┐ │ fallback │ │
|
||||
│ │ WM key check │─┼──────────────►│ ┌──────────────┐ │
|
||||
│ │ (gate) │ │ +header │ │ route handler │ │
|
||||
│ └──────────────┘ │ │ └──────────────┘ │
|
||||
└──────────────────┘ └──────────────────────┘
|
||||
```
|
||||
|
||||
## Required Environment Variables
|
||||
|
||||
### Vercel
|
||||
|
||||
| Variable | Description | Example |
|
||||
|----------|-------------|---------|
|
||||
| `WORLDMONITOR_VALID_KEYS` | Comma-separated list of valid API keys | `wm_abc123def456,wm_xyz789` |
|
||||
| `CONVEX_URL` | Convex deployment URL (from `npx convex deploy`) | `https://xyz-123.convex.cloud` |
|
||||
|
||||
### Generating API keys
|
||||
|
||||
Keys must be at least 16 characters (validated client-side). Recommended format:
|
||||
|
||||
```bash
|
||||
# Generate a key
|
||||
openssl rand -hex 24 | sed 's/^/wm_/'
|
||||
# Example output: wm_a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6
|
||||
```
|
||||
|
||||
Add to `WORLDMONITOR_VALID_KEYS` in Vercel dashboard (comma-separated, no spaces).
|
||||
|
||||
## Convex Setup
|
||||
|
||||
### First-time deployment
|
||||
|
||||
```bash
|
||||
# 1. Install (already in package.json)
|
||||
npm install
|
||||
|
||||
# 2. Login to Convex
|
||||
npx convex login
|
||||
|
||||
# 3. Initialize project (creates .env.local with CONVEX_URL)
|
||||
npx convex init
|
||||
|
||||
# 4. Deploy schema and functions
|
||||
npx convex deploy
|
||||
|
||||
# 5. Copy the deployment URL to Vercel env vars
|
||||
# The URL is printed by `npx convex deploy` and saved in .env.local
|
||||
```
|
||||
|
||||
### Verify Convex deployment
|
||||
|
||||
```bash
|
||||
# Typecheck Convex functions
|
||||
npx convex dev --typecheck
|
||||
|
||||
# Open Convex dashboard to see registrations
|
||||
npx convex dashboard
|
||||
```
|
||||
|
||||
### Schema
|
||||
|
||||
The `registrations` table stores:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `email` | string | Original email (for display) |
|
||||
| `normalizedEmail` | string | Lowercased email (for dedup) |
|
||||
| `registeredAt` | number | Unix timestamp |
|
||||
| `source` | string? | Where the registration came from |
|
||||
| `appVersion` | string? | Desktop app version |
|
||||
|
||||
Indexed by `normalizedEmail` for duplicate detection.
|
||||
|
||||
## Security Model
|
||||
|
||||
### Client-side (desktop app)
|
||||
|
||||
- `installRuntimeFetchPatch()` checks `WORLDMONITOR_API_KEY` before allowing cloud fallback
|
||||
- Key must be present AND valid (min 16 chars)
|
||||
- `secretsReady` promise ensures secrets are loaded before first fetch (2s timeout)
|
||||
- Fail-closed: any error in key check blocks cloud fallback
|
||||
|
||||
### Server-side (Vercel edge)
|
||||
|
||||
- `api/_api-key.js` validates `X-WorldMonitor-Key` header on sebuf routes
|
||||
- **Origin-aware**: desktop origins (`tauri.localhost`, `tauri://`, `asset://`) require a key
|
||||
- Web origins (`worldmonitor.app`) pass through without a key
|
||||
- Non-desktop origin with key header: key is still validated
|
||||
- Invalid key returns `401 { error: "Invalid API key" }`
|
||||
|
||||
### CORS
|
||||
|
||||
`X-WorldMonitor-Key` is allowed in both `server/cors.ts` and `api/_cors.js`.
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After deployment:
|
||||
|
||||
- [ ] Set `WORLDMONITOR_VALID_KEYS` in Vercel
|
||||
- [ ] Set `CONVEX_URL` in Vercel
|
||||
- [ ] Run `npx convex deploy` to push schema
|
||||
- [ ] Desktop without key: cloud fallback blocked (console shows `cloud fallback blocked`)
|
||||
- [ ] Desktop with invalid key: sebuf requests get `401`
|
||||
- [ ] Desktop with valid key: cloud fallback works as before
|
||||
- [ ] Web access: no key required, works normally
|
||||
- [ ] Registration form: submit email, check Convex dashboard
|
||||
- [ ] Duplicate email: shows "already registered"
|
||||
- [ ] Existing settings tabs (LLMs, API Keys, Debug) unchanged
|
||||
|
||||
## Files Reference
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `src/services/runtime.ts` | Client-side key gate + header attachment |
|
||||
| `src/services/runtime-config.ts` | `WORLDMONITOR_API_KEY` type, validation, `secretsReady` |
|
||||
| `api/_api-key.js` | Server-side key validation (origin-aware) |
|
||||
| `api/[domain]/v1/[rpc].ts` | Sebuf gateway — calls `validateApiKey` |
|
||||
| `api/register-interest.js` | Registration endpoint → Convex |
|
||||
| `server/cors.ts` / `api/_cors.js` | CORS headers with `X-WorldMonitor-Key` |
|
||||
| `src/components/WorldMonitorTab.ts` | Settings UI for key + registration |
|
||||
| `convex/schema.ts` | Convex DB schema |
|
||||
| `convex/registerInterest.ts` | Convex mutation |
|
||||
@@ -0,0 +1,184 @@
|
||||
# World Monitor — Community Promotion Guide
|
||||
|
||||
Thank you for helping spread the word about World Monitor! This guide provides talking points, must-see features, and visual suggestions to help you create compelling content for your audience.
|
||||
|
||||
---
|
||||
|
||||
## What is World Monitor?
|
||||
|
||||
**One-line pitch**: A free, open-source, real-time global intelligence dashboard — like Bloomberg Terminal meets OSINT, for everyone.
|
||||
|
||||
**Longer description**: World Monitor aggregates 150+ news feeds, military tracking, financial markets, conflict data, protest monitoring, satellite imagery, and AI-powered analysis into a single unified dashboard with an interactive globe. Available as a web app, desktop app (macOS/Windows/Linux), and installable PWA.
|
||||
|
||||
---
|
||||
|
||||
## Key URLs
|
||||
|
||||
| Link | Description |
|
||||
|------|-------------|
|
||||
| [worldmonitor.app](https://worldmonitor.app) | Main dashboard — geopolitics, military, conflicts |
|
||||
| [tech.worldmonitor.app](https://tech.worldmonitor.app) | Tech variant — startups, AI/ML, cybersecurity |
|
||||
| [finance.worldmonitor.app](https://finance.worldmonitor.app) | Finance variant — markets, exchanges, central banks |
|
||||
| [GitHub](https://github.com/koala73/worldmonitor) | Source code (AGPL-3.0) |
|
||||
|
||||
---
|
||||
|
||||
## Must-See Features (Top 10)
|
||||
|
||||
### 1. Interactive Globe with 35+ Data Layers
|
||||
|
||||
The centerpiece. A WebGL-accelerated globe (deck.gl) with toggleable layers for conflicts, military bases, nuclear facilities, undersea cables, pipelines, satellite fires, protests, cyber threats, and more. Zoom in and the detail layers progressively reveal.
|
||||
|
||||
**Show**: Toggle different layers on/off. Zoom into a conflict region. Show the layer panel.
|
||||
|
||||
### 2. AI-Powered World Brief
|
||||
|
||||
One-click AI summary of the top global developments. Three-tier LLM provider chain: local Ollama/LM Studio (fully private, offline), Groq (fast cloud), or OpenRouter (fallback). Redis caching for instant responses on repeat queries.
|
||||
|
||||
**Show**: The summary card at the top of the news panel.
|
||||
|
||||
### 3. Country Intelligence Dossiers
|
||||
|
||||
Click any country on the map for a full-page intelligence brief: instability score ring, AI-generated analysis, top headlines, prediction markets, 7-day event timeline, active signal chips, infrastructure exposure, and stock market data.
|
||||
|
||||
**Show**: Click a country (e.g., Japan, Ukraine, or Iran) → full dossier page.
|
||||
|
||||
### 4. 14 Languages Support
|
||||
|
||||
Full UI in 14 languages including Japanese. Regional news feeds auto-adapt — Japanese users see NHK World, Nikkei Asia, and Japan-relevant sources. Language bundles are lazy-loaded for fast performance.
|
||||
|
||||
**Show**: Switch language to Japanese in the settings. Note how feeds change.
|
||||
|
||||
### 5. Live Military Tracking
|
||||
|
||||
Real-time ADS-B military flight tracking and AIS naval vessel monitoring. Strategic Posture panel shows theater-level risk assessment across 9 global regions (Baltic, Black Sea, South China Sea, Eastern Mediterranean, etc.).
|
||||
|
||||
**Show**: Enable the Military layer. Show the Strategic Posture panel.
|
||||
|
||||
### 6. Three Variant Dashboards
|
||||
|
||||
One codebase, three specialized views — switch between World (geopolitics), Tech (startups/AI), and Finance (markets/exchanges) with one click in the header bar.
|
||||
|
||||
**Show**: Click the variant switcher (🌍 WORLD | 💻 TECH | 📈 FINANCE).
|
||||
|
||||
### 7. Market & Crypto Intelligence
|
||||
|
||||
7-signal macro radar with composite BUY/CASH verdict, BTC spot ETF flow tracker, stablecoin peg monitor, Fear & Greed Index, and Bitcoin technical indicators. Sparkline charts and donut gauges for visual trends.
|
||||
|
||||
**Show**: Scroll to the crypto/market panels. Point out the sparklines.
|
||||
|
||||
### 8. Live Video & Webcam Feeds
|
||||
|
||||
8 live news streams (Bloomberg, Al Jazeera, Sky News, etc.) + 19 live webcams from geopolitical hotspots across 4 regions. Idle-aware — auto-pauses after 5 minutes of inactivity.
|
||||
|
||||
**Show**: Open the video panel or webcam panel.
|
||||
|
||||
### 9. Desktop Application (Free)
|
||||
|
||||
Native app for macOS, Windows, and Linux via Tauri. API keys stored in OS keychain (not plaintext). Local Node.js sidecar runs all 60+ API handlers offline-capable. Run local LLMs for fully private, offline AI summaries.
|
||||
|
||||
**Show**: The download buttons on the site, or the desktop app running natively.
|
||||
|
||||
### 10. Story Sharing & Social Export
|
||||
|
||||
Generate intelligence briefs for any country and share to Twitter/X, LinkedIn, WhatsApp, Telegram, Reddit. Includes canvas-rendered PNG images with QR codes linking back to the live dashboard.
|
||||
|
||||
**Show**: Generate a story for a country → share dialog with platform options.
|
||||
|
||||
### 11. Local LLM Support (Ollama / LM Studio)
|
||||
|
||||
Run AI summarization entirely on your own hardware — no API keys, no cloud, no data leaving your machine. The desktop app auto-discovers models from Ollama or LM Studio, with a three-tier fallback chain: local → Groq → OpenRouter. Settings are split into dedicated LLMs and API Keys tabs for easy configuration.
|
||||
|
||||
**Show**: Open Settings → LLMs tab → Ollama model dropdown auto-populated → generate a summary with the local model.
|
||||
|
||||
---
|
||||
|
||||
## Visual Content Suggestions
|
||||
|
||||
### Screenshots Worth Taking
|
||||
|
||||
1. **Full dashboard overview** — globe in center, panels on sides, news feed visible
|
||||
2. **Country dossier page** — click Japan or a hotspot country, show the full brief
|
||||
3. **Layer toggle demo** — before/after with conflicts + military bases enabled
|
||||
4. **Finance variant** — stock exchanges, financial centers, market panels
|
||||
5. **Japanese UI** — show the language switcher and Japanese interface
|
||||
6. **Webcam grid** — 4 live feeds from different regions
|
||||
7. **Strategic Posture** — theater risk levels panel
|
||||
8. **Settings LLMs tab** — Ollama model dropdown with local models discovered
|
||||
|
||||
### Video/GIF Ideas
|
||||
|
||||
1. **30-second tour**: Open site → rotate globe → toggle layers → click country → show brief
|
||||
2. **Language switch**: English → Japanese, show how feeds adapt
|
||||
3. **Layer stacking**: Start empty → add conflicts → military → cyber → fires → wow
|
||||
4. **Variant switching**: World → Tech → Finance in quick succession
|
||||
|
||||
---
|
||||
|
||||
## Talking Points for Posts
|
||||
|
||||
### For General Audience
|
||||
|
||||
- "An open-source Bloomberg Terminal for everyone — free, no login required"
|
||||
- "150+ news sources, military tracking, AI analysis — all in one dashboard"
|
||||
- "Run AI summaries locally with Ollama — your data never leaves your machine"
|
||||
- "Available in Japanese with NHK and Nikkei feeds built in"
|
||||
- "Native desktop app for macOS/Windows/Linux, completely free"
|
||||
|
||||
### For Tech Audience
|
||||
|
||||
- "Built with TypeScript, Vite, deck.gl, MapLibre GL, Tauri"
|
||||
- "35+ WebGL data layers running at 60fps"
|
||||
- "ONNX Runtime Web for browser-based ML inference (sentiment, NER, summarization)"
|
||||
- "Local LLM support — plug in Ollama or LM Studio, zero cloud dependency"
|
||||
- "Open source under AGPL-3.0 — contribute on GitHub"
|
||||
|
||||
### For Finance/OSINT Audience
|
||||
|
||||
- "7-signal crypto macro radar with BUY/CASH composite verdict"
|
||||
- "92 global stock exchanges mapped with market caps and trading hours"
|
||||
- "Country Instability Index tracking 22 nations in real-time"
|
||||
- "Prediction market integration for geopolitical forecasting"
|
||||
- "Air-gapped AI analysis — run Ollama locally for sensitive intelligence work"
|
||||
|
||||
### For Japanese Audience Specifically
|
||||
|
||||
- 日本語完全対応 — UI、ニュースフィード、AI要約すべて日本語で利用可能
|
||||
- NHK World、日経アジアなど日本向けニュースソース内蔵
|
||||
- 無料・オープンソース — アカウント登録不要
|
||||
- macOS/Windows/Linux対応のデスクトップアプリあり
|
||||
|
||||
---
|
||||
|
||||
## Recent Major Features (Changelog Highlights)
|
||||
|
||||
| Version | Feature |
|
||||
|---------|---------|
|
||||
| v2.5.1 | Batch FRED fetching, parallel UCDP, partial cache TTL, bot middleware |
|
||||
| v2.5.0 | Ollama/LM Studio local LLM support, settings split into LLMs + API Keys tabs, keychain vault consolidation |
|
||||
| v2.4.1 | Ultra-wide layout (panels wrap around map on 2000px+ screens) |
|
||||
| v2.4.0 | Live webcams from 19 geopolitical hotspots, 4 regions |
|
||||
| v2.3.9 | Full i18n: 14 languages including Japanese, Arabic (RTL), Chinese |
|
||||
| v2.3.8 | Finance variant with 92 exchanges, Gulf FDI investments |
|
||||
| v2.3.7 | Light/dark theme system, UCDP/UNHCR/Climate panels |
|
||||
| v2.3.6 | Desktop app with Tauri, OS keychain, auto-updates |
|
||||
| v2.3.0 | Country Intelligence dossiers, story sharing |
|
||||
|
||||
---
|
||||
|
||||
## Branding Notes
|
||||
|
||||
- **Name**: "World Monitor" (two words, capitalized)
|
||||
- **Tagline**: "Real-time global intelligence dashboard"
|
||||
- **License**: AGPL-3.0 (free and open source)
|
||||
- **Creator**: Credit "World Monitor by Elie Habib" or link to the GitHub repo
|
||||
- **Variants**: You can mention all three (World/Tech/Finance) or focus on the main one
|
||||
- **No login required**: Anyone can use the web app immediately — no signup, no paywall
|
||||
|
||||
---
|
||||
|
||||
## Thank You
|
||||
|
||||
We genuinely appreciate community members helping grow World Monitor's reach. Feel free to interpret these guidelines creatively — there's no strict template. The most compelling content comes from showing what YOU find most interesting or useful about the tool.
|
||||
|
||||
If you have questions or want specific screenshots/assets, open a Discussion on the GitHub repo or reach out directly.
|
||||
@@ -4,7 +4,7 @@ World Monitor desktop now uses a runtime configuration schema with per-feature t
|
||||
|
||||
## Secret keys
|
||||
|
||||
The desktop vault schema supports the following 17 keys used by services and relays:
|
||||
The desktop vault schema (Rust `SUPPORTED_SECRET_KEYS`) supports the following 22 keys:
|
||||
|
||||
- `GROQ_API_KEY`
|
||||
- `OPENROUTER_API_KEY`
|
||||
@@ -18,11 +18,18 @@ The desktop vault schema supports the following 17 keys used by services and rel
|
||||
- `ABUSEIPDB_API_KEY`
|
||||
- `NASA_FIRMS_API_KEY`
|
||||
- `WINGBITS_API_KEY`
|
||||
- `WS_RELAY_URL`
|
||||
- `VITE_WS_RELAY_URL`
|
||||
- `VITE_OPENSKY_RELAY_URL`
|
||||
- `OPENSKY_CLIENT_ID`
|
||||
- `OPENSKY_CLIENT_SECRET`
|
||||
- `AISSTREAM_API_KEY`
|
||||
- `VITE_WS_RELAY_URL`
|
||||
- `OLLAMA_API_URL`
|
||||
- `OLLAMA_MODEL`
|
||||
- `WORLDMONITOR_API_KEY` — gates cloud fallback access (min 16 chars)
|
||||
- `WTO_API_KEY`
|
||||
|
||||
Note: `UC_DP_KEY` exists in the TypeScript `RuntimeSecretKey` union but is not in the desktop Rust keychain or sidecar.
|
||||
|
||||
## Feature schema
|
||||
|
||||
@@ -51,3 +58,4 @@ If required secrets are missing/disabled:
|
||||
- NASA FIRMS: satellite fire detection returns empty state.
|
||||
- Wingbits: flight enrichment disabled, heuristic-only flight classification remains.
|
||||
- AIS / OpenSky relay: live tracking features are disabled cleanly.
|
||||
- WorldMonitor API key: cloud fallback is blocked; desktop operates local-only.
|
||||
|
||||
+31
-17
@@ -7,7 +7,7 @@ AI-powered real-time global intelligence dashboard aggregating news, markets, ge
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
||||

|
||||
|
||||
@@ -3298,13 +3298,15 @@ const header = `World Monitor v${__APP_VERSION__}`;
|
||||
|
||||
## Installation
|
||||
|
||||
**Requirements:** Go 1.21+ and Node.js 18+.
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/koala73/worldmonitor.git
|
||||
cd worldmonitor
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
# Install everything (buf, sebuf plugins, npm deps, proto deps)
|
||||
make install
|
||||
|
||||
# Start development server
|
||||
npm run dev
|
||||
@@ -3313,6 +3315,14 @@ npm run dev
|
||||
npm run build
|
||||
```
|
||||
|
||||
If you modify any `.proto` files, regenerate before building or pushing:
|
||||
|
||||
```bash
|
||||
make generate # regenerate TypeScript clients, servers, and OpenAPI docs
|
||||
```
|
||||
|
||||
See [ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md) for the full proto workflow.
|
||||
|
||||
## API Dependencies
|
||||
|
||||
The dashboard fetches data from various public APIs and data sources:
|
||||
@@ -3642,7 +3652,7 @@ The system degrades gracefully—blocked sources are skipped while others contin
|
||||
|
||||
## Roadmap
|
||||
|
||||
See [ROADMAP.md](ROADMAP.md) for detailed planning. Recent intelligence enhancements:
|
||||
See [ROADMAP.md](../.planning/ROADMAP.md) for detailed planning. Recent intelligence enhancements:
|
||||
|
||||
### Completed
|
||||
|
||||
@@ -3698,7 +3708,7 @@ See [ROADMAP.md](ROADMAP.md) for detailed planning. Recent intelligence enhancem
|
||||
- **Additional Data Sources** - World Bank, IMF, OFAC sanctions, UNHCR refugee data, FAO food security
|
||||
- **Think Tank Feeds** - RUSI, Chatham House, ECFR, CFR, Wilson Center, CNAS, Arms Control Association
|
||||
|
||||
The full [ROADMAP.md](ROADMAP.md) documents implementation details, API endpoints, and 30+ free data sources for future integration.
|
||||
The full [ROADMAP.md](../.planning/ROADMAP.md) documents implementation details, API endpoints, and 30+ free data sources for future integration.
|
||||
|
||||
---
|
||||
|
||||
@@ -3994,20 +4004,24 @@ PRs that don't follow the code style or introduce security issues will be asked
|
||||
|
||||
### Development Tips
|
||||
|
||||
**Adding or Modifying API Endpoints**
|
||||
|
||||
All JSON API endpoints **must** use sebuf. Do not create standalone `api/*.js` files — the legacy pattern is deprecated.
|
||||
|
||||
See **[docs/ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md)** for the complete guide covering:
|
||||
|
||||
- Adding an RPC to an existing service
|
||||
- Adding an entirely new service
|
||||
- Proto conventions (validation, time fields, shared types)
|
||||
- Generated OpenAPI documentation
|
||||
|
||||
**Adding a New Data Layer**
|
||||
|
||||
1. Create service in `src/services/` for data fetching
|
||||
2. Add layer toggle in `src/components/Map.ts`
|
||||
3. Add rendering logic for map markers/overlays
|
||||
4. Add to help panel documentation
|
||||
5. Update README with layer description
|
||||
|
||||
**Adding a New API Proxy**
|
||||
|
||||
1. Create handler in `api/` directory
|
||||
2. Implement input validation (see existing proxies)
|
||||
3. Add appropriate cache headers
|
||||
4. Document any required environment variables
|
||||
1. Define the proto contract and generate code (see [ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md))
|
||||
2. Implement the handler in `server/worldmonitor/{domain}/v1/`
|
||||
3. Create the frontend service wrapper in `src/services/`
|
||||
4. Add layer toggle in `src/components/Map.ts`
|
||||
5. Add rendering logic for map markers/overlays
|
||||
|
||||
**Debugging**
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,937 @@
|
||||
# Architecture
|
||||
|
||||
World Monitor is an AI-powered real-time global intelligence dashboard built as a TypeScript single-page application. It aggregates 30+ external data sources — covering geopolitics, military activity, financial markets, cyber threats, climate events, and more — into a unified operational picture rendered through an interactive 3D globe and a grid of specialised panels.
|
||||
|
||||
This document covers the full system architecture: deployment topology, variant configuration, data pipelines, signal intelligence, map rendering, caching, desktop packaging, machine-learning inference, and error handling.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [High-Level System Diagram](#1-high-level-system-diagram)
|
||||
2. [Variant Architecture](#2-variant-architecture)
|
||||
3. [Data Flow: RSS Ingestion to Display](#3-data-flow-rss-ingestion-to-display)
|
||||
4. [Signal Intelligence Pipeline](#4-signal-intelligence-pipeline)
|
||||
5. [Map Rendering Pipeline](#5-map-rendering-pipeline)
|
||||
6. [Caching Architecture](#6-caching-architecture)
|
||||
7. [Desktop Architecture](#7-desktop-architecture)
|
||||
8. [ML Pipeline](#8-ml-pipeline)
|
||||
9. [Error Handling Hierarchy](#9-error-handling-hierarchy)
|
||||
|
||||
---
|
||||
|
||||
## 1. High-Level System Diagram
|
||||
|
||||
The system follows a classic edge-compute pattern: a static SPA served from a CDN communicates with serverless API endpoints that proxy, normalise, and cache upstream data.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Browser
|
||||
SPA["TypeScript SPA<br/>(Vite 6, class-based)"]
|
||||
SW["Service Worker<br/>(Workbox)"]
|
||||
IDB["IndexedDB<br/>(snapshots & baselines)"]
|
||||
MLW["ML Web Worker<br/>(ONNX / Transformers.js)"]
|
||||
SPA --> SW
|
||||
SPA --> IDB
|
||||
SPA --> MLW
|
||||
end
|
||||
|
||||
subgraph Vercel["Vercel Edge Functions"]
|
||||
API["60+ API Endpoints<br/>(api/ directory, plain JS)"]
|
||||
end
|
||||
|
||||
subgraph External["External APIs (30+)"]
|
||||
RSS["RSS Feeds"]
|
||||
ACLED["ACLED"]
|
||||
UCDP["UCDP"]
|
||||
GDELT["GDELT"]
|
||||
OpenSky["OpenSky"]
|
||||
Finnhub["Finnhub"]
|
||||
Yahoo["Yahoo Finance"]
|
||||
FRED["FRED"]
|
||||
CoinGecko["CoinGecko"]
|
||||
Polymarket["Polymarket"]
|
||||
FIRMS["NASA FIRMS"]
|
||||
GROQ["Groq / OpenRouter"]
|
||||
Others["+ 20 more"]
|
||||
end
|
||||
|
||||
subgraph Cache["Upstash Redis"]
|
||||
Redis["Server-side<br/>API Response Cache"]
|
||||
end
|
||||
|
||||
subgraph Desktop["Tauri Desktop Shell"]
|
||||
Tauri["Tauri 2 (Rust)"]
|
||||
Sidecar["Node.js Sidecar<br/>127.0.0.1:46123"]
|
||||
Tauri --> Sidecar
|
||||
end
|
||||
|
||||
SPA <-->|"fetch()"| API
|
||||
SPA <-->|"Tauri IPC"| Tauri
|
||||
SPA <-->|"fetch()"| Sidecar
|
||||
API <--> Redis
|
||||
API <--> RSS
|
||||
API <--> ACLED
|
||||
API <--> UCDP
|
||||
API <--> GDELT
|
||||
API <--> OpenSky
|
||||
API <--> Finnhub
|
||||
API <--> Yahoo
|
||||
API <--> FRED
|
||||
API <--> CoinGecko
|
||||
API <--> Polymarket
|
||||
API <--> FIRMS
|
||||
API <--> GROQ
|
||||
API <--> Others
|
||||
```
|
||||
|
||||
### Component Summary
|
||||
|
||||
| Layer | Technology | Role |
|
||||
|---|---|---|
|
||||
| **SPA** | TypeScript, Vite 6, no framework | UI rendering via class-based components extending a `Panel` base class. 44 panels in the full variant. |
|
||||
| **Vercel Edge Functions** | Plain JS (60+ files in api/) | Proxy, normalise, and cache upstream API calls. Each file exports a default Vercel handler. |
|
||||
| **External APIs** | 30+ heterogeneous sources | RSS feeds, conflict databases (ACLED, UCDP), geospatial (GDELT, NASA FIRMS, OpenSky), markets (Finnhub, Yahoo Finance, CoinGecko), LLMs (Groq, OpenRouter), and more. |
|
||||
| **Upstash Redis** | Redis REST API | Server-side response cache with TTL-based expiry. Falls back to in-memory Map in sidecar mode. |
|
||||
| **Service Worker** | Workbox | Offline support, runtime caching strategies, background sync. |
|
||||
| **IndexedDB** | `worldmonitor_db` | Client-side storage for playback snapshots and temporal baseline data. |
|
||||
| **Tauri Shell** | Tauri 2 (Rust) + Node.js sidecar | Desktop packaging. Sidecar runs a local API server; Rust layer provides OS keychain, window management, and IPC. |
|
||||
| **ML Worker** | Web Worker + ONNX Runtime / Transformers.js | In-browser inference for embeddings, sentiment, summarisation, and NER. |
|
||||
|
||||
---
|
||||
|
||||
## 2. Variant Architecture
|
||||
|
||||
World Monitor ships as three product variants from a single codebase. Each variant surfaces a different subset of panels, map layers, and data sources.
|
||||
|
||||
| Variant | Domain | Focus |
|
||||
|---|---|---|
|
||||
| `full` | worldmonitor.app | Geopolitics, military, OSINT, conflicts, markets |
|
||||
| `tech` | tech.worldmonitor.app | AI/ML, startups, cybersecurity, developer tools |
|
||||
| `finance` | finance.worldmonitor.app | Markets, trading, central banks, macro indicators |
|
||||
|
||||
### Variant Resolution
|
||||
|
||||
The active variant is resolved at startup in src/config/variant.ts via a strict priority chain:
|
||||
|
||||
```
|
||||
localStorage('worldmonitor-variant') → import.meta.env.VITE_VARIANT → default 'full'
|
||||
```
|
||||
|
||||
The exported constant `SITE_VARIANT` is computed once as an IIFE:
|
||||
|
||||
```typescript
|
||||
export const SITE_VARIANT: string = (() => {
|
||||
if (typeof window !== 'undefined') {
|
||||
const stored = localStorage.getItem('worldmonitor-variant');
|
||||
if (stored === 'tech' || stored === 'full' || stored === 'finance') return stored;
|
||||
}
|
||||
return import.meta.env.VITE_VARIANT || 'full';
|
||||
})();
|
||||
```
|
||||
|
||||
The `localStorage` override enables runtime variant switching on the settings page without a rebuild. The `VITE_VARIANT` env var is set at deploy time (one Vercel project per subdomain).
|
||||
|
||||
### Configuration Tree-Shaking
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph ConfigTree["src/config/variants/"]
|
||||
Base["base.ts<br/>VariantConfig interface<br/>API_URLS, REFRESH_INTERVALS<br/>STORAGE_KEYS, MONITOR_COLORS"]
|
||||
Full["full.ts<br/>VARIANT_CONFIG"]
|
||||
Tech["tech.ts<br/>VARIANT_CONFIG"]
|
||||
Finance["finance.ts<br/>VARIANT_CONFIG"]
|
||||
Base --> Full
|
||||
Base --> Tech
|
||||
Base --> Finance
|
||||
end
|
||||
|
||||
subgraph Panels["src/config/panels.ts"]
|
||||
FP["FULL_PANELS (44)"]
|
||||
FM["FULL_MAP_LAYERS (35+)"]
|
||||
FMM["FULL_MOBILE_MAP_LAYERS"]
|
||||
TP["TECH_PANELS"]
|
||||
TM["TECH_MAP_LAYERS"]
|
||||
FiP["FINANCE_PANELS"]
|
||||
FiM["FINANCE_MAP_LAYERS"]
|
||||
end
|
||||
|
||||
Variant["SITE_VARIANT"] --> Switch{"Ternary switch"}
|
||||
Switch -->|"full"| FP
|
||||
Switch -->|"tech"| TP
|
||||
Switch -->|"finance"| FiP
|
||||
|
||||
Switch --> DefaultPanels["DEFAULT_PANELS"]
|
||||
Switch --> DefaultLayers["DEFAULT_MAP_LAYERS"]
|
||||
Switch --> MobileLayers["MOBILE_DEFAULT_MAP_LAYERS"]
|
||||
```
|
||||
|
||||
The `VariantConfig` interface in src/config/variants/base.ts defines the shape:
|
||||
|
||||
```typescript
|
||||
interface VariantConfig {
|
||||
name: string;
|
||||
description: string;
|
||||
panels: Record<string, PanelConfig>;
|
||||
mapLayers: MapLayers;
|
||||
mobileMapLayers: MapLayers;
|
||||
}
|
||||
```
|
||||
|
||||
Each variant file (full.ts, tech.ts, finance.ts) exports a `VARIANT_CONFIG` conforming to this interface. The shared base re-exports common constants: `API_URLS`, `REFRESH_INTERVALS`, `STORAGE_KEYS`, `MONITOR_COLORS`, `SECTORS`, `COMMODITIES`, `MARKET_SYMBOLS`, `UNDERSEA_CABLES`, and `AI_DATA_CENTERS`.
|
||||
|
||||
At build time, Vite's tree-shaking eliminates the unused variant configs. If `VITE_VARIANT=tech`, the full and finance panel definitions are dead-code-eliminated from the production bundle.
|
||||
|
||||
At runtime, src/config/panels.ts selects the active config via ternary expressions:
|
||||
|
||||
```typescript
|
||||
export const DEFAULT_PANELS = SITE_VARIANT === 'tech'
|
||||
? TECH_PANELS
|
||||
: SITE_VARIANT === 'finance'
|
||||
? FINANCE_PANELS
|
||||
: FULL_PANELS;
|
||||
```
|
||||
|
||||
The same pattern applies to `DEFAULT_MAP_LAYERS` and `MOBILE_DEFAULT_MAP_LAYERS`.
|
||||
|
||||
### Panel and Layer Counts
|
||||
|
||||
| Variant | Panels | Desktop Map Layers | Mobile Map Layers |
|
||||
|---|---|---|---|
|
||||
| `full` | 44 | 35+ | Reduced subset |
|
||||
| `tech` | ~20 | Tech-focused layers (cloud regions, startup hubs, accelerators) | Minimal |
|
||||
| `finance` | ~18 | Finance-focused layers (stock exchanges, financial centres, central banks) | Minimal |
|
||||
|
||||
The `MapLayers` interface contains 35+ boolean toggle keys including: `conflicts`, `bases`, `cables`, `pipelines`, `hotspots`, `ais`, `nuclear`, `irradiators`, `sanctions`, `weather`, `economic`, `waterways`, `outages`, `cyberThreats`, `datacenters`, `protests`, `flights`, `military`, `natural`, `spaceports`, `minerals`, `fires`, `ucdpEvents`, `displacement`, `climate`, `startupHubs`, `cloudRegions`, `accelerators`, `techHQs`, `techEvents`, `stockExchanges`, `financialCenters`, `centralBanks`, `commodityHubs`, and `gulfInvestments`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Data Flow: RSS Ingestion to Display
|
||||
|
||||
The core intelligence pipeline transforms raw RSS feeds into clustered, classified, and scored events displayed across panels. This pipeline runs entirely in the browser.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant RSS as RSS Sources
|
||||
participant Proxy as /api/rss-proxy
|
||||
participant Cache as Upstash Redis
|
||||
participant SPA as Browser SPA
|
||||
participant Cluster as clustering.ts
|
||||
participant ML as ML Worker
|
||||
participant Threat as threat-classifier.ts
|
||||
participant Entity as entity-extraction.ts
|
||||
participant Panel as Panel Components
|
||||
|
||||
SPA->>Proxy: fetch(feedUrl)
|
||||
Proxy->>Cache: getCachedJson(key)
|
||||
alt Cache hit
|
||||
Cache-->>Proxy: cached response
|
||||
else Cache miss
|
||||
Proxy->>RSS: GET feed XML/JSON
|
||||
RSS-->>Proxy: raw feed data
|
||||
Proxy->>Cache: setCachedJson(key, data, ttl)
|
||||
end
|
||||
Proxy-->>SPA: NewsItem[]
|
||||
|
||||
SPA->>Cluster: clusterNews(items)
|
||||
Note over Cluster: Jaccard similarity<br/>on title token sets
|
||||
|
||||
alt ML Worker available
|
||||
SPA->>Cluster: clusterNewsHybrid(items)
|
||||
Cluster->>ML: embed(clusterTexts)
|
||||
ML-->>Cluster: embeddings[][]
|
||||
Cluster->>Cluster: mergeSemanticallySimilarClusters()
|
||||
end
|
||||
|
||||
Cluster-->>SPA: ClusteredEvent[]
|
||||
SPA->>Threat: classifyCluster(event)
|
||||
Threat-->>SPA: ThreatClassification
|
||||
SPA->>Entity: extractEntitiesFromCluster(event)
|
||||
Entity-->>SPA: NewsEntityContext
|
||||
SPA->>Panel: render(scoredEvents)
|
||||
```
|
||||
|
||||
### Pipeline Stages
|
||||
|
||||
**Stage 1 — RSS Fetch** (src/services/rss.ts)
|
||||
|
||||
The `fetchFeed()` function calls the `/api/rss-proxy` endpoint, which fetches and parses upstream RSS/Atom feeds on the server side. Responses are cached in Upstash Redis (or the sidecar in-memory cache). On the client, a per-feed in-memory cache (`feedCache` Map) prevents redundant network requests within the refresh interval, and a persistent cache layer (via src/services/persistent-cache.ts) provides resilience across page reloads and desktop restarts.
|
||||
|
||||
The `fetchAllFeeds()` function orchestrates concurrent fetching across all enabled feeds with configurable `onBatch` callbacks for progressive rendering.
|
||||
|
||||
**Stage 2 — Clustering** (src/services/clustering.ts)
|
||||
|
||||
Two clustering strategies are available:
|
||||
|
||||
- `clusterNews(items)` — fast Jaccard similarity over title token sets via `clusterNewsCore()`. Groups headlines with high textual overlap into `ClusteredEvent[]`. This is the default path when ML is unavailable.
|
||||
- `clusterNewsHybrid(items)` — first runs Jaccard clustering, then refines results using semantic embeddings from the ML Worker. `mergeSemanticallySimilarClusters()` reduces fragmentation by joining clusters whose embedding centroids exceed the `semanticClusterThreshold` (default 0.75). Requires at least `minClustersForML` (5) initial clusters to activate.
|
||||
|
||||
**Stage 3 — Classification** (src/services/threat-classifier.ts)
|
||||
|
||||
Each clustered event receives a `ThreatClassification` with a `ThreatLevel` (`critical | high | medium | low | info`). The classifier uses keyword pattern matching and source-tier weighting. Threat levels map to CSS variables (`--threat-critical`, `--threat-high`, etc.) for consistent colour coding across panels.
|
||||
|
||||
**Stage 4 — Entity Extraction** (src/services/entity-extraction.ts + src/services/entity-index.ts)
|
||||
|
||||
The `extractEntitiesFromTitle()` function matches text against a pre-built entity index. The `extractEntitiesFromCluster()` function aggregates entities across all items in a cluster to produce a `NewsEntityContext` containing primary and related entities.
|
||||
|
||||
The entity index (src/services/entity-index.ts) is a multi-index structure with five `Map` lookups:
|
||||
|
||||
| Index | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `byId` | `Map<string, EntityEntry>` | Canonical lookup by entity ID |
|
||||
| `byAlias` | `Map<string, string>` | Alias-to-ID resolution (case-insensitive) |
|
||||
| `byKeyword` | `Map<string, Set<string>>` | Keyword-to-entity-IDs for text matching |
|
||||
| `bySector` | `Map<string, Set<string>>` | Sector-based grouping |
|
||||
| `byType` | `Map<string, Set<string>>` | Entity type grouping (person, org, country, etc.) |
|
||||
|
||||
**Stage 5 — Display**
|
||||
|
||||
Classified and entity-enriched events are distributed to panels. The `Panel` base class provides a consistent rendering contract. Each panel subclass (LiveNewsPanel, IntelligencePanel, etc.) decides how to filter, sort, and present events relevant to its domain.
|
||||
|
||||
---
|
||||
|
||||
## 4. Signal Intelligence Pipeline
|
||||
|
||||
The signal aggregator fuses heterogeneous geospatial data sources into a unified intelligence picture with country-level clustering and regional convergence detection.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Sources["Data Sources"]
|
||||
IO["Internet Outages"]
|
||||
MF["Military Flights<br/>(OpenSky)"]
|
||||
MV["Military Vessels<br/>(AIS)"]
|
||||
PR["Protests<br/>(ACLED)"]
|
||||
AD["AIS Disruptions"]
|
||||
SF["Satellite Fires<br/>(NASA FIRMS)"]
|
||||
TA["Temporal Anomalies<br/>(Baseline Deviations)"]
|
||||
end
|
||||
|
||||
subgraph Aggregator["SignalAggregator (src/services/signal-aggregator.ts)"]
|
||||
Extract["Signal Extraction<br/>normalise to GeoSignal"]
|
||||
Geo["Geo-Spatial Correlation<br/>country code lookup"]
|
||||
Country["Country Clustering<br/>CountrySignalCluster"]
|
||||
Regional["Regional Convergence<br/>REGION_DEFINITIONS (6 regions)"]
|
||||
Score["Convergence Scoring<br/>multi-signal co-occurrence"]
|
||||
Summary["SignalSummary<br/>AI context generation"]
|
||||
end
|
||||
|
||||
IO --> Extract
|
||||
MF --> Extract
|
||||
MV --> Extract
|
||||
PR --> Extract
|
||||
AD --> Extract
|
||||
SF --> Extract
|
||||
TA --> Extract
|
||||
|
||||
Extract --> Geo
|
||||
Geo --> Country
|
||||
Country --> Regional
|
||||
Regional --> Score
|
||||
Score --> Summary
|
||||
|
||||
Summary --> Insights["AI Insights Panel"]
|
||||
Summary --> MapVis["Map Visualisation"]
|
||||
Summary --> SignalModal["Signal Modal"]
|
||||
```
|
||||
|
||||
### Type Hierarchy
|
||||
|
||||
The pipeline defined in src/services/signal-aggregator.ts operates on a layered type system:
|
||||
|
||||
```
|
||||
SignalType (enum-like union)
|
||||
├── internet_outage
|
||||
├── military_flight
|
||||
├── military_vessel
|
||||
├── protest
|
||||
├── ais_disruption
|
||||
├── satellite_fire
|
||||
└── temporal_anomaly
|
||||
|
||||
GeoSignal (individual signal)
|
||||
├── type: SignalType
|
||||
├── country: string (ISO 3166-1 alpha-2)
|
||||
├── countryName: string
|
||||
├── lat / lon: number
|
||||
├── severity: 'low' | 'medium' | 'high'
|
||||
├── title: string
|
||||
└── timestamp: Date
|
||||
|
||||
CountrySignalCluster (per-country aggregation)
|
||||
├── country / countryName
|
||||
├── signals: GeoSignal[]
|
||||
├── signalTypes: Set<SignalType>
|
||||
├── totalCount / highSeverityCount
|
||||
└── convergenceScore: number
|
||||
|
||||
RegionalConvergence (cross-country pattern)
|
||||
├── region: string
|
||||
├── countries: string[]
|
||||
├── signalTypes: SignalType[]
|
||||
├── totalSignals: number
|
||||
└── description: string
|
||||
|
||||
SignalSummary (final output)
|
||||
├── timestamp: Date
|
||||
├── totalSignals: number
|
||||
├── byType: Record<SignalType, number>
|
||||
├── convergenceZones: RegionalConvergence[]
|
||||
├── topCountries: CountrySignalCluster[]
|
||||
└── aiContext: string
|
||||
```
|
||||
|
||||
### Region Definitions
|
||||
|
||||
The `REGION_DEFINITIONS` constant maps six monitored regions to their constituent country codes:
|
||||
|
||||
| Region | Name | Countries |
|
||||
|---|---|---|
|
||||
| `middle_east` | Middle East | IR, IL, SA, AE, IQ, SY, YE, JO, LB, KW, QA, OM, BH |
|
||||
| `east_asia` | East Asia | CN, TW, JP, KR, KP, HK, MN |
|
||||
| `south_asia` | South Asia | IN, PK, BD, AF, NP, LK, MM |
|
||||
| `europe_east` | Eastern Europe | UA, RU, BY, PL, RO, MD, HU, CZ, SK, BG |
|
||||
| `africa_north` | North Africa | EG, LY, DZ, TN, MA, SD, SS |
|
||||
| `africa_sahel` | Sahel Region | ML, NE, BF, TD, NG, CM, CF |
|
||||
|
||||
### Convergence Scoring
|
||||
|
||||
The `convergenceScore` on each `CountrySignalCluster` quantifies multi-signal co-occurrence. A high score indicates that multiple independent signal types are present in the same country within the 24-hour analysis window (`WINDOW_MS`). This score drives the AI Insights panel prioritisation and the signal modal display.
|
||||
|
||||
The `SignalAggregator` class maintains a rolling window of signals and a `WeakMap`-based source tracking for temporal anomaly provenance. Individual `ingest*()` methods (e.g., `ingestInternetOutages()`, `ingestMilitaryFlights()`) clear stale signals by type before inserting fresh data, ensuring the aggregation always reflects the latest state.
|
||||
|
||||
---
|
||||
|
||||
## 5. Map Rendering Pipeline
|
||||
|
||||
The map system combines a 2D vector tile base map (MapLibre GL JS) with a 3D WebGL overlay (deck.gl) for globe rendering, supporting 35+ toggleable data layers.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph MapStack["Map Rendering Stack"]
|
||||
Container["MapContainer.ts<br/>Layout & resize management"]
|
||||
BaseMap["Map.ts<br/>MapLibre GL JS<br/>Vector tiles, region controls"]
|
||||
DeckGL["DeckGLMap.ts<br/>deck.gl WebGL overlay<br/>3D globe & data layers"]
|
||||
Popup["MapPopup.ts<br/>Feature interaction"]
|
||||
end
|
||||
|
||||
subgraph LayerConfig["Layer Configuration"]
|
||||
Defaults["FULL_MAP_LAYERS<br/>(35+ boolean toggles)"]
|
||||
UserPref["localStorage overrides<br/>(worldmonitor-layers)"]
|
||||
URLState["URL state overrides"]
|
||||
Variant["Variant-specific defaults"]
|
||||
end
|
||||
|
||||
subgraph DataLayers["Data Layers (toggleable)"]
|
||||
Geo["Geopolitical:<br/>conflicts, bases, nuclear,<br/>sanctions, waterways"]
|
||||
Military["Military:<br/>flights, military, ais"]
|
||||
Infra["Infrastructure:<br/>cables, pipelines,<br/>datacenters, spaceports"]
|
||||
Environmental["Environmental:<br/>weather, fires, climate,<br/>natural, minerals"]
|
||||
Threat["Threat:<br/>outages, cyberThreats,<br/>protests, hotspots"]
|
||||
Data["Data Sources:<br/>ucdpEvents, displacement"]
|
||||
TechLayers["Tech:<br/>startupHubs, cloudRegions,<br/>accelerators, techHQs"]
|
||||
FinanceLayers["Finance:<br/>stockExchanges,<br/>financialCenters,<br/>centralBanks"]
|
||||
end
|
||||
|
||||
Defaults --> Merge["Layer Merge Logic"]
|
||||
UserPref --> Merge
|
||||
URLState --> Merge
|
||||
Variant --> Merge
|
||||
Merge --> ActiveLayers["Active MapLayers"]
|
||||
|
||||
ActiveLayers --> DeckGL
|
||||
Container --> BaseMap
|
||||
Container --> DeckGL
|
||||
BaseMap --> Popup
|
||||
DeckGL --> Popup
|
||||
|
||||
Geo --> DeckGL
|
||||
Military --> DeckGL
|
||||
Infra --> DeckGL
|
||||
Environmental --> DeckGL
|
||||
Threat --> DeckGL
|
||||
Data --> DeckGL
|
||||
TechLayers --> DeckGL
|
||||
FinanceLayers --> DeckGL
|
||||
```
|
||||
|
||||
### Layer Toggle Resolution
|
||||
|
||||
Map layers follow a three-tier override system:
|
||||
|
||||
1. **Variant defaults** — `FULL_MAP_LAYERS`, `TECH_MAP_LAYERS`, or `FINANCE_MAP_LAYERS` define the base layer state for each variant. The full variant enables `conflicts`, `bases`, `hotspots`, `nuclear`, `sanctions`, `weather`, `economic`, `waterways`, `outages`, and `military` by default.
|
||||
|
||||
2. **User localStorage** — Stored under the key `worldmonitor-layers`. Users toggle layers in the map controls UI, and their preferences persist across sessions.
|
||||
|
||||
3. **URL state** — Query parameters can override individual layers for shareable links and embeds.
|
||||
|
||||
The merge logic applies overrides in this order, meaning URL state has the highest priority.
|
||||
|
||||
### Mobile Adaptation
|
||||
|
||||
Mobile devices receive a reduced layer set via `MOBILE_DEFAULT_MAP_LAYERS` (variant-specific). This disables heavier layers (bases, nuclear, cables, pipelines, spaceports, minerals) that would degrade performance on constrained devices while retaining the most operationally relevant overlays (conflicts, hotspots, sanctions, weather).
|
||||
|
||||
### Rendering Pipeline
|
||||
|
||||
The rendering stack works in two layers:
|
||||
|
||||
- **MapLibre GL JS** (src/components/Map.ts) provides the base map with vector tiles, region-specific map controls, and the 2D rendering context. It handles camera management, style loading, and base interaction events.
|
||||
|
||||
- **deck.gl** (src/components/DeckGLMap.ts) overlays a WebGL context for 3D globe rendering and data-driven layers. Each toggleable layer maps to a deck.gl layer instance (ScatterplotLayer, IconLayer, ArcLayer, etc.) that is conditionally created based on the active `MapLayers` state.
|
||||
|
||||
The **MapPopup** component (src/components/MapPopup.ts) provides a unified popup system for feature interaction across both rendering layers, displaying contextual information when users click or hover over map features.
|
||||
|
||||
---
|
||||
|
||||
## 6. Caching Architecture
|
||||
|
||||
World Monitor employs a five-tier caching strategy to minimise API costs, reduce latency, and enable offline operation.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Tier1["Tier 1: Upstash Redis (Server)"]
|
||||
Redis["api/_upstash-cache.js<br/>getCachedJson() / setCachedJson()<br/>TTL-based expiry"]
|
||||
end
|
||||
|
||||
subgraph Tier1b["Tier 1b: Sidecar In-Memory Cache"]
|
||||
MemCache["In-memory Map<br/>+ disk persistence (api-cache.json)<br/>Max 5000 entries"]
|
||||
end
|
||||
|
||||
subgraph Tier2["Tier 2: Vercel CDN"]
|
||||
CDN["s-maxage headers<br/>stale-while-revalidate<br/>Edge caching"]
|
||||
end
|
||||
|
||||
subgraph Tier3["Tier 3: Service Worker"]
|
||||
Workbox["Workbox Runtime Caching<br/>Offline support<br/>Cache-first / network-first strategies"]
|
||||
end
|
||||
|
||||
subgraph Tier4["Tier 4: IndexedDB (Client)"]
|
||||
IDB["worldmonitor_db"]
|
||||
Baselines["baselines store<br/>(keyPath: 'key')"]
|
||||
Snapshots["snapshots store<br/>(keyPath: 'timestamp'<br/>index: 'by_time')"]
|
||||
IDB --> Baselines
|
||||
IDB --> Snapshots
|
||||
end
|
||||
|
||||
subgraph Tier5["Tier 5: Persistent Cache"]
|
||||
PC["persistent-cache.ts<br/>CacheEnvelope<T>"]
|
||||
TauriInvoke["Tauri invoke<br/>(OS filesystem)"]
|
||||
LSFallback["localStorage fallback<br/>prefix: worldmonitor-persistent-cache:"]
|
||||
PC --> TauriInvoke
|
||||
PC --> LSFallback
|
||||
end
|
||||
|
||||
Browser["Browser SPA"] --> Workbox
|
||||
Workbox --> CDN
|
||||
CDN --> Redis
|
||||
Redis --> ExternalAPI["External APIs"]
|
||||
|
||||
Browser --> IDB
|
||||
Browser --> PC
|
||||
|
||||
Sidecar["Desktop Sidecar"] --> MemCache
|
||||
MemCache --> ExternalAPI
|
||||
```
|
||||
|
||||
### Tier 1: Upstash Redis (Server-Side)
|
||||
|
||||
The api/_upstash-cache.js module wraps all API fetch operations with Redis GET/SET. Every API endpoint calls `getCachedJson(key)` before hitting upstream. On cache miss, the upstream response is stored with `setCachedJson(key, value, ttlSeconds)`. The module lazily initialises the Redis client from `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` environment variables.
|
||||
|
||||
A `hashString()` utility produces compact cache keys from request parameters using a DJB2 hash.
|
||||
|
||||
### Tier 1b: Sidecar In-Memory Cache
|
||||
|
||||
When running in desktop/sidecar mode (`LOCAL_API_MODE=sidecar`), Redis is bypassed entirely. An in-memory `Map` stores cache entries with expiry timestamps. Entries persist to disk as `api-cache.json` via debounced writes (2-second delay). A periodic cleanup interval (60 seconds) evicts expired entries. The maximum persisted entry count is capped at `MAX_PERSIST_ENTRIES` (default 5000).
|
||||
|
||||
The disk persistence uses atomic writes: data is written to a `.tmp` file first, then renamed to the final path. A `persistInFlight` flag with `persistQueued` prevents concurrent writes.
|
||||
|
||||
### Tier 2: Vercel CDN
|
||||
|
||||
API responses include `Cache-Control` headers with `s-maxage` and `stale-while-revalidate` directives. This enables Vercel's CDN edge nodes to serve cached responses without invoking the serverless function, reducing cold starts and upstream API calls.
|
||||
|
||||
### Tier 3: Service Worker (Workbox)
|
||||
|
||||
The Service Worker (configured via Workbox) provides runtime caching with strategy selection per route:
|
||||
|
||||
- **Cache-first** for static assets and infrequently changing data
|
||||
- **Network-first** for real-time feeds and market data
|
||||
- **Stale-while-revalidate** for semi-static resources
|
||||
|
||||
The offline fallback page (public/offline.html) is served when the network is unavailable and no cached response exists.
|
||||
|
||||
### Tier 4: IndexedDB
|
||||
|
||||
The `worldmonitor_db` IndexedDB database contains two object stores:
|
||||
|
||||
| Store | keyPath | Index | Purpose |
|
||||
|---|---|---|---|
|
||||
| `baselines` | `key` | — | Stores baseline values for temporal deviation tracking. The signal aggregator compares current values against baselines to detect anomalies. |
|
||||
| `snapshots` | `timestamp` | `by_time` | Stores periodic system state snapshots for the playback control feature, enabling users to replay historical states. |
|
||||
|
||||
### Tier 5: Persistent Cache
|
||||
|
||||
The src/services/persistent-cache.ts module provides a cross-platform persistent storage abstraction. Data is wrapped in a `CacheEnvelope<T>`:
|
||||
|
||||
```typescript
|
||||
type CacheEnvelope<T> = {
|
||||
key: string;
|
||||
updatedAt: number;
|
||||
data: T;
|
||||
};
|
||||
```
|
||||
|
||||
On desktop, `getPersistentCache()` and `setPersistentCache()` attempt Tauri IPC invocations (`read_cache_entry` / `write_cache_entry`) first, which store data on the OS filesystem via the Rust backend. If the Tauri call fails (or in web mode), the module falls back to `localStorage` with the prefix `worldmonitor-persistent-cache:`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Desktop Architecture
|
||||
|
||||
The desktop application uses Tauri 2 (Rust) as a native shell around the web SPA, with a Node.js sidecar process providing a local API server.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph TauriApp["Tauri 2 Desktop Application"]
|
||||
subgraph Rust["Rust Backend (src-tauri/)"]
|
||||
TauriCore["tauri.conf.json<br/>(+ variant overrides)"]
|
||||
BuildRS["build.rs"]
|
||||
Cargo["Cargo.toml"]
|
||||
Commands["IPC Commands<br/>(read_cache_entry,<br/>write_cache_entry, etc.)"]
|
||||
Keychain["OS Keychain<br/>(18 RuntimeSecretKeys)"]
|
||||
end
|
||||
|
||||
subgraph SidecarProc["Node.js Sidecar"]
|
||||
LocalAPI["Local API Server<br/>http://127.0.0.1:46123"]
|
||||
MemCache["In-memory Map<br/>+ api-cache.json"]
|
||||
LocalAPI --> MemCache
|
||||
end
|
||||
|
||||
subgraph WebView["WebView (SPA)"]
|
||||
Runtime["runtime.ts<br/>detectDesktopRuntime()"]
|
||||
Bridge["tauri-bridge.ts<br/>Typed IPC wrapper"]
|
||||
Config["runtime-config.ts<br/>Feature toggles & secrets"]
|
||||
PCache["persistent-cache.ts"]
|
||||
end
|
||||
end
|
||||
|
||||
Runtime -->|"isDesktopRuntime()"| Bridge
|
||||
Bridge -->|"invokeTauri()"| Commands
|
||||
Config -->|"readSecret()"| Keychain
|
||||
PCache -->|"read/write_cache_entry"| Commands
|
||||
WebView -->|"fetch() via patch"| LocalAPI
|
||||
```
|
||||
|
||||
### Runtime Detection
|
||||
|
||||
The src/services/runtime.ts module detects the desktop environment through multiple signals:
|
||||
|
||||
```typescript
|
||||
function detectDesktopRuntime(probe: RuntimeProbe): boolean {
|
||||
// Checks: window.__TAURI__, user agent, location host (127.0.0.1)
|
||||
}
|
||||
```
|
||||
|
||||
When desktop mode is detected, `getApiBaseUrl()` returns `http://127.0.0.1:46123` instead of relative paths, routing all API calls through the local sidecar. A global `fetch()` monkey-patch (applied once via `__wmFetchPatched` guard) rewrites API URLs to point at the sidecar.
|
||||
|
||||
### Tauri Configuration
|
||||
|
||||
The src-tauri/ directory contains:
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| tauri.conf.json | Base Tauri configuration (window size, CSP, bundle settings) |
|
||||
| tauri.tech.conf.json | Tech variant overrides (app name, window title, icons) |
|
||||
| tauri.finance.conf.json | Finance variant overrides |
|
||||
| build.rs | Rust build script for Tauri codegen |
|
||||
| Cargo.toml | Rust dependencies |
|
||||
| sidecar/ | Node.js sidecar source (local API server) |
|
||||
| capabilities/ | Tauri capability definitions (permissions) |
|
||||
| icons/ | Application icons for each platform |
|
||||
|
||||
### Tauri Bridge
|
||||
|
||||
The src/services/tauri-bridge.ts module provides a typed TypeScript wrapper around Tauri's IPC invoke mechanism. It exposes functions like `invokeTauri<T>(command, args)` that handle serialisation and error mapping.
|
||||
|
||||
### Runtime Configuration
|
||||
|
||||
The src/services/runtime-config.ts module manages two concerns:
|
||||
|
||||
**1. Runtime Secrets** — 18 `RuntimeSecretKey` values representing API keys and credentials:
|
||||
|
||||
`GROQ_API_KEY`, `OPENROUTER_API_KEY`, `FRED_API_KEY`, `EIA_API_KEY`, `CLOUDFLARE_API_TOKEN`, `ACLED_ACCESS_TOKEN`, `URLHAUS_AUTH_KEY`, `OTX_API_KEY`, `ABUSEIPDB_API_KEY`, `WINGBITS_API_KEY`, `WS_RELAY_URL`, `VITE_OPENSKY_RELAY_URL`, `OPENSKY_CLIENT_ID`, `OPENSKY_CLIENT_SECRET`, `AISSTREAM_API_KEY`, `FINNHUB_API_KEY`, `NASA_FIRMS_API_KEY`, `UC_DP_KEY`.
|
||||
|
||||
On desktop, secrets are read from the OS keychain via Tauri IPC. In web mode, they fall back to environment variables. A `validateSecret()` function provides format validation with user-facing hints.
|
||||
|
||||
**2. Feature Toggles** — 14 `RuntimeFeatureId` values stored in localStorage under the key `worldmonitor-runtime-feature-toggles`:
|
||||
|
||||
`aiGroq`, `aiOpenRouter`, `economicFred`, `energyEia`, `internetOutages`, `acledConflicts`, `abuseChThreatIntel`, `alienvaultOtxThreatIntel`, `abuseIpdbThreatIntel`, `wingbitsEnrichment`, `aisRelay`, `openskyRelay`, `finnhubMarkets`, `nasaFirms`.
|
||||
|
||||
Each `RuntimeFeatureDefinition` declares its required secrets (and optionally desktop-specific overrides via `desktopRequiredSecrets`), along with a `fallback` description explaining behaviour when the feature is unavailable. The `isFeatureAvailable()` function checks both the toggle state and secret availability.
|
||||
|
||||
The settings page listens for `storage` events on the toggles key, enabling cross-tab synchronisation.
|
||||
|
||||
---
|
||||
|
||||
## 8. ML Pipeline
|
||||
|
||||
World Monitor runs machine-learning inference directly in the browser using ONNX Runtime Web via Transformers.js, with API-based fallbacks for constrained devices.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Capabilities["Capability Detection"]
|
||||
Detect["ml-capabilities.ts<br/>detectMLCapabilities()"]
|
||||
WebGPU["WebGPU check"]
|
||||
WebGL["WebGL check"]
|
||||
SIMD["SIMD check"]
|
||||
Threads["SharedArrayBuffer check"]
|
||||
Memory["Device memory estimation"]
|
||||
Detect --> WebGPU
|
||||
Detect --> WebGL
|
||||
Detect --> SIMD
|
||||
Detect --> Threads
|
||||
Detect --> Memory
|
||||
end
|
||||
|
||||
subgraph Config["Model Configuration (ml-config.ts)"]
|
||||
Models["MODEL_CONFIGS"]
|
||||
Embed["embeddings<br/>all-MiniLM-L6-v2<br/>23 MB"]
|
||||
Sentiment["sentiment<br/>DistilBERT-SST2<br/>65 MB"]
|
||||
Summarize["summarization<br/>Flan-T5-base<br/>250 MB"]
|
||||
SumSmall["summarization-beta<br/>Flan-T5-small<br/>60 MB"]
|
||||
NER["ner<br/>BERT-NER<br/>65 MB"]
|
||||
Models --> Embed
|
||||
Models --> Sentiment
|
||||
Models --> Summarize
|
||||
Models --> SumSmall
|
||||
Models --> NER
|
||||
end
|
||||
|
||||
subgraph WorkerPipeline["ML Worker Pipeline"]
|
||||
Manager["MLWorkerManager<br/>(ml-worker.ts)"]
|
||||
Worker["ml.worker.ts<br/>(Web Worker)"]
|
||||
ONNX["ONNX Runtime Web<br/>(@xenova/transformers)"]
|
||||
Manager -->|"postMessage"| Worker
|
||||
Worker --> ONNX
|
||||
end
|
||||
|
||||
subgraph Fallback["Fallback Chain"]
|
||||
Groq["Groq API<br/>(cloud LLM)"]
|
||||
OpenRouter["OpenRouter API<br/>(cloud LLM)"]
|
||||
BrowserML["Browser Transformers.js<br/>(offline capable)"]
|
||||
Groq -->|"unavailable"| OpenRouter
|
||||
OpenRouter -->|"unavailable"| BrowserML
|
||||
end
|
||||
|
||||
subgraph Results["Worker Message Types"]
|
||||
EmbedR["embed-result"]
|
||||
SumR["summarize-result"]
|
||||
SentR["sentiment-result"]
|
||||
EntR["entities-result"]
|
||||
ClusterR["cluster-semantic-result"]
|
||||
end
|
||||
|
||||
Detect -->|"isSupported"| Manager
|
||||
Config --> Worker
|
||||
Manager --> Results
|
||||
```
|
||||
|
||||
### Capability Detection
|
||||
|
||||
The src/services/ml-capabilities.ts module probes the browser environment before loading any models:
|
||||
|
||||
```typescript
|
||||
interface MLCapabilities {
|
||||
isSupported: boolean;
|
||||
isDesktop: boolean;
|
||||
hasWebGL: boolean;
|
||||
hasWebGPU: boolean;
|
||||
hasSIMD: boolean;
|
||||
hasThreads: boolean;
|
||||
estimatedMemoryMB: number;
|
||||
recommendedExecutionProvider: 'webgpu' | 'webgl' | 'wasm';
|
||||
recommendedThreads: number;
|
||||
}
|
||||
```
|
||||
|
||||
ML is only enabled on desktop-class devices (`!isMobileDevice()`) with at least WebGL support and an estimated 100+ MB of available memory. The `recommendedExecutionProvider` selects the optimal ONNX backend: WebGPU (fastest, if available), WebGL, or WASM fallback.
|
||||
|
||||
### Model Configuration
|
||||
|
||||
The src/config/ml-config.ts module defines five model configurations:
|
||||
|
||||
| Model ID | HuggingFace Model | Size | Task | Required |
|
||||
|---|---|---|---|---|
|
||||
| `embeddings` | Xenova/all-MiniLM-L6-v2 | 23 MB | feature-extraction | Yes |
|
||||
| `sentiment` | Xenova/distilbert-base-uncased-finetuned-sst-2-english | 65 MB | text-classification | No |
|
||||
| `summarization` | Xenova/flan-t5-base | 250 MB | text2text-generation | No |
|
||||
| `summarization-beta` | Xenova/flan-t5-small | 60 MB | text2text-generation | No |
|
||||
| `ner` | Xenova/bert-base-NER | 65 MB | token-classification | No |
|
||||
|
||||
Only the embeddings model is marked as `required` — it powers semantic clustering. Other models are loaded on-demand based on feature flags (`ML_FEATURE_FLAGS`) and available memory budget (`ML_THRESHOLDS.memoryBudgetMB`, default 200 MB).
|
||||
|
||||
### ML Thresholds
|
||||
|
||||
```typescript
|
||||
const ML_THRESHOLDS = {
|
||||
semanticClusterThreshold: 0.75, // cosine similarity for merging clusters
|
||||
minClustersForML: 5, // minimum clusters before ML refinement
|
||||
maxTextsPerBatch: 20, // batch size for embedding requests
|
||||
modelLoadTimeoutMs: 600_000, // 10 min model download/compile timeout
|
||||
inferenceTimeoutMs: 120_000, // 2 min per inference call
|
||||
memoryBudgetMB: 200, // max memory for all loaded models
|
||||
};
|
||||
```
|
||||
|
||||
### Worker Architecture
|
||||
|
||||
The `MLWorkerManager` class (src/services/ml-worker.ts) manages the lifecycle of a dedicated Web Worker (src/workers/ml.worker.ts). Communication uses a request-response pattern over `postMessage`:
|
||||
|
||||
1. **Initialisation** — `init()` calls `detectMLCapabilities()`, creates the worker if supported, and waits for a `worker-ready` message (10-second timeout).
|
||||
|
||||
2. **Request dispatch** — Each method (`embed()`, `summarize()`, `sentiment()`, `entities()`, `clusterSemantic()`) generates a unique request ID, posts a message to the worker, and returns a `Promise` that resolves when the worker posts back a matching result message.
|
||||
|
||||
3. **Timeout handling** — Each pending request has an independent timeout. If the worker fails to respond within `inferenceTimeoutMs`, the promise rejects and the request is cleaned up.
|
||||
|
||||
4. **Model lifecycle** — Models are loaded lazily on first use. The worker emits `model-progress` events during download, enabling progress UI. `model-loaded` and `model-unloaded` events track the loaded model set.
|
||||
|
||||
### Worker Result Message Types
|
||||
|
||||
| Message Type | Payload | Used By |
|
||||
|---|---|---|
|
||||
| `embed-result` | `embeddings: number[][]` | Semantic clustering |
|
||||
| `summarize-result` | `summaries: string[]` | AI Insights panel |
|
||||
| `sentiment-result` | `results: SentimentResult[]` | Threat classification augmentation |
|
||||
| `entities-result` | `entities: NEREntity[][]` | Entity extraction (ML-backed) |
|
||||
| `cluster-semantic-result` | `clusters: number[][]` | Cluster merging |
|
||||
|
||||
### Fallback Chain
|
||||
|
||||
When browser-based ML is not available (mobile devices, constrained hardware, or feature disabled), the system falls back to cloud-based LLM APIs:
|
||||
|
||||
1. **Groq API** — Primary cloud fallback. Used for summarisation and classification via /api/groq-summarize.
|
||||
2. **OpenRouter API** — Secondary cloud fallback via /api/openrouter-summarize.
|
||||
3. **Browser Transformers.js** — Tertiary fallback for offline operation. Even without API access, the embeddings model enables basic semantic clustering.
|
||||
|
||||
The fallback is not automatic at the ML worker level; each consumer service chooses its preferred provider and handles degradation independently.
|
||||
|
||||
---
|
||||
|
||||
## 9. Error Handling Hierarchy
|
||||
|
||||
World Monitor uses a circuit-breaker pattern to manage transient failures across its many data sources, preventing cascade failures and providing graceful degradation.
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> Closed: Initial state
|
||||
|
||||
Closed --> Closed: fetch() success → recordSuccess()
|
||||
Closed --> HalfOpen: fetch() failure<br/>(failures < MAX_FAILURES)
|
||||
HalfOpen --> Open: fetch() failure<br/>(failures >= MAX_FAILURES)
|
||||
Open --> Recovery: COOLDOWN_MS elapsed
|
||||
Recovery --> Closed: retry success → reset
|
||||
Recovery --> Open: retry failure → extend cooldown
|
||||
|
||||
state Closed {
|
||||
[*] --> Live
|
||||
Live: mode = 'live'
|
||||
Live: Serve fresh data
|
||||
}
|
||||
|
||||
state HalfOpen {
|
||||
[*] --> Degraded
|
||||
Degraded: failures > 0
|
||||
Degraded: Still attempting fetches
|
||||
}
|
||||
|
||||
state Open {
|
||||
[*] --> CircuitOpen
|
||||
CircuitOpen: mode = 'cached' or 'unavailable'
|
||||
CircuitOpen: Serve cached data if available
|
||||
CircuitOpen: Skip fetch until cooldown expires
|
||||
}
|
||||
|
||||
state Recovery {
|
||||
[*] --> Retry
|
||||
Retry: Single probe request
|
||||
Retry: On success → reset to Closed
|
||||
}
|
||||
```
|
||||
|
||||
### Circuit Breaker Implementation
|
||||
|
||||
The `CircuitBreaker<T>` class in src/utils/circuit-breaker.ts implements per-feed failure tracking with automatic cooldowns:
|
||||
|
||||
```typescript
|
||||
interface CircuitState {
|
||||
failures: number;
|
||||
cooldownUntil: number;
|
||||
lastError?: string;
|
||||
}
|
||||
|
||||
type BreakerDataMode = 'live' | 'cached' | 'unavailable';
|
||||
```
|
||||
|
||||
**Constants:**
|
||||
|
||||
| Constant | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `DEFAULT_MAX_FAILURES` | 2 | Consecutive failures before opening the circuit |
|
||||
| `DEFAULT_COOLDOWN_MS` | 5 min (300,000 ms) | How long to wait before retrying |
|
||||
| `DEFAULT_CACHE_TTL_MS` | 10 min (600,000 ms) | How long cached data remains valid |
|
||||
|
||||
### Lifecycle
|
||||
|
||||
1. **Closed (Live)** — Normal operation. Each successful `fetch()` calls `recordSuccess()`, resetting the failure counter.
|
||||
|
||||
2. **Failure Tracking** — On fetch failure, the failure counter increments. The `lastError` is recorded for diagnostics.
|
||||
|
||||
3. **Open (Circuit Tripped)** — When `failures >= maxFailures`, the circuit opens. `cooldownUntil` is set to `Date.now() + cooldownMs`. While open:
|
||||
- `isOnCooldown()` returns `true`
|
||||
- No fetch attempts are made
|
||||
- `getCached()` serves the last successful response if within `cacheTtlMs`
|
||||
- If no cached data exists, the data mode is `'unavailable'`
|
||||
|
||||
4. **Recovery (Cooldown Expired)** — After the cooldown period, `isOnCooldown()` returns `false` and resets the state. The next fetch attempt acts as a probe:
|
||||
- On success → circuit fully resets to closed
|
||||
- On failure → circuit re-opens with a fresh cooldown
|
||||
|
||||
### Data State Reporting
|
||||
|
||||
Each breaker tracks a `BreakerDataState` for UI display:
|
||||
|
||||
```typescript
|
||||
interface BreakerDataState {
|
||||
mode: BreakerDataMode; // 'live' | 'cached' | 'unavailable'
|
||||
timestamp: number | null;
|
||||
offline: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
Panels use this state to display freshness indicators — e.g., showing a "cached" badge with the last successful timestamp, or an "unavailable" state with the `lastError` message.
|
||||
|
||||
### Desktop Offline Mode
|
||||
|
||||
The `isDesktopOfflineMode()` helper detects when the Tauri desktop app loses network connectivity (`navigator.onLine === false`). In this mode, all circuit breakers immediately fall back to cached data without attempting network requests, preserving the user experience during temporary disconnections.
|
||||
|
||||
### Global Breaker Registry
|
||||
|
||||
A module-level `Map<string, CircuitBreaker<unknown>>` maintains all active breakers. Utility functions provide system-wide observability:
|
||||
|
||||
| Function | Purpose |
|
||||
|---|---|
|
||||
| `createCircuitBreaker<T>(options)` | Create and register a new breaker |
|
||||
| `getCircuitBreakerStatus()` | Returns status of all breakers (for diagnostics) |
|
||||
| `isCircuitBreakerOnCooldown(name)` | Check if a specific breaker is in cooldown |
|
||||
| `getCircuitBreakerCooldownInfo(name)` | Get cooldown state and remaining seconds |
|
||||
| `removeCircuitBreaker(name)` | Deregister a breaker |
|
||||
|
||||
### Degradation Hierarchy
|
||||
|
||||
The overall error handling follows a predictable degradation path:
|
||||
|
||||
```
|
||||
Live data (fresh fetch)
|
||||
└── on failure → Stale cache (within cacheTtlMs)
|
||||
└── expired cache → 'unavailable' state in UI
|
||||
└── desktop offline → immediate cache fallback
|
||||
```
|
||||
|
||||
Each panel independently manages its breaker, so a failure in one data source (e.g., OpenSky API downtime) does not affect other panels. The AI Insights panel aggregates breaker states to provide a system-wide health summary.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,53 @@
|
||||
# Desktop Runtime Configuration Schema
|
||||
|
||||
World Monitor desktop now uses a runtime configuration schema with per-feature toggles and secret-backed credentials.
|
||||
|
||||
## Secret keys
|
||||
|
||||
The desktop vault schema supports the following 17 keys used by services and relays:
|
||||
|
||||
- `GROQ_API_KEY`
|
||||
- `OPENROUTER_API_KEY`
|
||||
- `FRED_API_KEY`
|
||||
- `EIA_API_KEY`
|
||||
- `FINNHUB_API_KEY`
|
||||
- `CLOUDFLARE_API_TOKEN`
|
||||
- `ACLED_ACCESS_TOKEN`
|
||||
- `URLHAUS_AUTH_KEY`
|
||||
- `OTX_API_KEY`
|
||||
- `ABUSEIPDB_API_KEY`
|
||||
- `NASA_FIRMS_API_KEY`
|
||||
- `WINGBITS_API_KEY`
|
||||
- `VITE_OPENSKY_RELAY_URL`
|
||||
- `OPENSKY_CLIENT_ID`
|
||||
- `OPENSKY_CLIENT_SECRET`
|
||||
- `AISSTREAM_API_KEY`
|
||||
- `VITE_WS_RELAY_URL`
|
||||
|
||||
## Feature schema
|
||||
|
||||
Each feature includes:
|
||||
|
||||
- `id`: stable feature identifier.
|
||||
- `requiredSecrets`: list of keys that must be present and valid.
|
||||
- `enabled`: user-toggle state from runtime settings panel.
|
||||
- `available`: computed (`enabled && requiredSecrets valid`).
|
||||
- `fallback`: user-facing degraded behavior description.
|
||||
|
||||
## Desktop secret storage
|
||||
|
||||
Desktop builds persist secrets in OS credential storage through Tauri command bindings backed by Rust `keyring` entries (`world-monitor` service namespace).
|
||||
|
||||
Secrets are **not stored in plaintext files** by the frontend.
|
||||
|
||||
## Degradation behavior
|
||||
|
||||
If required secrets are missing/disabled:
|
||||
|
||||
- Summarization: Groq/OpenRouter disabled, browser model fallback.
|
||||
- FRED / EIA / Finnhub: economic, oil analytics, and stock data return empty state.
|
||||
- Cloudflare / ACLED: outages/conflicts return empty state.
|
||||
- Cyber threat feeds (URLhaus, OTX, AbuseIPDB): cyber threat layer returns empty state.
|
||||
- NASA FIRMS: satellite fire detection returns empty state.
|
||||
- Wingbits: flight enrichment disabled, heuristic-only flight classification remains.
|
||||
- AIS / OpenSky relay: live tracking features are disabled cleanly.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user