Compare commits
272
Commits
layout-clean
...
v2.5.16
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c66e740f30 | ||
|
|
675b1106bd | ||
|
|
c54907f272 | ||
|
|
5f3ed04ed7 | ||
|
|
6e7625aeaa | ||
|
|
e40b2eb47e | ||
|
|
d6ad89d594 | ||
|
|
0102428bb0 | ||
|
|
b1ffc315ec | ||
|
|
caeb3ead2c | ||
|
|
ab502f6062 | ||
|
|
bbe814c985 | ||
|
|
30bd84abb2 | ||
|
|
dd4f4ad786 | ||
|
|
9c9323732e | ||
|
|
1b719701bf | ||
|
|
6d1821229e | ||
|
|
ba903aa02a | ||
|
|
af99606cf9 | ||
|
|
5fa5a73e30 | ||
|
|
d2e00bc581 | ||
|
|
e40fe1aeb6 | ||
|
|
012907ae84 | ||
|
|
2db217211e | ||
|
|
1cc961b491 | ||
|
|
fc7b91fa9d | ||
|
|
e5fc09d1fc | ||
|
|
812c6d0562 | ||
|
|
a560efff49 | ||
|
|
2839d884db | ||
|
|
0060a71a66 | ||
|
|
f912c5f15a | ||
|
|
384cc78daa | ||
|
|
5221486c88 | ||
|
|
e5b30ba28c | ||
|
|
8ac824ebb4 | ||
|
|
be82576037 | ||
|
|
66e9f818d4 | ||
|
|
75dc6418ea | ||
|
|
e0d0cf9790 | ||
|
|
3983278f53 | ||
|
|
dc30693135 | ||
|
|
b0071d1a14 | ||
|
|
1844276017 | ||
|
|
24d42f70f8 | ||
|
|
95228647b4 | ||
|
|
bf4e216efd | ||
|
|
6ac39d5166 | ||
|
|
5c4b1ffc6b | ||
|
|
07d0803014 | ||
|
|
b238618e13 | ||
|
|
5206027573 | ||
|
|
e7260be2f2 | ||
|
|
cc8f5f3f01 | ||
|
|
e3bdde0d92 | ||
|
|
cb0235ca6b | ||
|
|
797d127ce1 | ||
|
|
2f92dbee8b | ||
|
|
1d069229c0 | ||
|
|
1d72ac8ab5 | ||
|
|
61dcfb9ca8 | ||
|
|
37096e8954 | ||
|
|
7e0ccfe30e | ||
|
|
57af053d92 | ||
|
|
408d5d3374 | ||
|
|
b1d835b69f | ||
|
|
fb6c61d4d9 | ||
|
|
9ef3cd8402 | ||
|
|
e35f0f70e9 | ||
|
|
88434cb26b | ||
|
|
d4c768dc59 | ||
|
|
f24b04fbd7 | ||
|
|
9011c91801 | ||
|
|
2881088272 | ||
|
|
7942e0e8c7 | ||
|
|
49b73c56ec | ||
|
|
a2cff87b43 | ||
|
|
4a029420b9 | ||
|
|
59646df7d8 | ||
|
|
58da7b789c | ||
|
|
1d417af59f | ||
|
|
57a5bf44f6 | ||
|
|
09011ec479 | ||
|
|
6014f268a3 | ||
|
|
9b810e1836 | ||
|
|
b8aabdc373 | ||
|
|
eebf9a9c7b | ||
|
|
557706e2d7 | ||
|
|
3849f5135f | ||
|
|
57481e8f97 | ||
|
|
81d4777964 | ||
|
|
9b932dcfac | ||
|
|
49373e5054 | ||
|
|
03598a6cff | ||
|
|
17f5dd7267 | ||
|
|
7cec4b2113 | ||
|
|
b667b189ff | ||
|
|
da680d7397 | ||
|
|
d593fbf413 | ||
|
|
adee459791 | ||
|
|
4708e1d434 | ||
|
|
10e50e080d | ||
|
|
709545857b | ||
|
|
98132b92a5 | ||
|
|
4721fb0873 | ||
|
|
9e2432f098 | ||
|
|
d68221d7f8 | ||
|
|
dc7a1ae61a | ||
|
|
6271fafd40 | ||
|
|
a37dced84e | ||
|
|
cbfc06f315 | ||
|
|
eafc4cb955 | ||
|
|
a6f5284b09 | ||
|
|
a8068ff579 | ||
|
|
bfb7cec0d1 | ||
|
|
f06acd3c88 | ||
|
|
184603b53a | ||
|
|
080f49b0b2 | ||
|
|
353a46460b | ||
|
|
eee0eece50 | ||
|
|
f790f3f63c | ||
|
|
b8be2f6890 | ||
|
|
b06c8a17b3 | ||
|
|
8504d5649a | ||
|
|
d24d61f333 | ||
|
|
0f4420d9cc | ||
|
|
63a4c9ab9c | ||
|
|
8f8b0c843b | ||
|
|
fcd590a9e6 | ||
|
|
3e935e3541 | ||
|
|
5d7e77f8b0 | ||
|
|
b23cac04b9 | ||
|
|
df2cdafadb | ||
|
|
84f17e6c4c | ||
|
|
43cd5b3826 | ||
|
|
a2bcfeede4 | ||
|
|
584159f35c | ||
|
|
440e45a032 | ||
|
|
8cc82def37 | ||
|
|
21d7c8b7f3 | ||
|
|
1a9dc02ecc | ||
|
|
e6285e3de0 | ||
|
|
4365626df3 | ||
|
|
e1b9e9e8a9 | ||
|
|
86eff555f0 | ||
|
|
740e3513ef | ||
|
|
1922a781cd | ||
|
|
8699dae5e5 | ||
|
|
1fc6dc2dbb | ||
|
|
68e6a367d6 | ||
|
|
a388afe400 | ||
|
|
417b7d275d | ||
|
|
8243147886 | ||
|
|
5786a8f279 | ||
|
|
7e86e26a9e | ||
|
|
81577cbcfa | ||
|
|
d3da54bcb2 | ||
|
|
519da36ef6 | ||
|
|
67259da470 | ||
|
|
e1f392b45b | ||
|
|
c1b8ffb890 | ||
|
|
d2490bb7b7 | ||
|
|
f082e09d55 | ||
|
|
3bf4244cde | ||
|
|
4822a839eb | ||
|
|
9273facad1 | ||
|
|
48f8e24353 | ||
|
|
d07f14fef4 | ||
|
|
c939cc6296 | ||
|
|
f8d0c81f60 | ||
|
|
7440281d71 | ||
|
|
92892c5b7c | ||
|
|
7681202c6e | ||
|
|
b714e7b13e | ||
|
|
3ffe76b208 | ||
|
|
84336a9f94 | ||
|
|
919e7c996e | ||
|
|
fd51b57911 | ||
|
|
91c9982a41 | ||
|
|
ac7f3fac5e | ||
|
|
61c1dad6c9 | ||
|
|
b49329cdef | ||
|
|
5cfd827173 | ||
|
|
ab69f09c60 | ||
|
|
7aa60f954c | ||
|
|
7a57d39408 | ||
|
|
6e9615f428 | ||
|
|
4e10bbc283 | ||
|
|
13172cec4c | ||
|
|
c62fad7fce | ||
|
|
7ab3a8e76d | ||
|
|
f521377453 | ||
|
|
dd34c1e4ff | ||
|
|
49f9fc860f | ||
|
|
b7ca969228 | ||
|
|
383da59ffc | ||
|
|
ffb5ae0413 | ||
|
|
e549b6e5a6 | ||
|
|
db4b401b8f | ||
|
|
2b7b35efd8 | ||
|
|
4b0e50e0fe | ||
|
|
cbd7c29f03 | ||
|
|
86b83bae78 | ||
|
|
814c01fea2 | ||
|
|
7972248e1c | ||
|
|
380f1b7235 | ||
|
|
316df2997d | ||
|
|
fb1efd2ebc | ||
|
|
ef13263c7b | ||
|
|
2df58717e4 | ||
|
|
282eba9838 | ||
|
|
cf46ff8940 | ||
|
|
0735ce5c78 | ||
|
|
a1973d4681 | ||
|
|
aae90df9c9 | ||
|
|
a78410b8d2 | ||
|
|
44d2254b5b | ||
|
|
e10ed6c981 | ||
|
|
07e14deace | ||
|
|
f2a1a2ccb5 | ||
|
|
7b7fe3cbdd | ||
|
|
2e267cf307 | ||
|
|
8697c543ea | ||
|
|
81ccace81b | ||
|
|
c170ada79c | ||
|
|
01a65fb201 | ||
|
|
fe2181b92b | ||
|
|
2e85a64712 | ||
|
|
4121113547 | ||
|
|
2473c4bfd4 | ||
|
|
6c02bd2eb6 | ||
|
|
816015357a | ||
|
|
9fd1bf293d | ||
|
|
bf2c0b1598 | ||
|
|
a39aed1d64 | ||
|
|
58389ba440 | ||
|
|
fce6c52970 | ||
|
|
e040994e9f | ||
|
|
7dc53c0f4c | ||
|
|
bd3b71f4d8 | ||
|
|
20480feae4 | ||
|
|
8d20069830 | ||
|
|
6a10ca3cc0 | ||
|
|
bab0974407 | ||
|
|
6c3d2770f7 | ||
|
|
eedf43e94a | ||
|
|
1d1b1b209f | ||
|
|
bb14f0e9a8 | ||
|
|
5cdc41712c | ||
|
|
ba329e2a2a | ||
|
|
3f5fa51f40 | ||
|
|
f92de9f9da | ||
|
|
a1214c9b1e | ||
|
|
471d76223f | ||
|
|
c494f5b995 | ||
|
|
987a7b5e08 | ||
|
|
98797c9b02 | ||
|
|
bc96da9106 | ||
|
|
9cff125078 | ||
|
|
aa823bbb60 | ||
|
|
62a69acc11 | ||
|
|
4be7065f07 | ||
|
|
d7a6e95611 | ||
|
|
a851d5e8a1 | ||
|
|
172e7018a3 | ||
|
|
18f4bb98a4 | ||
|
|
22869e8062 | ||
|
|
ad32dd899d | ||
|
|
f6e7bbbfbc | ||
|
|
85e2384f0f | ||
|
|
be485ad1c2 | ||
|
|
434cd120fe |
@@ -1,32 +0,0 @@
|
||||
# Sentry Triage — 2026-02-19
|
||||
|
||||
Commit: `09174fd` on `main`
|
||||
|
||||
## Issues Triaged (5)
|
||||
|
||||
### ACTIONABLE — Fixed in Code
|
||||
|
||||
| ID | Title | Events | Users | Fix |
|
||||
|---|---|---|---|---|
|
||||
| WORLDMONITOR-1G | `Error: ML request unload-model timed out after 120000ms` | 30 | 27 | Wrapped `unloadModel()` in try/catch; timeout no longer leaks as unhandled rejection. Cleans up `loadedModels` set on failure. |
|
||||
| WORLDMONITOR-1F | `Error: ML request unload-model timed out after 120000ms` | 9 | 9 | Same root cause as 1G (different release build hash). |
|
||||
| WORLDMONITOR-1K | `TypeError: this.player.playVideo is not a function` | 1 | 1 | Added optional chaining (`playVideo?.()`, `pauseVideo?.()`) in `LiveNewsPanel.ts`. YT IFrame API player object may not have methods ready during initialization race. |
|
||||
|
||||
### NOISE — Filtered
|
||||
|
||||
| ID | Title | Events | Users | Filter |
|
||||
|---|---|---|---|---|
|
||||
| WORLDMONITOR-1J | `InternalError: too much recursion` | 1 | 1 | i18next internal `translate -> extractFromKey` cycle on Firefox 147. Added `/too much recursion/` to `ignoreErrors`. |
|
||||
| WORLDMONITOR-1H | `TypeError: Cannot read properties of null (reading 'id')` | 1 | 1 | maplibre-gl internal render crash (`_drawLayers -> renderLayers`). Extended `beforeSend` regex to suppress null `id`/`type` when stack is in map chunk. |
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `src/services/ml-worker.ts` | `unloadModel()`: try/catch around `this.request()`, clean `loadedModels` on failure |
|
||||
| `src/components/LiveNewsPanel.ts` | Optional chaining on `playVideo?.()` and `pauseVideo?.()` |
|
||||
| `src/main.ts` | Added `/too much recursion/` to `ignoreErrors`; extended maplibre `beforeSend` filter for null `id`/`type` |
|
||||
|
||||
## Sentry Status
|
||||
|
||||
All 5 issues marked **resolved (in next release)** via API. They will auto-reopen if errors recur after deployment.
|
||||
+50
-4
@@ -78,7 +78,8 @@ NASA_FIRMS_API_KEY=
|
||||
|
||||
|
||||
# ------ Railway Relay (scripts/ais-relay.cjs) ------
|
||||
# The relay server handles AIS vessel tracking and OpenSky aircraft data.
|
||||
# The relay server handles AIS vessel tracking + OpenSky aircraft data + RSS proxy.
|
||||
# It can also run the Telegram OSINT poller (stateful MTProto) when configured.
|
||||
# Deploy on Railway with: node scripts/ais-relay.cjs
|
||||
|
||||
# AISStream API key for live vessel positions
|
||||
@@ -91,22 +92,46 @@ OPENSKY_CLIENT_ID=
|
||||
OPENSKY_CLIENT_SECRET=
|
||||
|
||||
|
||||
# ------ Telegram OSINT (Railway relay) ------
|
||||
# Telegram MTProto keys (free): https://my.telegram.org/apps
|
||||
TELEGRAM_API_ID=
|
||||
TELEGRAM_API_HASH=
|
||||
|
||||
# GramJS StringSession generated locally (see: scripts/telegram/session-auth.mjs)
|
||||
TELEGRAM_SESSION=
|
||||
|
||||
# Which curated list bucket to ingest: full | tech | finance
|
||||
TELEGRAM_CHANNEL_SET=full
|
||||
|
||||
# ------ Railway Relay Connection (Vercel → Railway) ------
|
||||
|
||||
# Server-side URL (https://) — used by Vercel edge functions to reach the relay
|
||||
WS_RELAY_URL=
|
||||
|
||||
# Client-side URL (wss://) — used by the browser to connect via WebSocket
|
||||
# Optional client-side URL (wss://) — local/dev fallback only
|
||||
VITE_WS_RELAY_URL=
|
||||
|
||||
# Shared secret between Vercel and Railway relay.
|
||||
# Must be set to the SAME value on both platforms in production.
|
||||
RELAY_SHARED_SECRET=
|
||||
|
||||
# Header name used to send the relay secret (must match on both platforms)
|
||||
RELAY_AUTH_HEADER=x-relay-key
|
||||
|
||||
# Emergency production override to allow unauthenticated relay traffic.
|
||||
# Leave unset/false in production.
|
||||
ALLOW_UNAUTHENTICATED_RELAY=false
|
||||
|
||||
# Rolling window size (seconds) used by relay /metrics endpoint.
|
||||
RELAY_METRICS_WINDOW_SECONDS=60
|
||||
|
||||
|
||||
# ------ Public Data Sources (no keys required) ------
|
||||
|
||||
# UCDP (Uppsala Conflict Data Program) — public API, no auth
|
||||
# UNHCR (UN Refugee Agency) — public API, no auth (CC BY 4.0)
|
||||
# Open-Meteo — public API, no auth (processes Copernicus ERA5)
|
||||
# WorldPop — public API, optional key for higher rate limits
|
||||
# WORLDPOP_API_KEY=
|
||||
# WorldPop — public API, no auth needed
|
||||
|
||||
|
||||
# ------ Site Configuration ------
|
||||
@@ -114,7 +139,28 @@ VITE_WS_RELAY_URL=
|
||||
# Site variant: "full" (worldmonitor.app) or "tech" (tech.worldmonitor.app)
|
||||
VITE_VARIANT=full
|
||||
|
||||
# Client-side Sentry DSN (optional). Leave empty to disable error reporting.
|
||||
VITE_SENTRY_DSN=
|
||||
|
||||
# PostHog product analytics (optional). Leave empty to disable analytics.
|
||||
VITE_POSTHOG_KEY=
|
||||
VITE_POSTHOG_HOST=
|
||||
|
||||
# Map interaction mode:
|
||||
# - "flat" keeps pitch/rotation disabled (2D interaction)
|
||||
# - "3d" enables pitch/rotation interactions (default)
|
||||
VITE_MAP_INTERACTION_MODE=3d
|
||||
|
||||
|
||||
# ------ Desktop Cloud Fallback (Vercel) ------
|
||||
|
||||
# Comma-separated list of valid API keys for desktop cloud fallback.
|
||||
# Generate with: openssl rand -hex 24 | sed 's/^/wm_/'
|
||||
WORLDMONITOR_VALID_KEYS=
|
||||
|
||||
|
||||
# ------ Registration DB (Convex) ------
|
||||
|
||||
# Convex deployment URL for email registration storage.
|
||||
# Set up at: https://dashboard.convex.dev/
|
||||
CONVEX_URL=
|
||||
|
||||
@@ -18,6 +18,7 @@ body:
|
||||
- finance.worldmonitor.app (Finance)
|
||||
- Desktop app (Windows)
|
||||
- Desktop app (macOS)
|
||||
- Desktop app (Linux)
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -37,6 +38,8 @@ body:
|
||||
- Live video streams
|
||||
- Desktop app (Tauri)
|
||||
- Settings / API keys
|
||||
- Settings / LLMs (Ollama, Groq, OpenRouter)
|
||||
- Live webcams
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
@@ -50,11 +50,16 @@ jobs:
|
||||
node_target: 'x86_64-pc-windows-msvc'
|
||||
label: 'Windows-x64'
|
||||
timeout: 120
|
||||
- platform: 'ubuntu-22.04'
|
||||
- platform: 'ubuntu-24.04'
|
||||
args: ''
|
||||
node_target: 'x86_64-unknown-linux-gnu'
|
||||
label: 'Linux-x64'
|
||||
timeout: 120
|
||||
- platform: 'ubuntu-24.04-arm'
|
||||
args: '--target aarch64-unknown-linux-gnu'
|
||||
node_target: 'aarch64-unknown-linux-gnu'
|
||||
label: 'Linux-ARM64'
|
||||
timeout: 120
|
||||
|
||||
runs-on: ${{ matrix.platform }}
|
||||
name: Build (${{ matrix.label }})
|
||||
@@ -77,7 +82,7 @@ jobs:
|
||||
uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7
|
||||
with:
|
||||
toolchain: stable
|
||||
targets: ${{ contains(matrix.platform, 'macos') && 'aarch64-apple-darwin,x86_64-apple-darwin' || '' }}
|
||||
targets: ${{ contains(matrix.platform, 'macos') && 'aarch64-apple-darwin,x86_64-apple-darwin' || (matrix.label == 'Linux-ARM64' && 'aarch64-unknown-linux-gnu' || '') }}
|
||||
|
||||
- name: Rust cache
|
||||
uses: swatinem/rust-cache@ad397744b0d591a723ab90405b7247fac0e6b8db
|
||||
@@ -89,11 +94,24 @@ jobs:
|
||||
if: contains(matrix.platform, 'ubuntu')
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf
|
||||
sudo apt-get install -y \
|
||||
libwebkit2gtk-4.1-dev \
|
||||
libappindicator3-dev \
|
||||
librsvg2-dev \
|
||||
patchelf \
|
||||
gstreamer1.0-plugins-base \
|
||||
gstreamer1.0-plugins-good \
|
||||
gstreamer1.0-plugins-bad \
|
||||
gstreamer1.0-plugins-ugly \
|
||||
gstreamer1.0-libav \
|
||||
gstreamer1.0-gl
|
||||
|
||||
- name: Install frontend dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Check version consistency
|
||||
run: npm run version:check
|
||||
|
||||
- name: Bundle Node.js runtime
|
||||
shell: bash
|
||||
env:
|
||||
@@ -189,6 +207,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: full
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
|
||||
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
|
||||
APPLE_SIGNING_IDENTITY: ${{ env.APPLE_SIGNING_IDENTITY }}
|
||||
@@ -212,6 +231,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: full
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
with:
|
||||
tagName: v__VERSION__
|
||||
releaseName: 'World Monitor v__VERSION__'
|
||||
@@ -229,6 +249,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: tech
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
|
||||
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
|
||||
APPLE_SIGNING_IDENTITY: ${{ env.APPLE_SIGNING_IDENTITY }}
|
||||
@@ -253,6 +274,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: tech
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
with:
|
||||
tagName: v__VERSION__-tech
|
||||
releaseName: 'Tech Monitor v__VERSION__'
|
||||
@@ -280,6 +302,45 @@ jobs:
|
||||
fi
|
||||
echo "Verified signed app bundle and embedded Node runtime: $NODE_PATH"
|
||||
|
||||
- name: Smoke-test AppImage (Linux)
|
||||
if: contains(matrix.platform, 'ubuntu')
|
||||
shell: bash
|
||||
run: |
|
||||
sudo apt-get install -y xvfb imagemagick
|
||||
APPIMAGE=$(find src-tauri/target -path '*/bundle/appimage/*.AppImage' | head -1)
|
||||
if [ -z "$APPIMAGE" ]; then
|
||||
echo "::error::No AppImage found after build"
|
||||
exit 1
|
||||
fi
|
||||
chmod +x "$APPIMAGE"
|
||||
# Start Xvfb with known display number
|
||||
Xvfb :99 -screen 0 1440x900x24 &
|
||||
export DISPLAY=:99
|
||||
sleep 2
|
||||
# Launch AppImage under virtual framebuffer
|
||||
"$APPIMAGE" --no-sandbox &
|
||||
APP_PID=$!
|
||||
# Wait for app to render
|
||||
sleep 15
|
||||
# Screenshot the virtual display
|
||||
import -window root screenshot.png || true
|
||||
# Verify app is still running (didn't crash)
|
||||
if kill -0 $APP_PID 2>/dev/null; then
|
||||
echo "✅ AppImage launched successfully"
|
||||
kill $APP_PID || true
|
||||
else
|
||||
echo "❌ AppImage crashed during startup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Upload smoke test screenshot
|
||||
if: contains(matrix.platform, 'ubuntu')
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: linux-smoke-test-screenshot-${{ matrix.label }}
|
||||
path: screenshot.png
|
||||
if-no-files-found: warn
|
||||
|
||||
- name: Cleanup Apple signing materials
|
||||
if: always() && contains(matrix.platform, 'macos')
|
||||
shell: bash
|
||||
|
||||
@@ -8,6 +8,7 @@ on:
|
||||
|
||||
jobs:
|
||||
markdown:
|
||||
if: github.event.pull_request.head.repo.full_name == github.repository
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
name: 'Test Linux App'
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: test-linux-app-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
|
||||
|
||||
jobs:
|
||||
test-linux-app:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 120
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
|
||||
with:
|
||||
node-version: '22'
|
||||
cache: 'npm'
|
||||
|
||||
- name: Install Rust stable
|
||||
uses: dtolnay/rust-toolchain@631a55b12751854ce901bb631d5902ceb48146f7
|
||||
with:
|
||||
toolchain: stable
|
||||
|
||||
- name: Rust cache
|
||||
uses: swatinem/rust-cache@ad397744b0d591a723ab90405b7247fac0e6b8db
|
||||
with:
|
||||
workspaces: './src-tauri -> target'
|
||||
cache-on-failure: true
|
||||
|
||||
- name: Install Linux system dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y \
|
||||
libwebkit2gtk-4.1-dev \
|
||||
libappindicator3-dev \
|
||||
librsvg2-dev \
|
||||
patchelf \
|
||||
gstreamer1.0-plugins-base \
|
||||
gstreamer1.0-plugins-good \
|
||||
xwayland-run \
|
||||
xvfb \
|
||||
imagemagick \
|
||||
xdotool
|
||||
|
||||
- name: Install frontend dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Bundle Node.js runtime
|
||||
shell: bash
|
||||
env:
|
||||
NODE_VERSION: '22.14.0'
|
||||
NODE_TARGET: 'x86_64-unknown-linux-gnu'
|
||||
run: bash scripts/download-node.sh --target "$NODE_TARGET"
|
||||
|
||||
- name: Build Tauri app
|
||||
uses: tauri-apps/tauri-action@79c624843491f12ae9d63592534ed49df3bc4adb
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
VITE_VARIANT: full
|
||||
VITE_DESKTOP_RUNTIME: '1'
|
||||
CONVEX_URL: ${{ secrets.CONVEX_URL }}
|
||||
with:
|
||||
args: ''
|
||||
retryAttempts: 1
|
||||
|
||||
- name: Smoke-test AppImage
|
||||
shell: bash
|
||||
run: |
|
||||
APPIMAGE=$(find src-tauri/target/release/bundle/appimage -name '*.AppImage' | head -1)
|
||||
if [ -z "$APPIMAGE" ]; then
|
||||
echo "::error::No AppImage found after build"
|
||||
exit 1
|
||||
fi
|
||||
chmod +x "$APPIMAGE"
|
||||
APPIMAGE_ABS=$(realpath "$APPIMAGE")
|
||||
|
||||
# Write the inner test script (runs inside the display server)
|
||||
cat > /tmp/smoke-test.sh <<'SCRIPT'
|
||||
#!/bin/bash
|
||||
set -x
|
||||
echo "DISPLAY=$DISPLAY WAYLAND_DISPLAY=${WAYLAND_DISPLAY:-unset}"
|
||||
|
||||
GDK_BACKEND=x11 "$APPIMAGE_ABS" --no-sandbox 2>&1 | tee /tmp/app.log &
|
||||
APP_PID=$!
|
||||
sleep 20
|
||||
|
||||
# Screenshot via X11
|
||||
import -window root /tmp/screenshot.png 2>/dev/null || true
|
||||
|
||||
# Verify app is still running
|
||||
if kill -0 $APP_PID 2>/dev/null; then
|
||||
echo "APP_STATUS=running"
|
||||
else
|
||||
echo "APP_STATUS=crashed"
|
||||
echo "--- App log ---"
|
||||
tail -50 /tmp/app.log || true
|
||||
fi
|
||||
|
||||
# Window info
|
||||
xdotool search --name "" getwindowname 2>/dev/null | head -5 || true
|
||||
|
||||
kill $APP_PID 2>/dev/null || true
|
||||
SCRIPT
|
||||
chmod +x /tmp/smoke-test.sh
|
||||
|
||||
export APPIMAGE_ABS
|
||||
RESULT=0
|
||||
|
||||
# --- Try 1: xwfb-run (Xwayland on headless Wayland compositor) ---
|
||||
if command -v xwfb-run &>/dev/null; then
|
||||
echo "=== Using xwfb-run (Xwayland + headless compositor) ==="
|
||||
timeout 90 xwfb-run -- bash /tmp/smoke-test.sh 2>&1 | tee /tmp/display-server.log || RESULT=$?
|
||||
else
|
||||
echo "xwfb-run not found, skipping"
|
||||
RESULT=1
|
||||
fi
|
||||
|
||||
# --- Fallback: plain Xvfb ---
|
||||
if [ $RESULT -ne 0 ] || [ ! -f /tmp/screenshot.png ]; then
|
||||
echo "=== Falling back to Xvfb ==="
|
||||
Xvfb :99 -screen 0 1440x900x24 &
|
||||
XVFB_PID=$!
|
||||
export DISPLAY=:99
|
||||
sleep 2
|
||||
bash /tmp/smoke-test.sh 2>&1 | tee /tmp/display-server.log
|
||||
kill $XVFB_PID 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# --- Copy screenshot to workspace ---
|
||||
cp /tmp/screenshot.png screenshot.png 2>/dev/null || true
|
||||
|
||||
# --- Check results ---
|
||||
if grep -q "APP_STATUS=crashed" /tmp/display-server.log 2>/dev/null; then
|
||||
echo "❌ AppImage crashed during startup"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if grep -q "APP_STATUS=running" /tmp/display-server.log 2>/dev/null; then
|
||||
echo "✅ AppImage launched successfully"
|
||||
else
|
||||
echo "⚠️ Could not determine app status"
|
||||
fi
|
||||
|
||||
# --- Check screenshot has non-black content ---
|
||||
if [ -f screenshot.png ]; then
|
||||
COLORS=$(identify -verbose screenshot.png 2>/dev/null | grep "Colors:" | awk '{print $2}')
|
||||
echo "Screenshot unique colors: ${COLORS:-unknown}"
|
||||
if [ "${COLORS:-0}" -le 5 ]; then
|
||||
echo "⚠️ Screenshot appears blank (only $COLORS colors). App may not have rendered."
|
||||
else
|
||||
echo "✅ Screenshot has content ($COLORS unique colors)"
|
||||
fi
|
||||
fi
|
||||
|
||||
- name: Upload smoke test screenshot
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: linux-smoke-test-screenshot
|
||||
path: screenshot.png
|
||||
if-no-files-found: warn
|
||||
|
||||
- name: Upload logs
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: linux-smoke-test-logs
|
||||
path: |
|
||||
/tmp/display-server.log
|
||||
/tmp/app.log
|
||||
if-no-files-found: warn
|
||||
@@ -0,0 +1,23 @@
|
||||
name: Typecheck
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
paths-ignore:
|
||||
- '*.md'
|
||||
- '.planning/**'
|
||||
- 'docs/**'
|
||||
- 'e2e/**'
|
||||
- 'scripts/**'
|
||||
|
||||
jobs:
|
||||
typecheck:
|
||||
if: github.event.pull_request.head.repo.full_name == github.repository
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '22'
|
||||
cache: 'npm'
|
||||
- run: npm ci
|
||||
- run: npm run typecheck
|
||||
+10
-1
@@ -1,6 +1,5 @@
|
||||
node_modules/
|
||||
.idea/
|
||||
.planning/
|
||||
dist/
|
||||
.DS_Store
|
||||
*.log
|
||||
@@ -8,6 +7,8 @@ dist/
|
||||
.env.local
|
||||
.playwright-mcp/
|
||||
.vercel
|
||||
api/\[domain\]/v1/\[rpc\].js
|
||||
api/\[\[...path\]\].js
|
||||
.claude/
|
||||
.cursor/
|
||||
CLAUDE.md
|
||||
@@ -18,6 +19,14 @@ CLAUDE.md
|
||||
.windsurf/
|
||||
skills/
|
||||
ideas/
|
||||
docs/internal/
|
||||
test-results/
|
||||
src-tauri/sidecar/node/*
|
||||
!src-tauri/sidecar/node/.gitkeep
|
||||
|
||||
# AI planning session state
|
||||
.planning/
|
||||
|
||||
# Compiled sebuf gateway bundle (built by scripts/build-sidecar-sebuf.mjs)
|
||||
api/[[][[].*.js
|
||||
.claudedocs/
|
||||
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
echo "Running type check..."
|
||||
npm run typecheck || exit 1
|
||||
|
||||
echo "Running Vite build (catches esbuild errors in server/)..."
|
||||
npm run build:full || exit 1
|
||||
|
||||
echo "Running version sync check..."
|
||||
npm run version:check || exit 1
|
||||
@@ -6,5 +6,5 @@
|
||||
"MD022": true,
|
||||
"MD032": true
|
||||
},
|
||||
"ignores": ["node_modules/**", "dist/**", "src-tauri/target/**"]
|
||||
"ignores": ["node_modules/**", "dist/**", "src-tauri/target/**", ".planning/**"]
|
||||
}
|
||||
|
||||
+127
@@ -2,6 +2,133 @@
|
||||
|
||||
All notable changes to World Monitor are documented here.
|
||||
|
||||
## [2.5.10] - 2026-02-26
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Yahoo Finance rate-limit UX**: Show "rate limited — retrying shortly" instead of generic "Failed to load" on Markets, ETF, Commodities, and Sector panels when Yahoo returns 429 (#407)
|
||||
- **Sequential Yahoo calls**: Replace `Promise.all` with staggered batching in commodity quotes, ETF flows, and macro signals to prevent 429 rate limiting (#406)
|
||||
- **Sector heatmap Yahoo fallback**: Sector data now loads via Yahoo Finance when `FINNHUB_API_KEY` is missing (#406)
|
||||
- **Finnhub-to-Yahoo fallback**: Market quotes route Finnhub symbols through Yahoo when API key is not configured (#407)
|
||||
- **ETF early-exit on rate limit**: Skip retry loop and show rate-limit message immediately instead of waiting 60s (#407)
|
||||
- **Sidecar auth resilience**: 401-retry with token refresh for stale sidecar tokens after restart; `diagFetch` auth helper for settings window diagnostics (#407)
|
||||
- **Verbose toggle persistence**: Write verbose state to writable data directory instead of read-only app bundle on macOS (#407)
|
||||
- **AI summary verbosity**: Tighten prompts to 2 sentences / 60 words max with `max_tokens` reduced from 150 to 100 (#404)
|
||||
- **Settings modal title**: Rename from "PANELS" to "SETTINGS" across all 17 locales (#403)
|
||||
- **Sentry noise filters**: CSS.escape() for news ID selectors, player.destroy guard, 11 new ignoreErrors patterns, blob: URL extension frame filter (#402)
|
||||
|
||||
---
|
||||
|
||||
## [2.5.6] - 2026-02-23
|
||||
|
||||
### Added
|
||||
|
||||
- **Greek (Ελληνικά) locale** — full translation of all 1,397 i18n keys (#256)
|
||||
- **Nigeria RSS feeds** — 5 new sources: Premium Times, Vanguard, Channels TV, Daily Trust, ThisDay Live
|
||||
- **Greek locale feeds** — Naftemporiki, in.gr, iefimerida.gr for Greek-language news coverage
|
||||
- **Brasil Paralelo source** — Brazilian news with RSS feed and source tier (#260)
|
||||
|
||||
### Performance
|
||||
|
||||
- **AIS relay optimization** — backpressure queue with configurable watermarks, spatial indexing for chokepoint detection (O(chokepoints) vs O(chokepoints × vessels)), pre-serialized + pre-gzipped snapshot cache eliminating per-request JSON.stringify + gzip CPU (#266)
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Vietnam flag country code** — corrected flag emoji in language selector (#245)
|
||||
- **Sentry noise filters** — added patterns for SW FetchEvent, PostHog ingest; enabled SW POST method for PostHog analytics (#246)
|
||||
- **Service Worker same-origin routing** — restricted SW route patterns to same-origin only, preventing cross-origin fetch interception (#247, #251)
|
||||
- **Social preview bot allowlisting** — whitelisted Twitterbot, facebookexternalhit, and other crawlers on OG image assets (#251)
|
||||
- **Windows CORS for Tauri** — allow `http://` origin from `tauri.localhost` for Windows desktop builds (#262)
|
||||
- **Linux AppImage GLib crash** — fix GLib symbol mismatch on newer distros by bundling compatible libraries (#263)
|
||||
|
||||
---
|
||||
|
||||
## [2.5.2] - 2026-02-21
|
||||
|
||||
### Fixed
|
||||
|
||||
- **QuotaExceededError handling** — detect storage quota exhaustion and stop further writes to localStorage/IndexedDB instead of silently failing; shared `markStorageQuotaExceeded()` flag across persistent-cache and utility storage
|
||||
- **deck.gl null.getProjection crash** — wrap `setProps()` calls in try/catch to survive map mid-teardown races in debounced/RAF callbacks
|
||||
- **MapLibre "Style is not done loading"** — guard `setFilter()` in mousemove/mouseout handlers during theme switches
|
||||
- **YouTube invalid video ID** — validate video ID format (`/^[\w-]{10,12}$/`) before passing to IFrame Player constructor
|
||||
- **Vercel build skip on empty SHA** — guard `ignoreCommand` against unset `VERCEL_GIT_PREVIOUS_SHA` (first deploy, force deploy) which caused `git diff` to fail and cancel builds
|
||||
- **Sentry noise filters** — added 7 patterns: iOS readonly property, SW FetchEvent, toLowerCase/trim/indexOf injections, QuotaExceededError
|
||||
|
||||
---
|
||||
|
||||
## [2.5.1] - 2026-02-20
|
||||
|
||||
### Performance
|
||||
|
||||
- **Batch FRED API requests** — frontend now sends a single request with comma-separated series IDs instead of 7 parallel edge function invocations, eliminating Vercel 25s timeouts
|
||||
- **Parallel UCDP page fetches** — replaced sequential loop with Promise.all for up to 12 pages, cutting fetch time from ~96s worst-case to ~8s
|
||||
- **Bot protection middleware** — blocks known social-media crawlers from hitting API routes, reducing unnecessary edge function invocations
|
||||
- **Extended API cache TTLs** — country-intel 12h→24h, GDELT 2h→4h, nuclear 12h→24h; Vercel ignoreCommand skips non-code deploys
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Partial UCDP cache poisoning** — failed page fetches no longer silently produce incomplete results cached for 6h; partial results get 10-min TTL in both Redis and memory, with `partial: true` flag propagated to CDN cache headers
|
||||
- **FRED upstream error masking** — single-series failures now return 502 instead of empty 200; batch mode surfaces per-series errors and returns 502 when all fail
|
||||
- **Sentry `Load failed` filter** — widened regex from `^TypeError: Load failed$` to `^TypeError: Load failed( \(.*\))?$` to catch host-suffixed variants (e.g., gamma-api.polymarket.com)
|
||||
- **Tooltip XSS hardening** — replaced `rawHtml()` with `safeHtml()` allowlist sanitizer for panel info tooltips
|
||||
- **UCDP country endpoint** — added missing HTTP method guards (OPTIONS/GET)
|
||||
- **Middleware exact path matching** — social preview bot allowlist uses `Set.has()` instead of `startsWith()` prefix matching
|
||||
|
||||
### Changed
|
||||
|
||||
- FRED batch API supports up to 15 comma-separated series IDs with deduplication
|
||||
- Missing FRED API key returns 200 with `X-Data-Status: skipped-no-api-key` header instead of silent empty response
|
||||
- LAYER_TO_SOURCE config extracted from duplicate inline mappings into shared constant
|
||||
|
||||
---
|
||||
|
||||
## [2.5.0] - 2026-02-20
|
||||
|
||||
### Highlights
|
||||
|
||||
**Local LLM Support (Ollama / LM Studio)** — Run AI summarization entirely on your own hardware with zero cloud dependency. The desktop app auto-discovers models from any OpenAI-compatible local inference server (Ollama, LM Studio, llama.cpp, vLLM) and populates a selection dropdown. A 4-tier fallback chain ensures summaries always generate: Local LLM → Groq → OpenRouter → browser-side T5. Combined with the Tauri desktop app, this enables fully air-gapped intelligence analysis where no data leaves your machine.
|
||||
|
||||
### Added
|
||||
|
||||
- **Ollama / LM Studio integration** — local AI summarization via OpenAI-compatible `/v1/chat/completions` endpoint with automatic model discovery, embedding model filtering, and fallback to manual text input
|
||||
- **4-tier summarization fallback chain** — Ollama (local) → Groq (cloud) → OpenRouter (cloud) → Transformers.js T5 (browser), each with 5-second timeout before silently advancing to the next
|
||||
- **Shared summarization handler factory** — all three API tiers use identical logic for headline deduplication (Jaccard >0.6), variant-aware prompting, language-aware output, and Redis caching (`summary:v3:{mode}:{variant}:{lang}:{hash}`)
|
||||
- **Settings window with 3 tabs** — dedicated **LLMs** tab (Ollama endpoint/model, Groq, OpenRouter), **API Keys** tab (12+ data source credentials), and **Debug & Logs** tab (traffic log, verbose mode, log file access). Each tab runs an independent verification pipeline
|
||||
- **Consolidated keychain vault** — all desktop secrets stored as a single JSON blob in one OS keychain entry (`secrets-vault`), reducing macOS Keychain authorization prompts from 20+ to exactly 1 on app startup. One-time auto-migration from individual entries with cleanup
|
||||
- **Cross-window secret synchronization** — saving credentials in the Settings window immediately syncs to the main dashboard via `localStorage` broadcast, with no app restart needed
|
||||
- **API key verification pipeline** — each credential is validated against its provider's actual API endpoint. Network errors (timeouts, DNS failures) soft-pass to prevent transient failures from blocking key storage; only explicit 401/403 marks a key invalid
|
||||
- **Plaintext URL inputs** — endpoint URLs (Ollama API, relay URLs, model names) display as readable text instead of masked password dots in Settings
|
||||
- **5 new defense/intel RSS feeds** — Military Times, Task & Purpose, USNI News, Oryx OSINT, UK Ministry of Defence
|
||||
- **Koeberg nuclear power plant** — added to the nuclear facilities map layer (the only commercial reactor in Africa, Cape Town, South Africa)
|
||||
- **Privacy & Offline Architecture** documentation — README now details the three privacy levels: full cloud, desktop with cloud APIs, and air-gapped local with Ollama
|
||||
- **AI Summarization Chain** documentation — README includes provider fallback flow diagram and detailed explanation of headline deduplication, variant-aware prompting, and cross-user cache deduplication
|
||||
|
||||
### Changed
|
||||
|
||||
- AI fallback chain now starts with Ollama (local) before cloud providers
|
||||
- Feature toggles increased from 14 to 15 (added AI/Ollama)
|
||||
- Desktop architecture uses consolidated vault instead of per-key keychain entries
|
||||
- README expanded with ~85 lines of new content covering local LLM support, privacy architecture, summarization chain internals, and desktop readiness framework
|
||||
|
||||
### Fixed
|
||||
|
||||
- URL and model fields in Settings display as plaintext instead of masked password dots
|
||||
- OpenAI-compatible endpoint flow hardened for Ollama/LM Studio response format differences (thinking tokens, missing `choices` array edge cases)
|
||||
- Sentry null guard for `getProjection()` crash with 6 additional noise filters
|
||||
- PathLayer cache cleared on layer toggle-off to prevent stale WebGL buffer rendering
|
||||
|
||||
---
|
||||
|
||||
## [2.4.1] - 2026-02-19
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Map PathLayer cache**: Clear PathLayer on toggle-off to prevent stale WebGL buffers
|
||||
- **Sentry noise**: Null guard for `getProjection()` crash and 6 additional noise filters
|
||||
- **Markdown docs**: Resolve lint errors in documentation files
|
||||
|
||||
---
|
||||
|
||||
## [2.4.0] - 2026-02-19
|
||||
|
||||
### Added
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
# Contributor Covenant Code of Conduct
|
||||
|
||||
## Our Pledge
|
||||
|
||||
We as members, contributors, and leaders pledge to make participation in the
|
||||
World Monitor community a harassment-free experience for everyone, regardless of age, body
|
||||
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
||||
identity and expression, level of experience, education, socio-economic status,
|
||||
nationality, personal appearance, race, caste, color, religion, or sexual
|
||||
identity and orientation.
|
||||
|
||||
We pledge to act and interact in ways that contribute to an open, welcoming,
|
||||
diverse, inclusive, and healthy community.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Examples of behavior that contributes to a positive environment for our
|
||||
community include:
|
||||
|
||||
* Demonstrating empathy and kindness toward other people
|
||||
* Being respectful of differing opinions, viewpoints, and experiences
|
||||
* Giving and gracefully accepting constructive feedback
|
||||
* Accepting responsibility and apologizing to those affected by our mistakes,
|
||||
and learning from the experience
|
||||
* Focusing on what is best not just for us as individuals, but for the overall
|
||||
community
|
||||
|
||||
Examples of unacceptable behavior include:
|
||||
|
||||
* The use of sexualized language or imagery, and sexual attention or advances of
|
||||
any kind
|
||||
* Trolling, insulting or derogatory comments, and personal or political attacks
|
||||
* Public or private harassment
|
||||
* Publishing others' private information, such as a physical or email address,
|
||||
without their explicit permission
|
||||
* Other conduct which could reasonably be considered inappropriate in a
|
||||
professional setting
|
||||
|
||||
## Scope
|
||||
|
||||
This Code of Conduct applies within all community spaces (GitHub issues, pull
|
||||
requests, discussions, and any associated communication channels) and also
|
||||
applies when an individual is officially representing the community in public
|
||||
spaces.
|
||||
|
||||
## Enforcement
|
||||
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
||||
reported to the project maintainer at **[GitHub Issues](https://github.com/koala73/worldmonitor/issues)** or by contacting the
|
||||
repository owner directly through GitHub.
|
||||
|
||||
All complaints will be reviewed and investigated promptly and fairly. The project
|
||||
team is obligated to maintain confidentiality with regard to the reporter of an
|
||||
incident.
|
||||
|
||||
## Enforcement Guidelines
|
||||
|
||||
Community leaders will follow these Community Impact Guidelines in determining
|
||||
the consequences for any action they deem in violation of this Code of Conduct:
|
||||
|
||||
### 1. Correction
|
||||
|
||||
**Community Impact**: Use of inappropriate language or other behavior deemed
|
||||
unprofessional or unwelcome in the community.
|
||||
|
||||
**Consequence**: A private, written warning from community leaders, providing
|
||||
clarity around the nature of the violation and an explanation of why the
|
||||
behavior was inappropriate. A public apology may be requested.
|
||||
|
||||
### 2. Warning
|
||||
|
||||
**Community Impact**: A violation through a single incident or series of
|
||||
actions.
|
||||
|
||||
**Consequence**: A warning with consequences for continued behavior. No
|
||||
interaction with the people involved, including unsolicited interaction with
|
||||
those enforcing the Code of Conduct, for a specified period of time. This
|
||||
includes avoiding interactions in community spaces as well as external channels
|
||||
like social media. Violating these terms may lead to a temporary or permanent
|
||||
ban.
|
||||
|
||||
### 3. Temporary Ban
|
||||
|
||||
**Community Impact**: A serious violation of community standards, including
|
||||
sustained inappropriate behavior.
|
||||
|
||||
**Consequence**: A temporary ban from any sort of interaction or public
|
||||
communication with the community for a specified period of time. No public or
|
||||
private interaction with the people involved, including unsolicited interaction
|
||||
with those enforcing the Code of Conduct, is allowed during this period.
|
||||
Violating these terms may lead to a permanent ban.
|
||||
|
||||
### 4. Permanent Ban
|
||||
|
||||
**Community Impact**: Demonstrating a pattern of violation of community
|
||||
standards, including sustained inappropriate behavior, harassment of an
|
||||
individual, or aggression toward or disparagement of classes of individuals.
|
||||
|
||||
**Consequence**: A permanent ban from any sort of public interaction within the
|
||||
community.
|
||||
|
||||
## Attribution
|
||||
|
||||
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
||||
version 2.1, available at
|
||||
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
|
||||
|
||||
Community Impact Guidelines were inspired by
|
||||
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
|
||||
|
||||
For answers to common questions about this code of conduct, see the FAQ at
|
||||
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
|
||||
[https://www.contributor-covenant.org/translations][translations].
|
||||
|
||||
[homepage]: https://www.contributor-covenant.org
|
||||
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
|
||||
[Mozilla CoC]: https://github.com/mozilla/diversity
|
||||
[FAQ]: https://www.contributor-covenant.org/faq
|
||||
[translations]: https://www.contributor-covenant.org/translations
|
||||
+301
@@ -0,0 +1,301 @@
|
||||
# Contributing to World Monitor
|
||||
|
||||
Thank you for your interest in contributing to World Monitor! This project thrives on community contributions — whether it's code, data sources, documentation, or bug reports.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Architecture Overview](#architecture-overview)
|
||||
- [Getting Started](#getting-started)
|
||||
- [Development Setup](#development-setup)
|
||||
- [How to Contribute](#how-to-contribute)
|
||||
- [Pull Request Process](#pull-request-process)
|
||||
- [AI-Assisted Development](#ai-assisted-development)
|
||||
- [Coding Standards](#coding-standards)
|
||||
- [Working with Sebuf (RPC Framework)](#working-with-sebuf-rpc-framework)
|
||||
- [Adding Data Sources](#adding-data-sources)
|
||||
- [Adding RSS Feeds](#adding-rss-feeds)
|
||||
- [Reporting Bugs](#reporting-bugs)
|
||||
- [Feature Requests](#feature-requests)
|
||||
- [Code of Conduct](#code-of-conduct)
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
World Monitor is a real-time OSINT dashboard built with **Vanilla TypeScript** (no UI framework), **MapLibre GL + deck.gl** for map rendering, and a custom Proto-first RPC framework called **Sebuf** for all API communication.
|
||||
|
||||
### Key Technologies
|
||||
|
||||
| Technology | Purpose |
|
||||
|---|---|
|
||||
| **TypeScript** | All code — frontend, edge functions, and handlers |
|
||||
| **Vite** | Build tool and dev server |
|
||||
| **Sebuf** | Proto-first HTTP RPC framework for typed API contracts |
|
||||
| **Protobuf / Buf** | Service and message definitions across 17 domains |
|
||||
| **MapLibre GL** | Base map rendering (tiles, globe mode, camera) |
|
||||
| **deck.gl** | WebGL overlay layers (scatterplot, geojson, arcs, heatmaps) |
|
||||
| **d3** | Charts, sparklines, and data visualization |
|
||||
| **Vercel Edge Functions** | Serverless API gateway |
|
||||
| **Tauri v2** | Desktop app (Windows, macOS, Linux) |
|
||||
| **Convex** | Minimal backend (beta interest registration only) |
|
||||
| **Playwright** | End-to-end and visual regression testing |
|
||||
|
||||
### Variant System
|
||||
|
||||
The codebase produces three app variants from the same source, each targeting a different audience:
|
||||
|
||||
| Variant | Command | Focus |
|
||||
|---|---|---|
|
||||
| `full` | `npm run dev` | Geopolitics, military, conflicts, infrastructure |
|
||||
| `tech` | `npm run dev:tech` | Startups, AI/ML, cloud, cybersecurity |
|
||||
| `finance` | `npm run dev:finance` | Markets, trading, central banks, commodities |
|
||||
|
||||
Variants share all code but differ in default panels, map layers, and RSS feeds. Variant configs live in `src/config/variants/`.
|
||||
|
||||
### Directory Structure
|
||||
|
||||
| Directory | Purpose |
|
||||
|---|---|
|
||||
| `src/components/` | UI components — Panel subclasses, map, modals (~50 panels) |
|
||||
| `src/services/` | Data fetching modules — sebuf client wrappers, AI, signal analysis |
|
||||
| `src/config/` | Static data and variant configs (feeds, geo, military, pipelines, ports) |
|
||||
| `src/generated/` | Auto-generated sebuf client + server stubs (**do not edit by hand**) |
|
||||
| `src/types/` | TypeScript type definitions |
|
||||
| `src/locales/` | i18n JSON files (14 languages) |
|
||||
| `src/workers/` | Web Workers for analysis |
|
||||
| `server/` | Sebuf handler implementations for all 17 domain services |
|
||||
| `api/` | Vercel Edge Functions (sebuf gateway + legacy endpoints) |
|
||||
| `proto/` | Protobuf service and message definitions |
|
||||
| `data/` | Static JSON datasets |
|
||||
| `docs/` | Documentation + generated OpenAPI specs |
|
||||
| `src-tauri/` | Tauri v2 Rust app + Node.js sidecar for desktop builds |
|
||||
| `e2e/` | Playwright end-to-end tests |
|
||||
| `scripts/` | Build and packaging scripts |
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. **Fork** the repository on GitHub
|
||||
2. **Clone** your fork locally:
|
||||
```bash
|
||||
git clone https://github.com/<your-username>/worldmonitor.git
|
||||
cd worldmonitor
|
||||
```
|
||||
3. **Create a branch** for your work:
|
||||
```bash
|
||||
git checkout -b feature/your-feature-name
|
||||
```
|
||||
|
||||
## Development Setup
|
||||
|
||||
```bash
|
||||
# Install everything (buf CLI, sebuf plugins, npm deps, Playwright browsers)
|
||||
make install
|
||||
|
||||
# Start the development server (full variant, default)
|
||||
npm run dev
|
||||
|
||||
# Start other variants
|
||||
npm run dev:tech
|
||||
npm run dev:finance
|
||||
|
||||
# Run type checking
|
||||
npm run typecheck
|
||||
|
||||
# Run tests
|
||||
npm run test:data # Data integrity tests
|
||||
npm run test:e2e # Playwright end-to-end tests
|
||||
|
||||
# Production build (per variant)
|
||||
npm run build # full
|
||||
npm run build:tech
|
||||
npm run build:finance
|
||||
```
|
||||
|
||||
The dev server runs at `http://localhost:3000`. Run `make help` to see all available make targets.
|
||||
|
||||
### Environment Variables (Optional)
|
||||
|
||||
For full functionality, copy `.env.example` to `.env.local` and fill in the API keys you need. The app runs without any API keys — external data sources will simply be unavailable.
|
||||
|
||||
See [API Dependencies](docs/DOCUMENTATION.md#api-dependencies) for the full list.
|
||||
|
||||
## How to Contribute
|
||||
|
||||
### Types of Contributions We Welcome
|
||||
|
||||
- **Bug fixes** — found something broken? Fix it!
|
||||
- **New data layers** — add new geospatial data sources to the map
|
||||
- **RSS feeds** — expand our 100+ feed collection with quality sources
|
||||
- **UI/UX improvements** — make the dashboard more intuitive
|
||||
- **Performance optimizations** — faster loading, better caching
|
||||
- **Documentation** — improve docs, add examples, fix typos
|
||||
- **Accessibility** — make the dashboard usable by everyone
|
||||
- **Internationalization** — help make World Monitor available in more languages
|
||||
- **Tests** — add unit or integration tests
|
||||
|
||||
### What We're Especially Looking For
|
||||
|
||||
- New data layers (see [Adding Data Sources](#adding-data-sources))
|
||||
- Feed quality improvements and new RSS sources
|
||||
- Mobile responsiveness improvements
|
||||
- Performance optimizations for the map rendering pipeline
|
||||
- Better anomaly detection algorithms
|
||||
|
||||
## Pull Request Process
|
||||
|
||||
1. **Update documentation** if your change affects the public API or user-facing behavior
|
||||
2. **Run type checking** before submitting: `npm run typecheck`
|
||||
3. **Test your changes** locally with at least the `full` variant, and any other variant your change affects
|
||||
4. **Keep PRs focused** — one feature or fix per pull request
|
||||
5. **Write a clear description** explaining what your PR does and why
|
||||
6. **Link related issues** if applicable
|
||||
|
||||
### PR Title Convention
|
||||
|
||||
Use a descriptive title that summarizes the change:
|
||||
|
||||
- `feat: add earthquake magnitude filtering to map layer`
|
||||
- `fix: resolve RSS feed timeout for Al Jazeera`
|
||||
- `docs: update API dependencies section`
|
||||
- `perf: optimize marker clustering at low zoom levels`
|
||||
- `refactor: extract threat classifier into separate module`
|
||||
|
||||
### Review Process
|
||||
|
||||
- All PRs require review from a maintainer before merging
|
||||
- Maintainers may request changes — this is normal and collaborative
|
||||
- Once approved, a maintainer will merge your PR
|
||||
|
||||
## AI-Assisted Development
|
||||
|
||||
We fully embrace AI-assisted development. Many of our own PRs are labeled with the LLM that helped produce them (e.g., `claude`, `codex`, `cursor`), and contributors are welcome to use any AI tools they find helpful.
|
||||
|
||||
That said, **all code is held to the same quality bar regardless of how it was written**. AI-generated code will be reviewed with the same scrutiny as human-written code. Contributors are responsible for understanding and being able to explain every line they submit. Blindly pasting LLM output without review is discouraged — treat AI as a collaborator, not a replacement for your own judgement.
|
||||
|
||||
## Coding Standards
|
||||
|
||||
### TypeScript
|
||||
|
||||
- Use TypeScript for all new code
|
||||
- Avoid `any` types — use proper typing or `unknown` with type guards
|
||||
- Export interfaces/types for public APIs
|
||||
- Use meaningful variable and function names
|
||||
|
||||
### Code Style
|
||||
|
||||
- Follow the existing code style in the repository
|
||||
- Use `const` by default, `let` when reassignment is needed
|
||||
- Prefer functional patterns (map, filter, reduce) over imperative loops
|
||||
- Keep functions focused — one responsibility per function
|
||||
- Add JSDoc comments for exported functions and complex logic
|
||||
|
||||
### File Organization
|
||||
|
||||
- Static layer/geo data and variant configs go in `src/config/`
|
||||
- Sebuf handler implementations go in `server/worldmonitor/{domain}/v1/`
|
||||
- Edge function gateway and legacy endpoints go in `api/`
|
||||
- UI components (panels, map, modals) go in `src/components/`
|
||||
- Service modules (data fetching, client wrappers) go in `src/services/`
|
||||
- Proto definitions go in `proto/worldmonitor/{domain}/v1/`
|
||||
|
||||
## Working with Sebuf (RPC Framework)
|
||||
|
||||
Sebuf is the project's custom Proto-first HTTP RPC framework — a lightweight alternative to gRPC-Web. All API communication between client and server uses Sebuf.
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Proto definitions** in `proto/worldmonitor/{domain}/v1/` define services and messages
|
||||
2. **Code generation** (`make generate`) produces:
|
||||
- TypeScript clients in `src/generated/client/` (e.g., `MarketServiceClient`)
|
||||
- Server route factories in `src/generated/server/` (e.g., `createMarketServiceRoutes`)
|
||||
3. **Handlers** in `server/worldmonitor/{domain}/v1/handler.ts` implement the service interface
|
||||
4. **Gateway** in `api/[domain]/v1/[rpc].ts` registers all handlers and routes requests
|
||||
5. **Clients** in `src/services/{domain}/index.ts` wrap the generated client for app use
|
||||
|
||||
### Adding a New RPC Method
|
||||
|
||||
1. Add the method to the `.proto` service definition
|
||||
2. Run `make generate` to regenerate client/server stubs
|
||||
3. Implement the handler method in the domain's `handler.ts`
|
||||
4. The client stub is auto-generated — use it from `src/services/{domain}/`
|
||||
|
||||
Use `make lint` to lint proto files and `make breaking` to check for breaking changes against main.
|
||||
|
||||
### Proto Conventions
|
||||
|
||||
- **Time fields**: Use `int64` (Unix epoch milliseconds), not `google.protobuf.Timestamp`
|
||||
- **int64 encoding**: Apply `[(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER]` on time fields so TypeScript receives `number` instead of `string`
|
||||
- **HTTP annotations**: Every RPC method needs `option (sebuf.http.config) = { path: "...", method: POST }`
|
||||
|
||||
### Proto Codegen Requirements
|
||||
|
||||
Run `make install` to install everything automatically, or install individually:
|
||||
|
||||
```bash
|
||||
make install-buf # Install buf CLI (requires Go)
|
||||
make install-plugins # Install sebuf protoc-gen plugins (requires Go)
|
||||
```
|
||||
|
||||
## Adding Data Sources
|
||||
|
||||
To add a new data layer to the map:
|
||||
|
||||
1. **Define the data source** — identify the API or dataset you want to integrate
|
||||
2. **Add the proto service** (if the data needs a backend proxy) — define messages and RPC methods in `proto/worldmonitor/{domain}/v1/`
|
||||
3. **Generate stubs** — run `make generate`
|
||||
4. **Implement the handler** in `server/worldmonitor/{domain}/v1/`
|
||||
5. **Register the handler** in `api/[domain]/v1/[rpc].ts` and `vite.config.ts` (for local dev)
|
||||
6. **Create the service module** in `src/services/{domain}/` wrapping the generated client
|
||||
7. **Add the layer config** and implement the map renderer following existing layer patterns
|
||||
8. **Add to layer toggles** — make it toggleable in the UI
|
||||
9. **Document the source** — add it to `docs/DOCUMENTATION.md`
|
||||
|
||||
For endpoints that deal with non-JSON payloads (XML feeds, binary data, HTML embeds), you can add a standalone Edge Function in `api/` instead of Sebuf. For anything returning JSON, prefer Sebuf — the typed contracts are always worth it.
|
||||
|
||||
### Data Source Requirements
|
||||
|
||||
- Must be freely accessible (no paid-only APIs for core functionality)
|
||||
- Must have a permissive license or be public government data
|
||||
- Should update at least daily for real-time relevance
|
||||
- Must include geographic coordinates or be geo-locatable
|
||||
|
||||
## Adding RSS Feeds
|
||||
|
||||
To add new RSS feeds:
|
||||
|
||||
1. Verify the feed is reliable and actively maintained
|
||||
2. Assign a **source tier** (1-4) based on editorial reliability
|
||||
3. Flag any **state affiliation** or **propaganda risk**
|
||||
4. Categorize the feed (geopolitics, defense, energy, tech, etc.)
|
||||
5. Test that the feed parses correctly through the RSS proxy
|
||||
|
||||
## Reporting Bugs
|
||||
|
||||
When filing a bug report, please include:
|
||||
|
||||
- **Description** — clear description of the issue
|
||||
- **Steps to reproduce** — how to trigger the bug
|
||||
- **Expected behavior** — what should happen
|
||||
- **Actual behavior** — what actually happens
|
||||
- **Screenshots** — if applicable
|
||||
- **Browser/OS** — your environment details
|
||||
- **Console errors** — any relevant browser console output
|
||||
|
||||
Use the [Bug Report issue template](https://github.com/koala73/worldmonitor/issues/new/choose) when available.
|
||||
|
||||
## Feature Requests
|
||||
|
||||
We welcome feature ideas! When suggesting a feature:
|
||||
|
||||
- **Describe the problem** it solves
|
||||
- **Propose a solution** with as much detail as possible
|
||||
- **Consider alternatives** you've thought about
|
||||
- **Provide context** — who would benefit from this feature?
|
||||
|
||||
Use the [Feature Request issue template](https://github.com/koala73/worldmonitor/issues/new/choose) when available.
|
||||
|
||||
## Code of Conduct
|
||||
|
||||
This project follows the [Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior through GitHub issues or by contacting the repository owner.
|
||||
|
||||
---
|
||||
|
||||
Thank you for helping make World Monitor better! 🌍
|
||||
@@ -1,21 +1,669 @@
|
||||
MIT License
|
||||
World Monitor — Real-time global intelligence dashboard
|
||||
Copyright (C) 2024-2026 Elie Habib
|
||||
|
||||
Copyright (c) 2025-2026 Elie Habib
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU Affero General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
GNU AFFERO GENERAL PUBLIC LICENSE
|
||||
Version 3, 19 November 2007
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
||||
Everyone is permitted to copy and distribute verbatim copies
|
||||
of this license document, but changing it is not allowed.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
Preamble
|
||||
|
||||
The GNU Affero General Public License is a free, copyleft license for
|
||||
software and other kinds of works, specifically designed to ensure
|
||||
cooperation with the community in the case of network server software.
|
||||
|
||||
The licenses for most software and other practical works are designed
|
||||
to take away your freedom to share and change the works. By contrast,
|
||||
our General Public Licenses are intended to guarantee your freedom to
|
||||
share and change all versions of a program--to make sure it remains free
|
||||
software for all its users.
|
||||
|
||||
When we speak of free software, we are referring to freedom, not
|
||||
price. Our General Public Licenses are designed to make sure that you
|
||||
have the freedom to distribute copies of free software (and charge for
|
||||
them if you wish), that you receive source code or can get it if you
|
||||
want it, that you can change the software or use pieces of it in new
|
||||
free programs, and that you know you can do these things.
|
||||
|
||||
Developers that use our General Public Licenses protect your rights
|
||||
with two steps: (1) assert copyright on the software, and (2) offer
|
||||
you this License which gives you legal permission to copy, distribute
|
||||
and/or modify the software.
|
||||
|
||||
A secondary benefit of defending all users' freedom is that
|
||||
improvements made in alternate versions of the program, if they
|
||||
receive widespread use, become available for other developers to
|
||||
incorporate. Many developers of free software are heartened and
|
||||
encouraged by the resulting cooperation. However, in the case of
|
||||
software used on network servers, this result may fail to come about.
|
||||
The GNU General Public License permits making a modified version and
|
||||
letting the public access it on a server without ever releasing its
|
||||
source code to the public.
|
||||
|
||||
The GNU Affero General Public License is designed specifically to
|
||||
ensure that, in such cases, the modified source code becomes available
|
||||
to the community. It requires the operator of a network server to
|
||||
provide the source code of the modified version running there to the
|
||||
users of that server. Therefore, public use of a modified version, on
|
||||
a publicly accessible server, gives the public access to the source
|
||||
code of the modified version.
|
||||
|
||||
An older license, called the Affero General Public License and
|
||||
published by Affero, was designed to accomplish similar goals. This is
|
||||
a different license, not a version of the Affero GPL, but Affero has
|
||||
released a new version of the Affero GPL which permits relicensing under
|
||||
this license.
|
||||
|
||||
The precise terms and conditions for copying, distribution and
|
||||
modification follow.
|
||||
|
||||
TERMS AND CONDITIONS
|
||||
|
||||
0. Definitions.
|
||||
|
||||
"This License" refers to version 3 of the GNU Affero General Public License.
|
||||
|
||||
"Copyright" also means copyright-like laws that apply to other kinds of
|
||||
works, such as semiconductor masks.
|
||||
|
||||
"The Program" refers to any copyrightable work licensed under this
|
||||
License. Each licensee is addressed as "you". "Licensees" and
|
||||
"recipients" may be individuals or organizations.
|
||||
|
||||
To "modify" a work means to copy from or adapt all or part of the work
|
||||
in a fashion requiring copyright permission, other than the making of an
|
||||
exact copy. The resulting work is called a "modified version" of the
|
||||
earlier work or a work "based on" the earlier work.
|
||||
|
||||
A "covered work" means either the unmodified Program or a work based
|
||||
on the Program.
|
||||
|
||||
To "propagate" a work means to do anything with it that, without
|
||||
permission, would make you directly or secondarily liable for
|
||||
infringement under applicable copyright law, except executing it on a
|
||||
computer or modifying a private copy. Propagation includes copying,
|
||||
distribution (with or without modification), making available to the
|
||||
public, and in some countries other activities as well.
|
||||
|
||||
To "convey" a work means any kind of propagation that enables other
|
||||
parties to make or receive copies. Mere interaction with a user through
|
||||
a computer network, with no transfer of a copy, is not conveying.
|
||||
|
||||
An interactive user interface displays "Appropriate Legal Notices"
|
||||
to the extent that it includes a convenient and prominently visible
|
||||
feature that (1) displays an appropriate copyright notice, and (2)
|
||||
tells the user that there is no warranty for the work (except to the
|
||||
extent that warranties are provided), that licensees may convey the
|
||||
work under this License, and how to view a copy of this License. If
|
||||
the interface presents a list of user commands or options, such as a
|
||||
menu, a prominent item in the list meets this criterion.
|
||||
|
||||
1. Source Code.
|
||||
|
||||
The "source code" for a work means the preferred form of the work
|
||||
for making modifications to it. "Object code" means any non-source
|
||||
form of a work.
|
||||
|
||||
A "Standard Interface" means an interface that either is an official
|
||||
standard defined by a recognized standards body, or, in the case of
|
||||
interfaces specified for a particular programming language, one that
|
||||
is widely used among developers working in that language.
|
||||
|
||||
The "System Libraries" of an executable work include anything, other
|
||||
than the work as a whole, that (a) is included in the normal form of
|
||||
packaging a Major Component, but which is not part of that Major
|
||||
Component, and (b) serves only to enable use of the work with that
|
||||
Major Component, or to implement a Standard Interface for which an
|
||||
implementation is available to the public in source code form. A
|
||||
"Major Component", in this context, means a major essential component
|
||||
(kernel, window system, and so on) of the specific operating system
|
||||
(if any) on which the executable work runs, or a compiler used to
|
||||
produce the work, or an object code interpreter used to run it.
|
||||
|
||||
The "Corresponding Source" for a work in object code form means all
|
||||
the source code needed to generate, install, and (for an executable
|
||||
work) run the object code and to modify the work, including scripts to
|
||||
control those activities. However, it does not include the work's
|
||||
System Libraries, or general-purpose tools or generally available free
|
||||
programs which are used unmodified in performing those activities but
|
||||
which are not part of the work. For example, Corresponding Source
|
||||
includes interface definition files associated with source files for
|
||||
the work, and the source code for shared libraries and dynamically
|
||||
linked subprograms that the work is specifically designed to require,
|
||||
such as by intimate data communication or control flow between those
|
||||
subprograms and other parts of the work.
|
||||
|
||||
The Corresponding Source need not include anything that users
|
||||
can regenerate automatically from other parts of the Corresponding
|
||||
Source.
|
||||
|
||||
The Corresponding Source for a work in source code form is that
|
||||
same work.
|
||||
|
||||
2. Basic Permissions.
|
||||
|
||||
All rights granted under this License are granted for the term of
|
||||
copyright on the Program, and are irrevocable provided the stated
|
||||
conditions are met. This License explicitly affirms your unlimited
|
||||
permission to run the unmodified Program. The output from running a
|
||||
covered work is covered by this License only if the output, given its
|
||||
content, constitutes a covered work. This License acknowledges your
|
||||
rights of fair use or other equivalent, as provided by copyright law.
|
||||
|
||||
You may make, run and propagate covered works that you do not
|
||||
convey, without conditions so long as your license otherwise remains
|
||||
in force. You may convey covered works to others for the sole purpose
|
||||
of having them make modifications exclusively for you, or provide you
|
||||
with facilities for running those works, provided that you comply with
|
||||
the terms of this License in conveying all material for which you do
|
||||
not control copyright. Those thus making or running the covered works
|
||||
for you must do so exclusively on your behalf, under your direction
|
||||
and control, on terms that prohibit them from making any copies of
|
||||
your copyrighted material outside their relationship with you.
|
||||
|
||||
Conveying under any other circumstances is permitted solely under
|
||||
the conditions stated below. Sublicensing is not allowed; section 10
|
||||
makes it unnecessary.
|
||||
|
||||
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
||||
|
||||
No covered work shall be deemed part of an effective technological
|
||||
measure under any applicable law fulfilling obligations under article
|
||||
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
||||
similar laws prohibiting or restricting circumvention of such
|
||||
measures.
|
||||
|
||||
When you convey a covered work, you waive any legal power to forbid
|
||||
circumvention of technological measures to the extent such circumvention
|
||||
is effected by exercising rights under this License with respect to
|
||||
the covered work, and you disclaim any intention to limit operation or
|
||||
modification of the work as a means of enforcing, against the work's
|
||||
users, your or third parties' legal rights to forbid circumvention of
|
||||
technological measures.
|
||||
|
||||
4. Conveying Verbatim Copies.
|
||||
|
||||
You may convey verbatim copies of the Program's source code as you
|
||||
receive it, in any medium, provided that you conspicuously and
|
||||
appropriately publish on each copy an appropriate copyright notice;
|
||||
keep intact all notices stating that this License and any
|
||||
non-permissive terms added in accord with section 7 apply to the code;
|
||||
keep intact all notices of the absence of any warranty; and give all
|
||||
recipients a copy of this License along with the Program.
|
||||
|
||||
You may charge any price or no price for each copy that you convey,
|
||||
and you may offer support or warranty protection for a fee.
|
||||
|
||||
5. Conveying Modified Source Versions.
|
||||
|
||||
You may convey a work based on the Program, or the modifications to
|
||||
produce it from the Program, in the form of source code under the
|
||||
terms of section 4, provided that you also meet all of these conditions:
|
||||
|
||||
a) The work must carry prominent notices stating that you modified
|
||||
it, and giving a relevant date.
|
||||
|
||||
b) The work must carry prominent notices stating that it is
|
||||
released under this License and any conditions added under section
|
||||
7. This requirement modifies the requirement in section 4 to
|
||||
"keep intact all notices".
|
||||
|
||||
c) You must license the entire work, as a whole, under this
|
||||
License to anyone who comes into possession of a copy. This
|
||||
License will therefore apply, along with any applicable section 7
|
||||
additional terms, to the whole of the work, and all its parts,
|
||||
regardless of how they are packaged. This License gives no
|
||||
permission to license the work in any other way, but it does not
|
||||
invalidate such permission if you have separately received it.
|
||||
|
||||
d) If the work has interactive user interfaces, each must display
|
||||
Appropriate Legal Notices; however, if the Program has interactive
|
||||
interfaces that do not display Appropriate Legal Notices, your
|
||||
work need not make them do so.
|
||||
|
||||
A compilation of a covered work with other separate and independent
|
||||
works, which are not by their nature extensions of the covered work,
|
||||
and which are not combined with it such as to form a larger program,
|
||||
in or on a volume of a storage or distribution medium, is called an
|
||||
"aggregate" if the compilation and its resulting copyright are not
|
||||
used to limit the access or legal rights of the compilation's users
|
||||
beyond what the individual works permit. Inclusion of a covered work
|
||||
in an aggregate does not cause this License to apply to the other
|
||||
parts of the aggregate.
|
||||
|
||||
6. Conveying Non-Source Forms.
|
||||
|
||||
You may convey a covered work in object code form under the terms
|
||||
of sections 4 and 5, provided that you also convey the
|
||||
machine-readable Corresponding Source under the terms of this License,
|
||||
in one of these ways:
|
||||
|
||||
a) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by the
|
||||
Corresponding Source fixed on a durable physical medium
|
||||
customarily used for software interchange.
|
||||
|
||||
b) Convey the object code in, or embodied in, a physical product
|
||||
(including a physical distribution medium), accompanied by a
|
||||
written offer, valid for at least three years and valid for as
|
||||
long as you offer spare parts or customer support for that product
|
||||
model, to give anyone who possesses the object code either (1) a
|
||||
copy of the Corresponding Source for all the software in the
|
||||
product that is covered by this License, on a durable physical
|
||||
medium customarily used for software interchange, for a price no
|
||||
more than your reasonable cost of physically performing this
|
||||
conveying of source, or (2) access to copy the
|
||||
Corresponding Source from a network server at no charge.
|
||||
|
||||
c) Convey individual copies of the object code with a copy of the
|
||||
written offer to provide the Corresponding Source. This
|
||||
alternative is allowed only occasionally and noncommercially, and
|
||||
only if you received the object code with such an offer, in accord
|
||||
with subsection 6b.
|
||||
|
||||
d) Convey the object code by offering access from a designated
|
||||
place (gratis or for a charge), and offer equivalent access to the
|
||||
Corresponding Source in the same way through the same place at no
|
||||
further charge. You need not require recipients to copy the
|
||||
Corresponding Source along with the object code. If the place to
|
||||
copy the object code is a network server, the Corresponding Source
|
||||
may be on a different server (operated by you or a third party)
|
||||
that supports equivalent copying facilities, provided you maintain
|
||||
clear directions next to the object code saying where to find the
|
||||
Corresponding Source. Regardless of what server hosts the
|
||||
Corresponding Source, you remain obligated to ensure that it is
|
||||
available for as long as needed to satisfy these requirements.
|
||||
|
||||
e) Convey the object code using peer-to-peer transmission, provided
|
||||
you inform other peers where the object code and Corresponding
|
||||
Source of the work are being offered to the general public at no
|
||||
charge under subsection 6d.
|
||||
|
||||
A separable portion of the object code, whose source code is excluded
|
||||
from the Corresponding Source as a System Library, need not be
|
||||
included in conveying the object code work.
|
||||
|
||||
A "User Product" is either (1) a "consumer product", which means any
|
||||
tangible personal property which is normally used for personal, family,
|
||||
or household purposes, or (2) anything designed or sold for incorporation
|
||||
into a dwelling. In determining whether a product is a consumer product,
|
||||
doubtful cases shall be resolved in favor of coverage. For a particular
|
||||
product received by a particular user, "normally used" refers to a
|
||||
typical or common use of that class of product, regardless of the status
|
||||
of the particular user or of the way in which the particular user
|
||||
actually uses, or expects or is expected to use, the product. A product
|
||||
is a consumer product regardless of whether the product has substantial
|
||||
commercial, industrial or non-consumer uses, unless such uses represent
|
||||
the only significant mode of use of the product.
|
||||
|
||||
"Installation Information" for a User Product means any methods,
|
||||
procedures, authorization keys, or other information required to install
|
||||
and execute modified versions of a covered work in that User Product from
|
||||
a modified version of its Corresponding Source. The information must
|
||||
suffice to ensure that the continued functioning of the modified object
|
||||
code is in no case prevented or interfered with solely because
|
||||
modification has been made.
|
||||
|
||||
If you convey an object code work under this section in, or with, or
|
||||
specifically for use in, a User Product, and the conveying occurs as
|
||||
part of a transaction in which the right of possession and use of the
|
||||
User Product is transferred to the recipient in perpetuity or for a
|
||||
fixed term (regardless of how the transaction is characterized), the
|
||||
Corresponding Source conveyed under this section must be accompanied
|
||||
by the Installation Information. But this requirement does not apply
|
||||
if neither you nor any third party retains the ability to install
|
||||
modified object code on the User Product (for example, the work has
|
||||
been installed in ROM).
|
||||
|
||||
The requirement to provide Installation Information does not include a
|
||||
requirement to continue to provide support service, warranty, or updates
|
||||
for a work that has been modified or installed by the recipient, or for
|
||||
the User Product in which it has been modified or installed. Access to a
|
||||
network may be denied when the modification itself materially and
|
||||
adversely affects the operation of the network or violates the rules and
|
||||
protocols for communication across the network.
|
||||
|
||||
Corresponding Source conveyed, and Installation Information provided,
|
||||
in accord with this section must be in a format that is publicly
|
||||
documented (and with an implementation available to the public in
|
||||
source code form), and must require no special password or key for
|
||||
unpacking, reading or copying.
|
||||
|
||||
7. Additional Terms.
|
||||
|
||||
"Additional permissions" are terms that supplement the terms of this
|
||||
License by making exceptions from one or more of its conditions.
|
||||
Additional permissions that are applicable to the entire Program shall
|
||||
be treated as though they were included in this License, to the extent
|
||||
that they are valid under applicable law. If additional permissions
|
||||
apply only to part of the Program, that part may be used separately
|
||||
under those permissions, but the entire Program remains governed by
|
||||
this License without regard to the additional permissions.
|
||||
|
||||
When you convey a copy of a covered work, you may at your option
|
||||
remove any additional permissions from that copy, or from any part of
|
||||
it. (Additional permissions may be written to require their own
|
||||
removal in certain cases when you modify the work.) You may place
|
||||
additional permissions on material, added by you to a covered work,
|
||||
for which you have or can give appropriate copyright permission.
|
||||
|
||||
Notwithstanding any other provision of this License, for material you
|
||||
add to a covered work, you may (if authorized by the copyright holders of
|
||||
that material) supplement the terms of this License with terms:
|
||||
|
||||
a) Disclaiming warranty or limiting liability differently from the
|
||||
terms of sections 15 and 16 of this License; or
|
||||
|
||||
b) Requiring preservation of specified reasonable legal notices or
|
||||
author attributions in that material or in the Appropriate Legal
|
||||
Notices displayed by works containing it; or
|
||||
|
||||
c) Prohibiting misrepresentation of the origin of that material, or
|
||||
requiring that modified versions of such material be marked in
|
||||
reasonable ways as different from the original version; or
|
||||
|
||||
d) Limiting the use for publicity purposes of names of licensors or
|
||||
authors of the material; or
|
||||
|
||||
e) Declining to grant rights under trademark law for use of some
|
||||
trade names, trademarks, or service marks; or
|
||||
|
||||
f) Requiring indemnification of licensors and authors of that
|
||||
material by anyone who conveys the material (or modified versions of
|
||||
it) with contractual assumptions of liability to the recipient, for
|
||||
any liability that these contractual assumptions directly impose on
|
||||
those licensors and authors.
|
||||
|
||||
All other non-permissive additional terms are considered "further
|
||||
restrictions" within the meaning of section 10. If the Program as you
|
||||
received it, or any part of it, contains a notice stating that it is
|
||||
governed by this License along with a term that is a further
|
||||
restriction, you may remove that term. If a license document contains
|
||||
a further restriction but permits relicensing or conveying under this
|
||||
License, you may add to a covered work material governed by the terms
|
||||
of that license document, provided that the further restriction does
|
||||
not survive such relicensing or conveying.
|
||||
|
||||
If you add terms to a covered work in accord with this section, you
|
||||
must place, in the relevant source files, a statement of the
|
||||
additional terms that apply to those files, or a notice indicating
|
||||
where to find the applicable terms.
|
||||
|
||||
Additional terms, permissive or non-permissive, may be stated in the
|
||||
form of a separately written license, or stated as exceptions;
|
||||
the above requirements apply either way.
|
||||
|
||||
8. Termination.
|
||||
|
||||
You may not propagate or modify a covered work except as expressly
|
||||
provided under this License. Any attempt otherwise to propagate or
|
||||
modify it is void, and will automatically terminate your rights under
|
||||
this License (including any patent licenses granted under the third
|
||||
paragraph of section 11).
|
||||
|
||||
However, if you cease all violation of this License, then your
|
||||
license from a particular copyright holder is reinstated (a)
|
||||
provisionally, unless and until the copyright holder explicitly and
|
||||
finally terminates your license, and (b) permanently, if the copyright
|
||||
holder fails to notify you of the violation by some reasonable means
|
||||
prior to 60 days after the cessation.
|
||||
|
||||
Moreover, your license from a particular copyright holder is
|
||||
reinstated permanently if the copyright holder notifies you of the
|
||||
violation by some reasonable means, this is the first time you have
|
||||
received notice of violation of this License (for any work) from that
|
||||
copyright holder, and you cure the violation prior to 30 days after
|
||||
your receipt of the notice.
|
||||
|
||||
Termination of your rights under this section does not terminate the
|
||||
licenses of parties who have received copies or rights from you under
|
||||
this License. If your rights have been terminated and not permanently
|
||||
reinstated, you do not qualify to receive new licenses for the same
|
||||
material under section 10.
|
||||
|
||||
9. Acceptance Not Required for Having Copies.
|
||||
|
||||
You are not required to accept this License in order to receive or
|
||||
run a copy of the Program. Ancillary propagation of a covered work
|
||||
occurring solely as a consequence of using peer-to-peer transmission
|
||||
to receive a copy likewise does not require acceptance. However,
|
||||
nothing other than this License grants you permission to propagate or
|
||||
modify any covered work. These actions infringe copyright if you do
|
||||
not accept this License. Therefore, by modifying or propagating a
|
||||
covered work, you indicate your acceptance of this License to do so.
|
||||
|
||||
10. Automatic Licensing of Downstream Recipients.
|
||||
|
||||
Each time you convey a covered work, the recipient automatically
|
||||
receives a license from the original licensors, to run, modify and
|
||||
propagate that work, subject to this License. You are not responsible
|
||||
for enforcing compliance by third parties with this License.
|
||||
|
||||
An "entity transaction" is a transaction transferring control of an
|
||||
organization, or substantially all assets of one, or subdividing an
|
||||
organization, or merging organizations. If propagation of a covered
|
||||
work results from an entity transaction, each party to that
|
||||
transaction who receives a copy of the work also receives whatever
|
||||
licenses to the work the party's predecessor in interest had or could
|
||||
give under the previous paragraph, plus a right to possession of the
|
||||
Corresponding Source of the work from the predecessor in interest, if
|
||||
the predecessor has it or can get it with reasonable efforts.
|
||||
|
||||
You may not impose any further restrictions on the exercise of the
|
||||
rights granted or affirmed under this License. For example, you may
|
||||
not impose a license fee, royalty, or other charge for exercise of
|
||||
rights granted under this License, and you may not initiate litigation
|
||||
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
||||
any patent claim is infringed by making, using, selling, offering for
|
||||
sale, or importing the Program or any portion of it.
|
||||
|
||||
11. Patents.
|
||||
|
||||
A "contributor" is a copyright holder who authorizes use under this
|
||||
License of the Program or a work on which the Program is based. The
|
||||
work thus licensed is called the contributor's "contributor version".
|
||||
|
||||
A contributor's "essential patent claims" are all patent claims
|
||||
owned or controlled by the contributor, whether already acquired or
|
||||
hereafter acquired, that would be infringed by some manner, permitted
|
||||
by this License, of making, using, or selling its contributor version,
|
||||
but do not include claims that would be infringed only as a
|
||||
consequence of further modification of the contributor version. For
|
||||
purposes of this definition, "control" includes the right to grant
|
||||
patent sublicenses in a manner consistent with the requirements of
|
||||
this License.
|
||||
|
||||
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
||||
patent license under the contributor's essential patent claims, to
|
||||
make, use, sell, offer for sale, import and otherwise run, modify and
|
||||
propagate the contents of its contributor version.
|
||||
|
||||
In the following three paragraphs, a "patent license" is any express
|
||||
agreement or commitment, however denominated, not to enforce a patent
|
||||
(such as an express permission to practice a patent or covenant not to
|
||||
sue for patent infringement). To "grant" such a patent license to a
|
||||
party means to make such an agreement or commitment not to enforce a
|
||||
patent against the party.
|
||||
|
||||
If you convey a covered work, knowingly relying on a patent license,
|
||||
and the Corresponding Source of the work is not available for anyone
|
||||
to copy, free of charge and under the terms of this License, through a
|
||||
publicly available network server or other readily accessible means,
|
||||
then you must either (1) cause the Corresponding Source to be so
|
||||
available, or (2) arrange to deprive yourself of the benefit of the
|
||||
patent license for this particular work, or (3) arrange, in a manner
|
||||
consistent with the requirements of this License, to extend the patent
|
||||
license to downstream recipients. "Knowingly relying" means you have
|
||||
actual knowledge that, but for the patent license, your conveying the
|
||||
covered work in a country, or your recipient's use of the covered work
|
||||
in a country, would infringe one or more identifiable patents in that
|
||||
country that you have reason to believe are valid.
|
||||
|
||||
If, pursuant to or in connection with a single transaction or
|
||||
arrangement, you convey, or propagate by procuring conveyance of, a
|
||||
covered work, and grant a patent license to some of the parties
|
||||
receiving the covered work authorizing them to use, propagate, modify
|
||||
or convey a specific copy of the covered work, then the patent license
|
||||
you grant is automatically extended to all recipients of the covered
|
||||
work and works based on it.
|
||||
|
||||
A patent license is "discriminatory" if it does not include within
|
||||
the scope of its coverage, prohibits the exercise of, or is
|
||||
conditioned on the non-exercise of one or more of the rights that are
|
||||
specifically granted under this License. You may not convey a covered
|
||||
work if you are a party to an arrangement with a third party that is
|
||||
in the business of distributing software, under which you make payment
|
||||
to the third party based on the extent of your activity of conveying
|
||||
the work, and under which the third party grants, to any of the
|
||||
parties who would receive the covered work from you, a discriminatory
|
||||
patent license (a) in connection with copies of the covered work
|
||||
conveyed by you (or copies made from those copies), or (b) primarily
|
||||
for and in connection with specific products or compilations that
|
||||
contain the covered work, unless you entered into that arrangement,
|
||||
or that patent license was granted, prior to 28 March 2007.
|
||||
|
||||
Nothing in this License shall be construed as excluding or limiting
|
||||
any implied license or other defenses to infringement that may
|
||||
otherwise be available to you under applicable patent law.
|
||||
|
||||
12. No Surrender of Others' Freedom.
|
||||
|
||||
If conditions are imposed on you (whether by court order, agreement or
|
||||
otherwise) that contradict the conditions of this License, they do not
|
||||
excuse you from the conditions of this License. If you cannot convey a
|
||||
covered work so as to satisfy simultaneously your obligations under this
|
||||
License and any other pertinent obligations, then as a consequence you may
|
||||
not convey it at all. For example, if you agree to terms that obligate you
|
||||
to collect a royalty for further conveying from those to whom you convey
|
||||
the Program, the only way you could satisfy both those terms and this
|
||||
License would be to refrain entirely from conveying the Program.
|
||||
|
||||
13. Remote Network Interaction; Use with the GNU General Public License.
|
||||
|
||||
Notwithstanding any other provision of this License, if you modify the
|
||||
Program, your modified version must prominently offer all users
|
||||
interacting with it remotely through a computer network (if your version
|
||||
supports such interaction) an opportunity to receive the Corresponding
|
||||
Source of your version by providing access to the Corresponding Source
|
||||
from a network server at no charge, through some standard or customary
|
||||
means of facilitating copying of software. This Corresponding Source
|
||||
shall include the Corresponding Source for any work covered by version 3
|
||||
of the GNU General Public License that is incorporated pursuant to the
|
||||
following paragraph.
|
||||
|
||||
Notwithstanding any other provision of this License, you have
|
||||
permission to link or combine any covered work with a work licensed
|
||||
under version 3 of the GNU General Public License into a single
|
||||
combined work, and to convey the resulting work. The terms of this
|
||||
License will continue to apply to the part which is the covered work,
|
||||
but the work with which it is combined will remain governed by version
|
||||
3 of the GNU General Public License.
|
||||
|
||||
14. Revised Versions of this License.
|
||||
|
||||
The Free Software Foundation may publish revised and/or new versions of
|
||||
the GNU Affero General Public License from time to time. Such new versions
|
||||
will be similar in spirit to the present version, but may differ in detail to
|
||||
address new problems or concerns.
|
||||
|
||||
Each version is given a distinguishing version number. If the
|
||||
Program specifies that a certain numbered version of the GNU Affero General
|
||||
Public License "or any later version" applies to it, you have the
|
||||
option of following the terms and conditions either of that numbered
|
||||
version or of any later version published by the Free Software
|
||||
Foundation. If the Program does not specify a version number of the
|
||||
GNU Affero General Public License, you may choose any version ever published
|
||||
by the Free Software Foundation.
|
||||
|
||||
If the Program specifies that a proxy can decide which future
|
||||
versions of the GNU Affero General Public License can be used, that proxy's
|
||||
public statement of acceptance of a version permanently authorizes you
|
||||
to choose that version for the Program.
|
||||
|
||||
Later license versions may give you additional or different
|
||||
permissions. However, no additional obligations are imposed on any
|
||||
author or copyright holder as a result of your choosing to follow a
|
||||
later version.
|
||||
|
||||
15. Disclaimer of Warranty.
|
||||
|
||||
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
||||
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
||||
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
||||
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
||||
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
||||
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
||||
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
||||
|
||||
16. Limitation of Liability.
|
||||
|
||||
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
||||
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
||||
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
||||
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
||||
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
||||
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
||||
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
||||
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
||||
SUCH DAMAGES.
|
||||
|
||||
17. Interpretation of Sections 15 and 16.
|
||||
|
||||
If the disclaimer of warranty and limitation of liability provided
|
||||
above cannot be given local legal effect according to their terms,
|
||||
reviewing courts shall apply local law that most closely approximates
|
||||
an absolute waiver of all civil liability in connection with the
|
||||
Program, unless a warranty or assumption of liability accompanies a
|
||||
copy of the Program in return for a fee.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
How to Apply These Terms to Your New Programs
|
||||
|
||||
If you develop a new program, and you want it to be of the greatest
|
||||
possible use to the public, the best way to achieve this is to make it
|
||||
free software which everyone can redistribute and change under these terms.
|
||||
|
||||
To do so, attach the following notices to the program. It is safest
|
||||
to attach them to the start of each source file to most effectively
|
||||
state the exclusion of warranty; and each file should have at least
|
||||
the "copyright" line and a pointer to where the full notice is found.
|
||||
|
||||
<one line to give the program's name and a brief idea of what it does.>
|
||||
Copyright (C) <year> <name of author>
|
||||
|
||||
This program is free software: you can redistribute it and/or modify
|
||||
it under the terms of the GNU Affero General Public License as published by
|
||||
the Free Software Foundation, either version 3 of the License, or
|
||||
(at your option) any later version.
|
||||
|
||||
This program is distributed in the hope that it will be useful,
|
||||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
GNU Affero General Public License for more details.
|
||||
|
||||
You should have received a copy of the GNU Affero General Public License
|
||||
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
|
||||
Also add information on how to contact you by electronic and paper mail.
|
||||
|
||||
If your software can interact with users remotely through a computer
|
||||
network, you should also make sure that it provides a way for users to
|
||||
get its source. For example, if your program is a web application, its
|
||||
interface could display a "Source" link that leads users to an archive
|
||||
of the code. There are many ways you could offer source, and different
|
||||
solutions will be better for different programs; see section 13 for the
|
||||
specific requirements.
|
||||
|
||||
You should also get your employer (if you work as a programmer) or school,
|
||||
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
||||
For more information on this, and how to apply and follow the GNU AGPL, see
|
||||
<https://www.gnu.org/licenses/>.
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
.PHONY: help lint generate breaking format check clean deps install install-buf install-plugins install-npm install-playwright
|
||||
.DEFAULT_GOAL := help
|
||||
|
||||
# Variables
|
||||
PROTO_DIR := proto
|
||||
GEN_CLIENT_DIR := src/generated/client
|
||||
GEN_SERVER_DIR := src/generated/server
|
||||
DOCS_API_DIR := docs/api
|
||||
|
||||
# Go install settings
|
||||
GO_PROXY := GOPROXY=direct
|
||||
GO_PRIVATE := GOPRIVATE=github.com/SebastienMelki
|
||||
GO_INSTALL := $(GO_PROXY) $(GO_PRIVATE) go install
|
||||
|
||||
# Required tool versions
|
||||
BUF_VERSION := v1.64.0
|
||||
SEBUF_VERSION := v0.7.0
|
||||
|
||||
help: ## Show this help message
|
||||
@echo 'Usage: make [target]'
|
||||
@echo ''
|
||||
@echo 'Targets:'
|
||||
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {printf " %-20s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
|
||||
|
||||
install: install-buf install-plugins install-npm install-playwright deps ## Install everything (buf, sebuf plugins, npm deps, proto deps, browsers)
|
||||
|
||||
install-buf: ## Install buf CLI
|
||||
@if command -v buf >/dev/null 2>&1; then \
|
||||
echo "buf already installed: $$(buf --version)"; \
|
||||
else \
|
||||
echo "Installing buf..."; \
|
||||
$(GO_INSTALL) github.com/bufbuild/buf/cmd/buf@$(BUF_VERSION); \
|
||||
echo "buf installed!"; \
|
||||
fi
|
||||
|
||||
install-plugins: ## Install sebuf protoc plugins (requires Go)
|
||||
@echo "Installing sebuf protoc plugins $(SEBUF_VERSION)..."
|
||||
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-ts-client@$(SEBUF_VERSION)
|
||||
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-ts-server@$(SEBUF_VERSION)
|
||||
@$(GO_INSTALL) github.com/SebastienMelki/sebuf/cmd/protoc-gen-openapiv3@$(SEBUF_VERSION)
|
||||
@echo "Plugins installed!"
|
||||
|
||||
install-npm: ## Install npm dependencies
|
||||
npm install
|
||||
|
||||
install-playwright: ## Install Playwright browsers for e2e tests
|
||||
npx playwright install chromium
|
||||
|
||||
deps: ## Install/update buf proto dependencies
|
||||
cd $(PROTO_DIR) && buf dep update
|
||||
|
||||
lint: ## Lint protobuf files
|
||||
cd $(PROTO_DIR) && buf lint
|
||||
|
||||
generate: clean ## Generate code from proto definitions
|
||||
@mkdir -p $(GEN_CLIENT_DIR) $(GEN_SERVER_DIR) $(DOCS_API_DIR)
|
||||
cd $(PROTO_DIR) && buf generate
|
||||
@echo "Code generation complete!"
|
||||
|
||||
breaking: ## Check for breaking changes against main
|
||||
cd $(PROTO_DIR) && buf breaking --against '.git#branch=main,subdir=proto'
|
||||
|
||||
format: ## Format protobuf files
|
||||
cd $(PROTO_DIR) && buf format -w
|
||||
|
||||
check: lint generate ## Run all checks (lint + generate)
|
||||
|
||||
clean: ## Clean generated files
|
||||
@rm -rf $(GEN_CLIENT_DIR)
|
||||
@rm -rf $(GEN_SERVER_DIR)
|
||||
@rm -rf $(DOCS_API_DIR)
|
||||
@echo "Clean complete!"
|
||||
+112
@@ -0,0 +1,112 @@
|
||||
# Security Policy
|
||||
|
||||
## Supported Versions
|
||||
|
||||
| Version | Supported |
|
||||
| ------- | ------------------ |
|
||||
| main | :white_check_mark: |
|
||||
|
||||
Only the latest version on the `main` branch is actively maintained and receives security updates.
|
||||
|
||||
## Reporting a Vulnerability
|
||||
|
||||
**Please do NOT report security vulnerabilities through public GitHub issues.**
|
||||
|
||||
If you discover a security vulnerability in World Monitor, please report it responsibly:
|
||||
|
||||
1. **GitHub Private Vulnerability Reporting**: Use [GitHub's private vulnerability reporting](https://github.com/koala73/worldmonitor/security/advisories/new) to submit your report directly through the repository.
|
||||
|
||||
2. **Direct Contact**: Alternatively, reach out to the repository owner [@koala73](https://github.com/koala73) directly through GitHub.
|
||||
|
||||
### What to Include
|
||||
|
||||
- A description of the vulnerability and its potential impact
|
||||
- Steps to reproduce the issue
|
||||
- Affected components (edge functions, client-side code, data layers, etc.)
|
||||
- Any potential fixes or mitigations you've identified
|
||||
|
||||
### Response Timeline
|
||||
|
||||
- **Acknowledgment**: Within 48 hours of your report
|
||||
- **Initial Assessment**: Within 1 week
|
||||
- **Fix/Patch**: Depending on severity, critical issues will be prioritized
|
||||
|
||||
### What to Expect
|
||||
|
||||
- You will receive an acknowledgment of your report
|
||||
- We will work with you to understand and validate the issue
|
||||
- We will keep you informed of progress toward a fix
|
||||
- Credit will be given to reporters in the fix commit (unless you prefer anonymity)
|
||||
|
||||
## Security Considerations
|
||||
|
||||
World Monitor is a client-side intelligence dashboard that aggregates publicly available data. Here are the key security areas:
|
||||
|
||||
### API Keys & Secrets
|
||||
|
||||
- **Web deployment**: API keys are stored server-side in Vercel Edge Functions
|
||||
- **Desktop runtime**: API keys are stored in the OS keychain (macOS Keychain / Windows Credential Manager) via a consolidated vault entry, never on disk in plaintext
|
||||
- No API keys should ever be committed to the repository
|
||||
- Environment variables (`.env.local`) are gitignored
|
||||
- The RSS proxy uses domain allowlisting to prevent SSRF
|
||||
|
||||
### Edge Functions & Sebuf Handlers
|
||||
|
||||
- All 17 domain APIs are served through Sebuf (a Proto-first RPC framework) via Vercel Edge Functions
|
||||
- Edge functions and handlers should validate/sanitize all input
|
||||
- CORS headers are configured per-function
|
||||
- Rate limiting and circuit breakers protect against abuse
|
||||
|
||||
### Client-Side Security
|
||||
|
||||
- No sensitive data is stored in localStorage or sessionStorage
|
||||
- External content (RSS feeds, news) is sanitized before rendering
|
||||
- Map data layers use trusted, vetted data sources
|
||||
- Content Security Policy restricts script-src to `'self'` (no unsafe-inline/eval)
|
||||
|
||||
### Desktop Runtime Security (Tauri)
|
||||
|
||||
- **IPC origin validation**: Sensitive Tauri commands (secrets, cache, token) are gated to trusted windows only; external-origin windows (e.g., YouTube login) are blocked
|
||||
- **DevTools**: Disabled in production builds; gated behind an opt-in Cargo feature for development
|
||||
- **Sidecar authentication**: A per-session CSPRNG token (`LOCAL_API_TOKEN`) authenticates all renderer-to-sidecar requests, preventing other local processes from accessing the API
|
||||
- **Capability isolation**: The YouTube login window runs under a restricted capability with no access to secret or cache IPC commands
|
||||
- **Fetch patch trust boundary**: The global fetch interceptor injects the sidecar token with a 5-minute TTL; the renderer is the intended client — if renderer integrity is compromised, Tauri IPC provides strictly more access than the fetch patch
|
||||
|
||||
### Data Sources
|
||||
|
||||
- World Monitor aggregates publicly available OSINT data
|
||||
- No classified or restricted data sources are used
|
||||
- State-affiliated sources are flagged with propaganda risk ratings
|
||||
- All data is consumed read-only — the platform does not modify upstream sources
|
||||
|
||||
## Scope
|
||||
|
||||
The following are **in scope** for security reports:
|
||||
|
||||
- Vulnerabilities in the World Monitor codebase
|
||||
- Edge function security issues (SSRF, injection, auth bypass)
|
||||
- XSS or content injection through RSS feeds or external data
|
||||
- API key exposure or secret leakage
|
||||
- Tauri IPC command privilege escalation or capability bypass
|
||||
- Sidecar authentication bypass or token leakage
|
||||
- Dependency vulnerabilities with a viable attack vector
|
||||
|
||||
The following are **out of scope**:
|
||||
|
||||
- Vulnerabilities in third-party services we consume (report to the upstream provider)
|
||||
- Social engineering attacks
|
||||
- Denial of service attacks
|
||||
- Issues in forked copies of the repository
|
||||
- Security issues in user-provided environment configurations
|
||||
|
||||
## Best Practices for Contributors
|
||||
|
||||
- Never commit API keys, tokens, or secrets
|
||||
- Use environment variables for all sensitive configuration
|
||||
- Sanitize external input in edge functions
|
||||
- Keep dependencies updated — run `npm audit` regularly
|
||||
- Follow the principle of least privilege for API access
|
||||
|
||||
---
|
||||
|
||||
Thank you for helping keep World Monitor and its users safe! 🔒
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,150 @@
|
||||
/**
|
||||
* Vercel edge function for sebuf RPC routes.
|
||||
*
|
||||
* Matches /api/{domain}/v1/{rpc} via Vercel dynamic segment routing.
|
||||
* CORS headers are applied to every response (200, 204, 403, 404).
|
||||
*/
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { createRouter } from '../../../server/router';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../../../server/cors';
|
||||
// @ts-expect-error — JS module, no declaration file
|
||||
import { validateApiKey } from '../../_api-key.js';
|
||||
import { mapErrorToResponse } from '../../../server/error-mapper';
|
||||
import { createSeismologyServiceRoutes } from '../../../src/generated/server/worldmonitor/seismology/v1/service_server';
|
||||
import { seismologyHandler } from '../../../server/worldmonitor/seismology/v1/handler';
|
||||
import { createWildfireServiceRoutes } from '../../../src/generated/server/worldmonitor/wildfire/v1/service_server';
|
||||
import { wildfireHandler } from '../../../server/worldmonitor/wildfire/v1/handler';
|
||||
import { createClimateServiceRoutes } from '../../../src/generated/server/worldmonitor/climate/v1/service_server';
|
||||
import { climateHandler } from '../../../server/worldmonitor/climate/v1/handler';
|
||||
import { createPredictionServiceRoutes } from '../../../src/generated/server/worldmonitor/prediction/v1/service_server';
|
||||
import { predictionHandler } from '../../../server/worldmonitor/prediction/v1/handler';
|
||||
import { createDisplacementServiceRoutes } from '../../../src/generated/server/worldmonitor/displacement/v1/service_server';
|
||||
import { displacementHandler } from '../../../server/worldmonitor/displacement/v1/handler';
|
||||
import { createAviationServiceRoutes } from '../../../src/generated/server/worldmonitor/aviation/v1/service_server';
|
||||
import { aviationHandler } from '../../../server/worldmonitor/aviation/v1/handler';
|
||||
import { createResearchServiceRoutes } from '../../../src/generated/server/worldmonitor/research/v1/service_server';
|
||||
import { researchHandler } from '../../../server/worldmonitor/research/v1/handler';
|
||||
import { createUnrestServiceRoutes } from '../../../src/generated/server/worldmonitor/unrest/v1/service_server';
|
||||
import { unrestHandler } from '../../../server/worldmonitor/unrest/v1/handler';
|
||||
import { createConflictServiceRoutes } from '../../../src/generated/server/worldmonitor/conflict/v1/service_server';
|
||||
import { conflictHandler } from '../../../server/worldmonitor/conflict/v1/handler';
|
||||
import { createMaritimeServiceRoutes } from '../../../src/generated/server/worldmonitor/maritime/v1/service_server';
|
||||
import { maritimeHandler } from '../../../server/worldmonitor/maritime/v1/handler';
|
||||
import { createCyberServiceRoutes } from '../../../src/generated/server/worldmonitor/cyber/v1/service_server';
|
||||
import { cyberHandler } from '../../../server/worldmonitor/cyber/v1/handler';
|
||||
import { createEconomicServiceRoutes } from '../../../src/generated/server/worldmonitor/economic/v1/service_server';
|
||||
import { economicHandler } from '../../../server/worldmonitor/economic/v1/handler';
|
||||
import { createInfrastructureServiceRoutes } from '../../../src/generated/server/worldmonitor/infrastructure/v1/service_server';
|
||||
import { infrastructureHandler } from '../../../server/worldmonitor/infrastructure/v1/handler';
|
||||
import { createMarketServiceRoutes } from '../../../src/generated/server/worldmonitor/market/v1/service_server';
|
||||
import { marketHandler } from '../../../server/worldmonitor/market/v1/handler';
|
||||
import { createNewsServiceRoutes } from '../../../src/generated/server/worldmonitor/news/v1/service_server';
|
||||
import { newsHandler } from '../../../server/worldmonitor/news/v1/handler';
|
||||
import { createIntelligenceServiceRoutes } from '../../../src/generated/server/worldmonitor/intelligence/v1/service_server';
|
||||
import { intelligenceHandler } from '../../../server/worldmonitor/intelligence/v1/handler';
|
||||
import { createMilitaryServiceRoutes } from '../../../src/generated/server/worldmonitor/military/v1/service_server';
|
||||
import { militaryHandler } from '../../../server/worldmonitor/military/v1/handler';
|
||||
import { createPositiveEventsServiceRoutes } from '../../../src/generated/server/worldmonitor/positive_events/v1/service_server';
|
||||
import { positiveEventsHandler } from '../../../server/worldmonitor/positive-events/v1/handler';
|
||||
import { createGivingServiceRoutes } from '../../../src/generated/server/worldmonitor/giving/v1/service_server';
|
||||
import { givingHandler } from '../../../server/worldmonitor/giving/v1/handler';
|
||||
import { createTradeServiceRoutes } from '../../../src/generated/server/worldmonitor/trade/v1/service_server';
|
||||
import { tradeHandler } from '../../../server/worldmonitor/trade/v1/handler';
|
||||
import { createSupplyChainServiceRoutes } from '../../../src/generated/server/worldmonitor/supply_chain/v1/service_server';
|
||||
import { supplyChainHandler } from '../../../server/worldmonitor/supply-chain/v1/handler';
|
||||
|
||||
import type { ServerOptions } from '../../../src/generated/server/worldmonitor/seismology/v1/service_server';
|
||||
|
||||
const serverOptions: ServerOptions = { onError: mapErrorToResponse };
|
||||
|
||||
const allRoutes = [
|
||||
...createSeismologyServiceRoutes(seismologyHandler, serverOptions),
|
||||
...createWildfireServiceRoutes(wildfireHandler, serverOptions),
|
||||
...createClimateServiceRoutes(climateHandler, serverOptions),
|
||||
...createPredictionServiceRoutes(predictionHandler, serverOptions),
|
||||
...createDisplacementServiceRoutes(displacementHandler, serverOptions),
|
||||
...createAviationServiceRoutes(aviationHandler, serverOptions),
|
||||
...createResearchServiceRoutes(researchHandler, serverOptions),
|
||||
...createUnrestServiceRoutes(unrestHandler, serverOptions),
|
||||
...createConflictServiceRoutes(conflictHandler, serverOptions),
|
||||
...createMaritimeServiceRoutes(maritimeHandler, serverOptions),
|
||||
...createCyberServiceRoutes(cyberHandler, serverOptions),
|
||||
...createEconomicServiceRoutes(economicHandler, serverOptions),
|
||||
...createInfrastructureServiceRoutes(infrastructureHandler, serverOptions),
|
||||
...createMarketServiceRoutes(marketHandler, serverOptions),
|
||||
...createNewsServiceRoutes(newsHandler, serverOptions),
|
||||
...createIntelligenceServiceRoutes(intelligenceHandler, serverOptions),
|
||||
...createMilitaryServiceRoutes(militaryHandler, serverOptions),
|
||||
...createPositiveEventsServiceRoutes(positiveEventsHandler, serverOptions),
|
||||
...createGivingServiceRoutes(givingHandler, serverOptions),
|
||||
...createTradeServiceRoutes(tradeHandler, serverOptions),
|
||||
...createSupplyChainServiceRoutes(supplyChainHandler, serverOptions),
|
||||
];
|
||||
|
||||
const router = createRouter(allRoutes);
|
||||
|
||||
export default async function handler(request: Request): Promise<Response> {
|
||||
// Origin check first — skip CORS headers for disallowed origins (M-2 fix)
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
let corsHeaders: Record<string, string>;
|
||||
try {
|
||||
corsHeaders = getCorsHeaders(request);
|
||||
} catch {
|
||||
corsHeaders = { 'Access-Control-Allow-Origin': '*' };
|
||||
}
|
||||
|
||||
// OPTIONS preflight
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
// API key validation (origin-aware)
|
||||
const keyCheck = validateApiKey(request);
|
||||
if (keyCheck.required && !keyCheck.valid) {
|
||||
return new Response(JSON.stringify({ error: keyCheck.error }), {
|
||||
status: 401,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
// Route matching
|
||||
const matchedHandler = router.match(request);
|
||||
if (!matchedHandler) {
|
||||
return new Response(JSON.stringify({ error: 'Not found' }), {
|
||||
status: 404,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
// Execute handler with top-level error boundary (H-1 fix)
|
||||
let response: Response;
|
||||
try {
|
||||
response = await matchedHandler(request);
|
||||
} catch (err) {
|
||||
console.error('[gateway] Unhandled handler error:', err);
|
||||
response = new Response(JSON.stringify({ message: 'Internal server error' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Merge CORS headers into response
|
||||
const mergedHeaders = new Headers(response.headers);
|
||||
for (const [key, value] of Object.entries(corsHeaders)) {
|
||||
mergedHeaders.set(key, value);
|
||||
}
|
||||
|
||||
return new Response(response.body, {
|
||||
status: response.status,
|
||||
statusText: response.statusText,
|
||||
headers: mergedHeaders,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
const DESKTOP_ORIGIN_PATTERNS = [
|
||||
/^https?:\/\/tauri\.localhost(:\d+)?$/,
|
||||
/^https?:\/\/[a-z0-9-]+\.tauri\.localhost(:\d+)?$/i,
|
||||
/^tauri:\/\/localhost$/,
|
||||
/^asset:\/\/localhost$/,
|
||||
];
|
||||
|
||||
function isDesktopOrigin(origin) {
|
||||
return Boolean(origin) && DESKTOP_ORIGIN_PATTERNS.some(p => p.test(origin));
|
||||
}
|
||||
|
||||
export function validateApiKey(req) {
|
||||
const key = req.headers.get('X-WorldMonitor-Key');
|
||||
const origin = req.headers.get('Origin') || '';
|
||||
|
||||
if (isDesktopOrigin(origin)) {
|
||||
if (!key) return { valid: false, required: true, error: 'API key required for desktop access' };
|
||||
const validKeys = (process.env.WORLDMONITOR_VALID_KEYS || '').split(',').filter(Boolean);
|
||||
if (!validKeys.includes(key)) return { valid: false, required: true, error: 'Invalid API key' };
|
||||
return { valid: true, required: true };
|
||||
}
|
||||
|
||||
if (key) {
|
||||
const validKeys = (process.env.WORLDMONITOR_VALID_KEYS || '').split(',').filter(Boolean);
|
||||
if (!validKeys.includes(key)) return { valid: false, required: true, error: 'Invalid API key' };
|
||||
return { valid: true, required: true };
|
||||
}
|
||||
|
||||
return { valid: false, required: false };
|
||||
}
|
||||
@@ -1,53 +0,0 @@
|
||||
const statsByEndpoint = new Map();
|
||||
const MAX_ENDPOINTS = 128;
|
||||
const LOG_EVERY = Math.max(0, Number(process.env.CACHE_TELEMETRY_LOG_EVERY || 200));
|
||||
|
||||
function cleanupOldEndpoints() {
|
||||
if (statsByEndpoint.size <= MAX_ENDPOINTS) return;
|
||||
const entries = Array.from(statsByEndpoint.entries())
|
||||
.sort((a, b) => a[1].lastSeen - b[1].lastSeen);
|
||||
const overflow = statsByEndpoint.size - MAX_ENDPOINTS;
|
||||
for (let i = 0; i < overflow; i++) {
|
||||
statsByEndpoint.delete(entries[i][0]);
|
||||
}
|
||||
}
|
||||
|
||||
export function recordCacheTelemetry(endpoint, outcome) {
|
||||
if (!endpoint || !outcome) return;
|
||||
const now = Date.now();
|
||||
const current = statsByEndpoint.get(endpoint) || {
|
||||
total: 0,
|
||||
outcomes: {},
|
||||
firstSeen: now,
|
||||
lastSeen: now,
|
||||
};
|
||||
|
||||
current.total += 1;
|
||||
current.outcomes[outcome] = (current.outcomes[outcome] || 0) + 1;
|
||||
current.lastSeen = now;
|
||||
statsByEndpoint.set(endpoint, current);
|
||||
cleanupOldEndpoints();
|
||||
|
||||
if (LOG_EVERY > 0 && current.total % LOG_EVERY === 0) {
|
||||
console.log(`[CacheTelemetry] ${endpoint} total=${current.total} outcomes=${JSON.stringify(current.outcomes)}`);
|
||||
}
|
||||
}
|
||||
|
||||
export function getCacheTelemetrySnapshot() {
|
||||
const endpoints = Array.from(statsByEndpoint.entries())
|
||||
.sort((a, b) => b[1].lastSeen - a[1].lastSeen)
|
||||
.map(([endpoint, stats]) => ({
|
||||
endpoint,
|
||||
total: stats.total,
|
||||
outcomes: stats.outcomes,
|
||||
firstSeen: new Date(stats.firstSeen).toISOString(),
|
||||
lastSeen: new Date(stats.lastSeen).toISOString(),
|
||||
}));
|
||||
|
||||
return {
|
||||
generatedAt: new Date().toISOString(),
|
||||
endpointCount: endpoints.length,
|
||||
endpoints,
|
||||
note: 'In-memory per instance telemetry (resets on cold start).',
|
||||
};
|
||||
}
|
||||
+2
-2
@@ -1,6 +1,6 @@
|
||||
const ALLOWED_ORIGIN_PATTERNS = [
|
||||
/^https:\/\/(.*\.)?worldmonitor\.app$/,
|
||||
/^https:\/\/worldmonitor-[a-z0-9-]+-elie-habib-projects\.vercel\.app$/,
|
||||
/^https:\/\/worldmonitor-[a-z0-9-]+-elie-[a-z0-9]+\.vercel\.app$/,
|
||||
/^https?:\/\/localhost(:\d+)?$/,
|
||||
/^https?:\/\/127\.0\.0\.1(:\d+)?$/,
|
||||
/^https?:\/\/tauri\.localhost(:\d+)?$/,
|
||||
@@ -19,7 +19,7 @@ export function getCorsHeaders(req, methods = 'GET, OPTIONS') {
|
||||
return {
|
||||
'Access-Control-Allow-Origin': allowOrigin,
|
||||
'Access-Control-Allow-Methods': methods,
|
||||
'Access-Control-Allow-Headers': 'Content-Type',
|
||||
'Access-Control-Allow-Headers': 'Content-Type, Authorization, X-WorldMonitor-Key',
|
||||
'Access-Control-Max-Age': '86400',
|
||||
'Vary': 'Origin',
|
||||
};
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
export function createIpRateLimiter({
|
||||
limit,
|
||||
windowMs,
|
||||
maxEntries = 5000,
|
||||
cleanupIntervalMs = 30 * 1000,
|
||||
}) {
|
||||
const records = new Map();
|
||||
let lastCleanupAt = 0;
|
||||
|
||||
function cleanup(now) {
|
||||
if (now - lastCleanupAt < cleanupIntervalMs && records.size <= maxEntries) {
|
||||
return;
|
||||
}
|
||||
lastCleanupAt = now;
|
||||
|
||||
const cutoff = now - windowMs;
|
||||
for (const [ip, record] of records) {
|
||||
if (record.windowStart < cutoff) {
|
||||
records.delete(ip);
|
||||
}
|
||||
}
|
||||
|
||||
if (records.size <= maxEntries) {
|
||||
return;
|
||||
}
|
||||
|
||||
const overflow = records.size - maxEntries;
|
||||
const oldest = Array.from(records.entries())
|
||||
.sort((a, b) => a[1].windowStart - b[1].windowStart);
|
||||
for (let i = 0; i < overflow; i++) {
|
||||
const entry = oldest[i];
|
||||
if (!entry) break;
|
||||
records.delete(entry[0]);
|
||||
}
|
||||
}
|
||||
|
||||
function check(ip) {
|
||||
const now = Date.now();
|
||||
cleanup(now);
|
||||
|
||||
const key = (ip || 'unknown').trim() || 'unknown';
|
||||
const record = records.get(key);
|
||||
|
||||
if (!record || now - record.windowStart > windowMs) {
|
||||
records.set(key, { count: 1, windowStart: now });
|
||||
return true;
|
||||
}
|
||||
|
||||
if (record.count >= limit) {
|
||||
return false;
|
||||
}
|
||||
|
||||
record.count += 1;
|
||||
return true;
|
||||
}
|
||||
|
||||
return {
|
||||
check,
|
||||
size: () => records.size,
|
||||
};
|
||||
}
|
||||
@@ -1,185 +0,0 @@
|
||||
const isSidecar = (process.env.LOCAL_API_MODE || '').includes('sidecar');
|
||||
|
||||
// ── In-memory cache (desktop/sidecar) ──
|
||||
const mem = new Map();
|
||||
let persistPath = null;
|
||||
let persistTimer = null;
|
||||
let persistInFlight = false;
|
||||
let persistQueued = false;
|
||||
let loaded = false;
|
||||
const MAX_PERSIST_ENTRIES = Math.max(100, Number(process.env.LOCAL_API_CACHE_PERSIST_MAX || 5000));
|
||||
|
||||
async function ensureDesktopCache() {
|
||||
if (loaded) return;
|
||||
loaded = true;
|
||||
try {
|
||||
const { join } = await import('node:path');
|
||||
const { readFileSync } = await import('node:fs');
|
||||
const dir = process.env.LOCAL_API_RESOURCE_DIR || '.';
|
||||
persistPath = join(dir, 'api-cache.json');
|
||||
const data = JSON.parse(readFileSync(persistPath, 'utf8'));
|
||||
const now = Date.now();
|
||||
for (const [k, entry] of Object.entries(data)) {
|
||||
if (entry.expiresAt > now) mem.set(k, entry);
|
||||
}
|
||||
console.log(`[Cache] Loaded ${mem.size} entries from disk`);
|
||||
} catch {
|
||||
// File doesn't exist yet
|
||||
}
|
||||
setInterval(() => {
|
||||
const now = Date.now();
|
||||
for (const [k, v] of mem) {
|
||||
if (v.expiresAt <= now) mem.delete(k);
|
||||
}
|
||||
}, 60_000).unref?.();
|
||||
}
|
||||
|
||||
function buildPersistSnapshot() {
|
||||
const now = Date.now();
|
||||
const payload = Object.create(null);
|
||||
let kept = 0;
|
||||
|
||||
for (const [key, entry] of mem) {
|
||||
if (!entry || entry.expiresAt <= now) continue;
|
||||
payload[key] = entry;
|
||||
kept += 1;
|
||||
if (kept >= MAX_PERSIST_ENTRIES) break;
|
||||
}
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
async function persistToDisk() {
|
||||
if (!persistPath) return;
|
||||
if (persistInFlight) {
|
||||
persistQueued = true;
|
||||
return;
|
||||
}
|
||||
|
||||
persistInFlight = true;
|
||||
try {
|
||||
const snapshot = buildPersistSnapshot();
|
||||
const json = JSON.stringify(snapshot);
|
||||
const { writeFile, rename } = await import('node:fs/promises');
|
||||
const tmp = persistPath + '.tmp';
|
||||
await writeFile(tmp, json, 'utf8');
|
||||
await rename(tmp, persistPath);
|
||||
} catch (err) {
|
||||
console.warn('[Cache] Persist error:', err.message);
|
||||
} finally {
|
||||
persistInFlight = false;
|
||||
if (persistQueued) {
|
||||
persistQueued = false;
|
||||
void persistToDisk();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function debouncedPersist() {
|
||||
if (!persistPath) return;
|
||||
clearTimeout(persistTimer);
|
||||
persistTimer = setTimeout(() => {
|
||||
void persistToDisk();
|
||||
}, 2000);
|
||||
if (persistTimer?.unref) persistTimer.unref();
|
||||
}
|
||||
|
||||
// ── Redis (cloud/Vercel) ──
|
||||
let RedisClass = null;
|
||||
let redis = null;
|
||||
let redisInitFailed = false;
|
||||
|
||||
export async function getRedis() {
|
||||
if (isSidecar) return null;
|
||||
if (redis) return redis;
|
||||
if (redisInitFailed) return null;
|
||||
|
||||
const url = process.env.UPSTASH_REDIS_REST_URL;
|
||||
const token = process.env.UPSTASH_REDIS_REST_TOKEN;
|
||||
if (!url || !token) return null;
|
||||
|
||||
try {
|
||||
if (!RedisClass) {
|
||||
const mod = await import('@upstash/redis');
|
||||
RedisClass = mod.Redis;
|
||||
}
|
||||
redis = new RedisClass({ url, token });
|
||||
return redis;
|
||||
} catch (err) {
|
||||
redisInitFailed = true;
|
||||
console.warn('[Cache] Redis init failed:', err.message);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// ── Shared API ──
|
||||
|
||||
export async function getCachedJson(key) {
|
||||
if (isSidecar) {
|
||||
await ensureDesktopCache();
|
||||
const entry = mem.get(key);
|
||||
if (!entry) return null;
|
||||
if (entry.expiresAt <= Date.now()) {
|
||||
mem.delete(key);
|
||||
return null;
|
||||
}
|
||||
return entry.value;
|
||||
}
|
||||
|
||||
const r = await getRedis();
|
||||
if (!r) return null;
|
||||
try {
|
||||
return await r.get(key);
|
||||
} catch (err) {
|
||||
console.warn('[Cache] Read failed:', err.message);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function setCachedJson(key, value, ttlSeconds) {
|
||||
if (isSidecar) {
|
||||
await ensureDesktopCache();
|
||||
mem.set(key, { value, expiresAt: Date.now() + ttlSeconds * 1000 });
|
||||
debouncedPersist();
|
||||
return true;
|
||||
}
|
||||
|
||||
const r = await getRedis();
|
||||
if (!r) return false;
|
||||
try {
|
||||
await r.set(key, value, { ex: ttlSeconds });
|
||||
return true;
|
||||
} catch (err) {
|
||||
console.warn('[Cache] Write failed:', err.message);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export async function mget(...keys) {
|
||||
if (isSidecar) {
|
||||
await ensureDesktopCache();
|
||||
const now = Date.now();
|
||||
return keys.map(k => {
|
||||
const entry = mem.get(k);
|
||||
if (!entry || entry.expiresAt <= now) return null;
|
||||
return entry.value;
|
||||
});
|
||||
}
|
||||
|
||||
const r = await getRedis();
|
||||
if (!r) return keys.map(() => null);
|
||||
try {
|
||||
return await r.mget(...keys);
|
||||
} catch (err) {
|
||||
console.warn('[Cache] mget failed:', err.message);
|
||||
return keys.map(() => null);
|
||||
}
|
||||
}
|
||||
|
||||
export function hashString(input) {
|
||||
let hash = 5381;
|
||||
for (let i = 0; i < input.length; i++) {
|
||||
hash = ((hash << 5) + hash) + input.charCodeAt(i);
|
||||
}
|
||||
return (hash >>> 0).toString(36);
|
||||
}
|
||||
@@ -1,188 +0,0 @@
|
||||
// ACLED Conflict Events API proxy - battles, explosions, violence against civilians
|
||||
// Separate from protest proxy to avoid mixing data flows
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'acled:conflict:v2';
|
||||
const CACHE_TTL_SECONDS = 10 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const RATE_LIMIT = 10;
|
||||
const RATE_WINDOW_MS = 60 * 1000;
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: RATE_LIMIT,
|
||||
windowMs: RATE_WINDOW_MS,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed', data: [] }, {
|
||||
status: 405,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed', data: [] }, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited', data: [] }, {
|
||||
status: 429,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Retry-After': '60',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const token = process.env.ACLED_ACCESS_TOKEN;
|
||||
if (!token) {
|
||||
return Response.json({ error: 'ACLED not configured', data: [], configured: false }, {
|
||||
status: 200,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (cached && typeof cached === 'object' && Array.isArray(cached.data)) {
|
||||
recordCacheTelemetry('/api/acled-conflict', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (fallbackCache.data && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/acled-conflict', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const endDate = new Date().toISOString().split('T')[0];
|
||||
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
|
||||
const params = new URLSearchParams({
|
||||
event_type: 'Battles|Explosions/Remote violence|Violence against civilians',
|
||||
event_date: `${startDate}|${endDate}`,
|
||||
event_date_where: 'BETWEEN',
|
||||
limit: '500',
|
||||
_format: 'json',
|
||||
});
|
||||
|
||||
const response = await fetch(`https://acleddata.com/api/acled/read?${params}`, {
|
||||
headers: {
|
||||
Accept: 'application/json',
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const text = await response.text();
|
||||
return Response.json({ error: `ACLED API error: ${response.status}`, details: text.substring(0, 200), data: [] }, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const rawData = await response.json();
|
||||
const events = Array.isArray(rawData?.data) ? rawData.data : [];
|
||||
const sanitizedEvents = events.map((e) => ({
|
||||
event_id_cnty: e.event_id_cnty,
|
||||
event_date: e.event_date,
|
||||
event_type: e.event_type,
|
||||
sub_event_type: e.sub_event_type,
|
||||
actor1: e.actor1,
|
||||
actor2: e.actor2,
|
||||
country: e.country,
|
||||
admin1: e.admin1,
|
||||
location: e.location,
|
||||
latitude: e.latitude,
|
||||
longitude: e.longitude,
|
||||
fatalities: e.fatalities,
|
||||
notes: typeof e.notes === 'string' ? e.notes.substring(0, 500) : undefined,
|
||||
source: e.source,
|
||||
tags: e.tags,
|
||||
}));
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: sanitizedEvents.length,
|
||||
data: sanitizedEvents,
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/acled-conflict', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'MISS',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (fallbackCache.data) {
|
||||
recordCacheTelemetry('/api/acled-conflict', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
'X-Cache': 'STALE',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/acled-conflict', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, data: [] }, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
-200
@@ -1,200 +0,0 @@
|
||||
// ACLED API proxy - keeps token server-side only
|
||||
// Token is stored in ACLED_ACCESS_TOKEN (no VITE_ prefix)
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'acled:protests:v2';
|
||||
const CACHE_TTL_SECONDS = 10 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
|
||||
// In-memory fallback cache when Redis is unavailable.
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const RATE_LIMIT = 10; // requests per minute
|
||||
const RATE_WINDOW_MS = 60 * 1000;
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: RATE_LIMIT,
|
||||
windowMs: RATE_WINDOW_MS,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed', data: [] }, {
|
||||
status: 405,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed', data: [] }, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited', data: [] }, {
|
||||
status: 429,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Retry-After': '60',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const token = process.env.ACLED_ACCESS_TOKEN;
|
||||
if (!token) {
|
||||
return Response.json({
|
||||
error: 'ACLED not configured',
|
||||
data: [],
|
||||
configured: false,
|
||||
}, {
|
||||
status: 200,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (cached && typeof cached === 'object' && Array.isArray(cached.data)) {
|
||||
recordCacheTelemetry('/api/acled', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (fallbackCache.data && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/acled', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const endDate = new Date().toISOString().split('T')[0];
|
||||
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
|
||||
const params = new URLSearchParams({
|
||||
event_type: 'Protests',
|
||||
event_date: `${startDate}|${endDate}`,
|
||||
event_date_where: 'BETWEEN',
|
||||
limit: '500',
|
||||
_format: 'json',
|
||||
});
|
||||
|
||||
const response = await fetch(`https://acleddata.com/api/acled/read?${params}`, {
|
||||
headers: {
|
||||
Accept: 'application/json',
|
||||
Authorization: `Bearer ${token}`,
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const text = await response.text();
|
||||
return Response.json({
|
||||
error: `ACLED API error: ${response.status}`,
|
||||
details: text.substring(0, 200),
|
||||
data: [],
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const rawData = await response.json();
|
||||
const events = Array.isArray(rawData?.data) ? rawData.data : [];
|
||||
const sanitizedEvents = events.map((e) => ({
|
||||
event_id_cnty: e.event_id_cnty,
|
||||
event_date: e.event_date,
|
||||
event_type: e.event_type,
|
||||
sub_event_type: e.sub_event_type,
|
||||
actor1: e.actor1,
|
||||
actor2: e.actor2,
|
||||
country: e.country,
|
||||
admin1: e.admin1,
|
||||
location: e.location,
|
||||
latitude: e.latitude,
|
||||
longitude: e.longitude,
|
||||
fatalities: e.fatalities,
|
||||
notes: typeof e.notes === 'string' ? e.notes.substring(0, 500) : undefined,
|
||||
source: e.source,
|
||||
tags: e.tags,
|
||||
}));
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: sanitizedEvents.length,
|
||||
data: sanitizedEvents,
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/acled', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'MISS',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (fallbackCache.data) {
|
||||
recordCacheTelemetry('/api/acled', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
'X-Cache': 'STALE',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/acled', 'ERROR');
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${toErrorMessage(error)}`,
|
||||
data: [],
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
+49
-167
@@ -1,98 +1,37 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
|
||||
const CACHE_TTL_SECONDS = 8;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const CACHE_VERSION = 'v1';
|
||||
const MEMORY_CACHE_MAX_ENTRIES = 8;
|
||||
const MEMORY_FALLBACK_MAX_AGE_MS = 60 * 1000;
|
||||
const memoryCache = new Map();
|
||||
const inFlightByKey = new Map();
|
||||
|
||||
function getErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'Failed to fetch AIS snapshot');
|
||||
}
|
||||
|
||||
function getMemoryCachedSnapshot(cacheKey, allowStale = false) {
|
||||
const entry = memoryCache.get(cacheKey);
|
||||
if (!entry) return null;
|
||||
|
||||
const now = Date.now();
|
||||
const age = now - entry.timestamp;
|
||||
if (age > MEMORY_FALLBACK_MAX_AGE_MS) {
|
||||
memoryCache.delete(cacheKey);
|
||||
return null;
|
||||
}
|
||||
|
||||
if (!allowStale && age > CACHE_TTL_MS) {
|
||||
return null;
|
||||
}
|
||||
|
||||
entry.lastSeen = now;
|
||||
return entry.data;
|
||||
}
|
||||
|
||||
function setMemoryCachedSnapshot(cacheKey, data) {
|
||||
const now = Date.now();
|
||||
memoryCache.set(cacheKey, {
|
||||
data,
|
||||
timestamp: now,
|
||||
lastSeen: now,
|
||||
});
|
||||
|
||||
if (memoryCache.size <= MEMORY_CACHE_MAX_ENTRIES) return;
|
||||
|
||||
const overflow = memoryCache.size - MEMORY_CACHE_MAX_ENTRIES;
|
||||
const oldestEntries = Array.from(memoryCache.entries())
|
||||
.sort((a, b) => a[1].lastSeen - b[1].lastSeen);
|
||||
for (let i = 0; i < overflow; i++) {
|
||||
const entry = oldestEntries[i];
|
||||
if (!entry) break;
|
||||
memoryCache.delete(entry[0]);
|
||||
}
|
||||
}
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
function getRelayBaseUrl() {
|
||||
const relayUrl = process.env.WS_RELAY_URL;
|
||||
if (!relayUrl) return null;
|
||||
return relayUrl
|
||||
.replace('wss://', 'https://')
|
||||
.replace('ws://', 'http://')
|
||||
.replace(/\/$/, '');
|
||||
return relayUrl.replace('wss://', 'https://').replace('ws://', 'http://').replace(/\/$/, '');
|
||||
}
|
||||
|
||||
function isValidSnapshot(data) {
|
||||
return Boolean(
|
||||
data &&
|
||||
typeof data === 'object' &&
|
||||
data.status &&
|
||||
typeof data.status === 'object' &&
|
||||
Array.isArray(data.disruptions) &&
|
||||
Array.isArray(data.density)
|
||||
);
|
||||
function getRelayHeaders(baseHeaders = {}) {
|
||||
const headers = { ...baseHeaders };
|
||||
const relaySecret = process.env.RELAY_SHARED_SECRET || '';
|
||||
if (relaySecret) {
|
||||
const relayHeader = (process.env.RELAY_AUTH_HEADER || 'x-relay-key').toLowerCase();
|
||||
headers[relayHeader] = relaySecret;
|
||||
headers.Authorization = `Bearer ${relaySecret}`;
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
async function fetchWithTimeout(url, options, timeoutMs = 15000) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
@@ -100,107 +39,50 @@ export default async function handler(req) {
|
||||
});
|
||||
}
|
||||
|
||||
const requestUrl = new URL(req.url);
|
||||
const includeCandidates = requestUrl.searchParams.get('candidates') === 'true';
|
||||
const cacheKey = `ais-snapshot:${CACHE_VERSION}:${includeCandidates ? 'full' : 'lite'}`;
|
||||
const redisCached = await getCachedJson(cacheKey);
|
||||
if (isValidSnapshot(redisCached)) {
|
||||
setMemoryCachedSnapshot(cacheKey, redisCached);
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'REDIS-HIT');
|
||||
return new Response(JSON.stringify(redisCached), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const memoryCached = getMemoryCachedSnapshot(cacheKey);
|
||||
if (isValidSnapshot(memoryCached)) {
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'MEMORY-HIT');
|
||||
return new Response(JSON.stringify(memoryCached), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
...corsHeaders,
|
||||
},
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const relayBaseUrl = getRelayBaseUrl();
|
||||
if (!relayBaseUrl) {
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'NO-RELAY-CONFIG');
|
||||
return new Response(JSON.stringify({ vessels: [], skipped: true, reason: 'AIS relay not configured' }), {
|
||||
status: 200,
|
||||
return new Response(JSON.stringify({ error: 'WS_RELAY_URL is not configured' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
let requestPromise = inFlightByKey.get(cacheKey);
|
||||
if (!requestPromise) {
|
||||
requestPromise = (async () => {
|
||||
const upstreamUrl = `${relayBaseUrl}/ais/snapshot?candidates=${includeCandidates ? 'true' : 'false'}`;
|
||||
const response = await fetch(upstreamUrl, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`AIS relay HTTP ${response.status}`);
|
||||
}
|
||||
const requestUrl = new URL(req.url);
|
||||
const relayUrl = `${relayBaseUrl}/ais/snapshot${requestUrl.search || ''}`;
|
||||
const response = await fetchWithTimeout(relayUrl, {
|
||||
headers: getRelayHeaders({ Accept: 'application/json' }),
|
||||
}, 12000);
|
||||
|
||||
const data = await response.json();
|
||||
if (!isValidSnapshot(data)) {
|
||||
throw new Error('Invalid AIS snapshot payload');
|
||||
}
|
||||
return data;
|
||||
})();
|
||||
inFlightByKey.set(cacheKey, requestPromise);
|
||||
}
|
||||
const body = await response.text();
|
||||
const headers = {
|
||||
'Content-Type': response.headers.get('content-type') || 'application/json',
|
||||
'Cache-Control': response.headers.get('cache-control') || 'no-cache',
|
||||
...corsHeaders,
|
||||
};
|
||||
|
||||
const data = await requestPromise;
|
||||
if (!isValidSnapshot(data)) {
|
||||
throw new Error('Invalid AIS snapshot payload');
|
||||
}
|
||||
|
||||
setMemoryCachedSnapshot(cacheKey, data);
|
||||
void setCachedJson(cacheKey, data, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'MISS');
|
||||
|
||||
return new Response(JSON.stringify(data), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
|
||||
'X-Cache': 'MISS',
|
||||
...corsHeaders,
|
||||
},
|
||||
return new Response(body, {
|
||||
status: response.status,
|
||||
headers,
|
||||
});
|
||||
} catch (error) {
|
||||
const staleMemory = getMemoryCachedSnapshot(cacheKey, true);
|
||||
if (isValidSnapshot(staleMemory)) {
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'MEMORY-ERROR-FALLBACK');
|
||||
return new Response(JSON.stringify(staleMemory), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': `public, max-age=${CACHE_TTL_SECONDS}, s-maxage=${CACHE_TTL_SECONDS}, stale-while-revalidate=5`,
|
||||
'X-Cache': 'MEMORY-ERROR-FALLBACK',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/ais-snapshot', 'ERROR');
|
||||
return new Response(JSON.stringify({ error: getErrorMessage(error) }), {
|
||||
status: 502,
|
||||
const isTimeout = error?.name === 'AbortError';
|
||||
return new Response(JSON.stringify({
|
||||
error: isTimeout ? 'Relay timeout' : 'Relay request failed',
|
||||
details: error?.message || String(error),
|
||||
}), {
|
||||
status: isTimeout ? 504 : 502,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
} finally {
|
||||
inFlightByKey.delete(cacheKey);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
// Fetch AI/ML papers from ArXiv
|
||||
// Categories: cs.AI, cs.LG (Machine Learning), cs.CL (Computation and Language)
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const category = searchParams.get('category') || 'cs.AI'; // cs.AI, cs.LG, cs.CL
|
||||
const maxResults = searchParams.get('max_results') || '50';
|
||||
const sortBy = searchParams.get('sortBy') || 'submittedDate'; // submittedDate, lastUpdatedDate, relevance
|
||||
|
||||
// ArXiv API search query
|
||||
// Search for papers in specified category, sorted by date
|
||||
const query = `cat:${category}`;
|
||||
const apiUrl = `https://export.arxiv.org/api/query?search_query=${encodeURIComponent(query)}&start=0&max_results=${maxResults}&sortBy=${sortBy}&sortOrder=descending`;
|
||||
|
||||
const response = await fetch(apiUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'WorldMonitor/1.0 (AI Research Tracker)',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`ArXiv API returned ${response.status}`);
|
||||
}
|
||||
|
||||
const xmlData = await response.text();
|
||||
|
||||
// Parse XML to extract key information
|
||||
// Return raw XML for client-side parsing or transform here
|
||||
return new Response(xmlData, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/xml',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', // 1 hour cache
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: 'Failed to fetch ArXiv data',
|
||||
message: error.message
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1,38 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCacheTelemetrySnapshot } from './_cache-telemetry.js';
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(getCacheTelemetrySnapshot()), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,198 +0,0 @@
|
||||
import { getCachedJson, setCachedJson, mget, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
|
||||
const MODEL = 'llama-3.1-8b-instant';
|
||||
const CACHE_TTL_SECONDS = 86400;
|
||||
const CACHE_VERSION = 'v1';
|
||||
const MAX_BATCH_SIZE = 20;
|
||||
|
||||
const VALID_LEVELS = ['critical', 'high', 'medium', 'low', 'info'];
|
||||
const VALID_CATEGORIES = [
|
||||
'conflict', 'protest', 'disaster', 'diplomatic', 'economic',
|
||||
'terrorism', 'cyber', 'health', 'environmental', 'military',
|
||||
'crime', 'infrastructure', 'tech', 'general',
|
||||
];
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.GROQ_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ results: [], fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return new Response(JSON.stringify({ error: 'Payload too large' }), {
|
||||
status: 413,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
let body;
|
||||
try {
|
||||
body = await request.json();
|
||||
} catch {
|
||||
return new Response(JSON.stringify({ error: 'Invalid JSON body' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const { titles, variant = 'full' } = body;
|
||||
if (!Array.isArray(titles) || titles.length === 0) {
|
||||
return new Response(JSON.stringify({ error: 'titles array required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const batch = titles.slice(0, MAX_BATCH_SIZE);
|
||||
const results = new Array(batch.length).fill(null);
|
||||
const uncachedIndices = [];
|
||||
|
||||
const cacheKeys = batch.map(
|
||||
(t) => `classify:${CACHE_VERSION}:${hashString(t.toLowerCase() + ':' + variant)}`
|
||||
);
|
||||
const cached = await mget(...cacheKeys);
|
||||
for (let i = 0; i < cached.length; i++) {
|
||||
const val = cached[i];
|
||||
if (val && typeof val === 'object' && val.level) {
|
||||
results[i] = { level: val.level, category: val.category, cached: true };
|
||||
} else {
|
||||
uncachedIndices.push(i);
|
||||
}
|
||||
}
|
||||
|
||||
if (uncachedIndices.length === 0) {
|
||||
return new Response(JSON.stringify({ results }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const uncachedTitles = uncachedIndices.map((i) => batch[i]);
|
||||
const isTech = variant === 'tech';
|
||||
const numberedList = uncachedTitles.map((t, i) => `${i + 1}. ${t}`).join('\n');
|
||||
|
||||
const systemPrompt = `You classify news headlines into threat level and category. Return ONLY a valid JSON array, no other text.
|
||||
|
||||
Levels: critical, high, medium, low, info
|
||||
Categories: conflict, protest, disaster, diplomatic, economic, terrorism, cyber, health, environmental, military, crime, infrastructure, tech, general
|
||||
|
||||
${isTech ? 'Focus: technology, startups, AI, cybersecurity. Most tech news is "low" or "info" unless it involves outages, breaches, or major disruptions.' : 'Focus: geopolitical events, conflicts, disasters, diplomacy. Classify by real-world severity and impact.'}
|
||||
|
||||
Return a JSON array with one object per headline in order: [{"level":"...","category":"..."},...]`;
|
||||
|
||||
try {
|
||||
const response = await fetch(GROQ_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: numberedList },
|
||||
],
|
||||
temperature: 0,
|
||||
max_tokens: uncachedTitles.length * 60,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
console.error('[ClassifyBatch] Groq error:', response.status);
|
||||
return new Response(JSON.stringify({ results, fallback: true }), {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const raw = data.choices?.[0]?.message?.content?.trim();
|
||||
if (!raw) {
|
||||
return new Response(JSON.stringify({ results, fallback: true }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
const match = raw.match(/\[[\s\S]*\]/);
|
||||
if (match) {
|
||||
try { parsed = JSON.parse(match[0]); } catch { /* fall through */ }
|
||||
}
|
||||
}
|
||||
|
||||
if (!Array.isArray(parsed)) {
|
||||
return new Response(JSON.stringify({ results, fallback: true }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const cacheWrites = [];
|
||||
for (let i = 0; i < uncachedIndices.length; i++) {
|
||||
const classification = parsed[i];
|
||||
if (!classification) continue;
|
||||
|
||||
const level = VALID_LEVELS.includes(classification.level) ? classification.level : null;
|
||||
const category = VALID_CATEGORIES.includes(classification.category) ? classification.category : null;
|
||||
if (!level || !category) continue;
|
||||
|
||||
const idx = uncachedIndices[i];
|
||||
results[idx] = { level, category, cached: false };
|
||||
|
||||
const cacheKey = `classify:${CACHE_VERSION}:${hashString(batch[idx].toLowerCase() + ':' + variant)}`;
|
||||
cacheWrites.push(
|
||||
setCachedJson(cacheKey, { level, category, timestamp: Date.now() }, CACHE_TTL_SECONDS)
|
||||
);
|
||||
}
|
||||
|
||||
if (cacheWrites.length > 0) {
|
||||
await Promise.allSettled(cacheWrites);
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({ results }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[ClassifyBatch] Error:', error.message);
|
||||
return new Response(JSON.stringify({ results, fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,164 +0,0 @@
|
||||
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
|
||||
const MODEL = 'llama-3.1-8b-instant';
|
||||
const CACHE_TTL_SECONDS = 86400;
|
||||
const CACHE_VERSION = 'v1';
|
||||
|
||||
const VALID_LEVELS = ['critical', 'high', 'medium', 'low', 'info'];
|
||||
const VALID_CATEGORIES = [
|
||||
'conflict', 'protest', 'disaster', 'diplomatic', 'economic',
|
||||
'terrorism', 'cyber', 'health', 'environmental', 'military',
|
||||
'crime', 'infrastructure', 'tech', 'general',
|
||||
];
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'GET, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.GROQ_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const title = url.searchParams.get('title');
|
||||
const variant = url.searchParams.get('variant') || 'full';
|
||||
|
||||
if (!title) {
|
||||
return new Response(JSON.stringify({ error: 'title param required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const cacheKey = `classify:${CACHE_VERSION}:${hashString(title.toLowerCase() + ':' + variant)}`;
|
||||
|
||||
try {
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.level) {
|
||||
return new Response(JSON.stringify({
|
||||
level: cached.level,
|
||||
category: cached.category,
|
||||
confidence: 0.9,
|
||||
source: 'llm',
|
||||
cached: true,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const isTech = variant === 'tech';
|
||||
const systemPrompt = `You classify news headlines into threat level and category. Return ONLY valid JSON, no other text.
|
||||
|
||||
Levels: critical, high, medium, low, info
|
||||
Categories: conflict, protest, disaster, diplomatic, economic, terrorism, cyber, health, environmental, military, crime, infrastructure, tech, general
|
||||
|
||||
${isTech ? 'Focus: technology, startups, AI, cybersecurity. Most tech news is "low" or "info" unless it involves outages, breaches, or major disruptions.' : 'Focus: geopolitical events, conflicts, disasters, diplomacy. Classify by real-world severity and impact.'}
|
||||
|
||||
Return: {"level":"...","category":"..."}`;
|
||||
|
||||
const response = await fetch(GROQ_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: title },
|
||||
],
|
||||
temperature: 0,
|
||||
max_tokens: 50,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
console.error('[Classify] Groq error:', response.status);
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const raw = data.choices?.[0]?.message?.content?.trim();
|
||||
if (!raw) {
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
console.warn('[Classify] Invalid JSON from LLM:', raw);
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const level = VALID_LEVELS.includes(parsed.level) ? parsed.level : null;
|
||||
const category = VALID_CATEGORIES.includes(parsed.category) ? parsed.category : null;
|
||||
if (!level || !category) {
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
await setCachedJson(cacheKey, { level, category, timestamp: Date.now() }, CACHE_TTL_SECONDS);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
level,
|
||||
category,
|
||||
confidence: 0.9,
|
||||
source: 'llm',
|
||||
cached: false,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600',
|
||||
},
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('[Classify] Error:', error.message);
|
||||
return new Response(JSON.stringify({ fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,206 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'climate:anomalies:v1';
|
||||
const CACHE_TTL_SECONDS = 6 * 60 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: 15,
|
||||
windowMs: 60 * 1000,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(data && typeof data === 'object' && Array.isArray(data.anomalies));
|
||||
}
|
||||
|
||||
const MONITORED_ZONES = [
|
||||
{ name: 'Ukraine', lat: 48.4, lon: 31.2 },
|
||||
{ name: 'Middle East', lat: 33.0, lon: 44.0 },
|
||||
{ name: 'Sahel', lat: 14.0, lon: 0.0 },
|
||||
{ name: 'Horn of Africa', lat: 8.0, lon: 42.0 },
|
||||
{ name: 'South Asia', lat: 25.0, lon: 78.0 },
|
||||
{ name: 'California', lat: 36.8, lon: -119.4 },
|
||||
{ name: 'Amazon', lat: -3.4, lon: -60.0 },
|
||||
{ name: 'Australia', lat: -25.0, lon: 134.0 },
|
||||
{ name: 'Mediterranean', lat: 38.0, lon: 20.0 },
|
||||
{ name: 'Taiwan Strait', lat: 24.0, lon: 120.0 },
|
||||
{ name: 'Myanmar', lat: 19.8, lon: 96.7 },
|
||||
{ name: 'Central Africa', lat: 4.0, lon: 22.0 },
|
||||
{ name: 'Southern Africa', lat: -25.0, lon: 28.0 },
|
||||
{ name: 'Central Asia', lat: 42.0, lon: 65.0 },
|
||||
{ name: 'Caribbean', lat: 19.0, lon: -72.0 },
|
||||
];
|
||||
|
||||
function classifySeverity(tempDelta, precipDelta) {
|
||||
const absTemp = Math.abs(tempDelta);
|
||||
const absPrecip = Math.abs(precipDelta);
|
||||
if (absTemp >= 5 || absPrecip >= 80) return 'extreme';
|
||||
if (absTemp >= 3 || absPrecip >= 40) return 'moderate';
|
||||
return 'normal';
|
||||
}
|
||||
|
||||
function classifyType(tempDelta, precipDelta) {
|
||||
const absTemp = Math.abs(tempDelta);
|
||||
const absPrecip = Math.abs(precipDelta);
|
||||
if (absTemp >= absPrecip / 20) {
|
||||
if (tempDelta > 0 && precipDelta < -20) return 'mixed';
|
||||
if (tempDelta > 3) return 'warm';
|
||||
if (tempDelta < -3) return 'cold';
|
||||
}
|
||||
if (precipDelta > 40) return 'wet';
|
||||
if (precipDelta < -40) return 'dry';
|
||||
if (tempDelta > 0) return 'warm';
|
||||
return 'cold';
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited' }, {
|
||||
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const endDate = new Date();
|
||||
const startDate = new Date(Date.now() - 30 * 24 * 60 * 60 * 1000);
|
||||
const start = startDate.toISOString().split('T')[0];
|
||||
const end = endDate.toISOString().split('T')[0];
|
||||
|
||||
const fetchZone = async (zone) => {
|
||||
try {
|
||||
const params = new URLSearchParams({
|
||||
latitude: String(zone.lat),
|
||||
longitude: String(zone.lon),
|
||||
start_date: start,
|
||||
end_date: end,
|
||||
daily: 'temperature_2m_mean,precipitation_sum',
|
||||
timezone: 'UTC',
|
||||
});
|
||||
|
||||
const resp = await fetch(`https://archive-api.open-meteo.com/v1/archive?${params}`, {
|
||||
headers: { Accept: 'application/json' },
|
||||
});
|
||||
|
||||
if (!resp.ok) return null;
|
||||
const data = await resp.json();
|
||||
const temps = data.daily?.temperature_2m_mean || [];
|
||||
const precips = data.daily?.precipitation_sum || [];
|
||||
|
||||
if (temps.length < 14) return null;
|
||||
|
||||
const validTemps = temps.filter(t => t !== null);
|
||||
const validPrecips = precips.filter(p => p !== null);
|
||||
|
||||
const last7Temps = validTemps.slice(-7);
|
||||
const baseline30Temps = validTemps.slice(0, -7);
|
||||
const last7Precips = validPrecips.slice(-7);
|
||||
const baseline30Precips = validPrecips.slice(0, -7);
|
||||
|
||||
const avg = arr => arr.length ? arr.reduce((s, v) => s + v, 0) / arr.length : 0;
|
||||
|
||||
const tempDelta = avg(last7Temps) - avg(baseline30Temps);
|
||||
const precipDelta = avg(last7Precips) - avg(baseline30Precips);
|
||||
const severity = classifySeverity(tempDelta, precipDelta);
|
||||
|
||||
return {
|
||||
zone: zone.name,
|
||||
lat: zone.lat,
|
||||
lon: zone.lon,
|
||||
tempDelta: Math.round(tempDelta * 10) / 10,
|
||||
precipDelta: Math.round(precipDelta * 10) / 10,
|
||||
severity,
|
||||
type: classifyType(tempDelta, precipDelta),
|
||||
period: `${start} to ${end}`,
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
const results = await Promise.allSettled(MONITORED_ZONES.map(fetchZone));
|
||||
const anomalies = results
|
||||
.filter(r => r.status === 'fulfilled' && r.value)
|
||||
.map(r => r.value);
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
anomalies,
|
||||
timestamp: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/climate-anomalies', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, anomalies: [] }, {
|
||||
status: 500, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,64 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
function clampLimit(rawLimit) {
|
||||
const parsed = Number.parseInt(rawLimit || '', 10);
|
||||
if (!Number.isFinite(parsed)) return 50;
|
||||
return Math.max(1, Math.min(100, parsed));
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const dateRange = url.searchParams.get('dateRange') || '7d';
|
||||
const limit = clampLimit(url.searchParams.get('limit'));
|
||||
|
||||
const token = process.env.CLOUDFLARE_API_TOKEN;
|
||||
if (!token) {
|
||||
// Signal to client that outages feature is not configured
|
||||
return new Response(JSON.stringify({ configured: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(
|
||||
`https://api.cloudflare.com/client/v4/radar/annotations/outages?dateRange=${dateRange}&limit=${limit}`,
|
||||
{ headers: { 'Authorization': `Bearer ${token}` } }
|
||||
);
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=120, s-maxage=120, stale-while-revalidate=60', ...corsHeaders },
|
||||
});
|
||||
} catch (error) {
|
||||
// Return empty result on error so client circuit breaker doesn't trigger unnecessarily
|
||||
return new Response(JSON.stringify({ success: true, result: { annotations: [] } }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,154 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCachedJson, hashString, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const ALLOWED_CURRENCIES = ['usd', 'eur', 'gbp', 'jpy', 'cny', 'btc', 'eth'];
|
||||
const MAX_COIN_IDS = 20;
|
||||
const COIN_ID_PATTERN = /^[a-z0-9-]+$/;
|
||||
|
||||
const CACHE_TTL_SECONDS = 120; // 2 minutes
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const RESPONSE_CACHE_CONTROL = 'public, max-age=120, s-maxage=120, stale-while-revalidate=60';
|
||||
const CACHE_VERSION = 'v2';
|
||||
|
||||
// In-memory fallback cache for the current instance.
|
||||
let fallbackCache = { key: '', payload: null, timestamp: 0 };
|
||||
|
||||
function validateCoinIds(idsParam) {
|
||||
if (!idsParam) return 'bitcoin,ethereum,solana';
|
||||
|
||||
const ids = idsParam.split(',')
|
||||
.map(id => id.trim().toLowerCase())
|
||||
.filter(id => COIN_ID_PATTERN.test(id) && id.length <= 50)
|
||||
.slice(0, MAX_COIN_IDS);
|
||||
|
||||
return ids.length > 0 ? ids.join(',') : 'bitcoin,ethereum,solana';
|
||||
}
|
||||
|
||||
function validateCurrency(val) {
|
||||
const currency = (val || 'usd').toLowerCase();
|
||||
return ALLOWED_CURRENCIES.includes(currency) ? currency : 'usd';
|
||||
}
|
||||
|
||||
function validateBoolean(val, defaultVal) {
|
||||
if (val === 'true' || val === 'false') return val;
|
||||
return defaultVal;
|
||||
}
|
||||
|
||||
function getHeaders(cors, xCache, cacheControl = RESPONSE_CACHE_CONTROL) {
|
||||
return {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': cacheControl,
|
||||
'X-Cache': xCache,
|
||||
};
|
||||
}
|
||||
|
||||
function isValidPayload(payload) {
|
||||
return Boolean(
|
||||
payload &&
|
||||
typeof payload === 'object' &&
|
||||
typeof payload.body === 'string' &&
|
||||
Number.isFinite(payload.status)
|
||||
);
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
|
||||
const ids = validateCoinIds(url.searchParams.get('ids'));
|
||||
const vsCurrencies = validateCurrency(url.searchParams.get('vs_currencies'));
|
||||
const include24hrChange = validateBoolean(url.searchParams.get('include_24hr_change'), 'true');
|
||||
|
||||
const now = Date.now();
|
||||
const cacheKey = `${ids}:${vsCurrencies}:${include24hrChange}`;
|
||||
const redisKey = `coingecko:${CACHE_VERSION}:${hashString(cacheKey)}`;
|
||||
|
||||
const redisCached = await getCachedJson(redisKey);
|
||||
if (isValidPayload(redisCached)) {
|
||||
recordCacheTelemetry('/api/coingecko', 'REDIS-HIT');
|
||||
return new Response(redisCached.body, {
|
||||
status: redisCached.status,
|
||||
headers: getHeaders(cors, 'REDIS-HIT'),
|
||||
});
|
||||
}
|
||||
|
||||
if (
|
||||
isValidPayload(fallbackCache.payload) &&
|
||||
fallbackCache.key === cacheKey &&
|
||||
now - fallbackCache.timestamp < CACHE_TTL_MS
|
||||
) {
|
||||
recordCacheTelemetry('/api/coingecko', 'MEMORY-HIT');
|
||||
return new Response(fallbackCache.payload.body, {
|
||||
status: fallbackCache.payload.status,
|
||||
headers: getHeaders(cors, 'MEMORY-HIT'),
|
||||
});
|
||||
}
|
||||
|
||||
const endpoint = url.searchParams.get('endpoint');
|
||||
|
||||
try {
|
||||
let geckoUrl;
|
||||
if (endpoint === 'markets') {
|
||||
geckoUrl = `https://api.coingecko.com/api/v3/coins/markets?vs_currency=${vsCurrencies}&ids=${ids}&order=market_cap_desc&sparkline=true&price_change_percentage=24h`;
|
||||
} else {
|
||||
geckoUrl = `https://api.coingecko.com/api/v3/simple/price?ids=${ids}&vs_currencies=${vsCurrencies}&include_24hr_change=${include24hrChange}`;
|
||||
}
|
||||
const response = await fetch(geckoUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
// If rate limited, return cached data if available
|
||||
if (
|
||||
response.status === 429 &&
|
||||
isValidPayload(fallbackCache.payload) &&
|
||||
fallbackCache.key === cacheKey
|
||||
) {
|
||||
recordCacheTelemetry('/api/coingecko', 'STALE');
|
||||
return new Response(fallbackCache.payload.body, {
|
||||
status: fallbackCache.payload.status,
|
||||
headers: getHeaders(cors, 'STALE'),
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
|
||||
// Cache successful responses
|
||||
if (response.ok) {
|
||||
const payload = { body: data, status: response.status };
|
||||
fallbackCache = { key: cacheKey, payload, timestamp: Date.now() };
|
||||
void setCachedJson(redisKey, payload, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/coingecko', 'MISS');
|
||||
} else {
|
||||
recordCacheTelemetry('/api/coingecko', 'UPSTREAM-ERROR');
|
||||
}
|
||||
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: getHeaders(cors, 'MISS'),
|
||||
});
|
||||
} catch (error) {
|
||||
// Return cached data on error if available
|
||||
if (isValidPayload(fallbackCache.payload) && fallbackCache.key === cacheKey) {
|
||||
recordCacheTelemetry('/api/coingecko', 'ERROR-FALLBACK');
|
||||
return new Response(fallbackCache.payload.body, {
|
||||
status: fallbackCache.payload.status,
|
||||
headers: getHeaders(cors, 'ERROR-FALLBACK', 'public, max-age=120'),
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/coingecko', 'ERROR');
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,191 +0,0 @@
|
||||
/**
|
||||
* Country Intelligence Brief Endpoint
|
||||
* Generates AI-powered country situation briefs using Groq
|
||||
* Redis cached (2h TTL) for cross-user deduplication
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
|
||||
const MODEL = 'llama-3.1-8b-instant';
|
||||
const CACHE_TTL_SECONDS = 7200; // 2 hours
|
||||
const CACHE_VERSION = 'ci-v2';
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.GROQ_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ intel: null, fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return new Response(JSON.stringify({ error: 'Payload too large' }), {
|
||||
status: 413,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const { country, code, context } = await request.json();
|
||||
|
||||
if (!country || !code) {
|
||||
return new Response(JSON.stringify({ error: 'country and code required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Cache key includes country code + context hash (context changes as data updates)
|
||||
const contextHash = context ? hashString(JSON.stringify(context)).slice(0, 8) : 'no-ctx';
|
||||
const cacheKey = `${CACHE_VERSION}:${code}:${contextHash}`;
|
||||
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.brief) {
|
||||
console.log('[CountryIntel] Cache hit:', code);
|
||||
return new Response(JSON.stringify({ ...cached, cached: true }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
// Build data context section
|
||||
const dataLines = [];
|
||||
if (context?.score != null) {
|
||||
const changeStr = context.change24h ? ` (${context.change24h > 0 ? '+' : ''}${context.change24h} in 24h)` : '';
|
||||
dataLines.push(`Instability Score: ${context.score}/100 (${context.level || 'unknown'}) — trend: ${context.trend || 'unknown'}${changeStr}`);
|
||||
}
|
||||
if (context?.components) {
|
||||
const c = context.components;
|
||||
dataLines.push(`Score Components: Unrest ${c.unrest ?? '?'}/100, Security ${c.security ?? '?'}/100, Information ${c.information ?? '?'}/100`);
|
||||
}
|
||||
if (context?.protests != null) dataLines.push(`Active protests in/near country (7d): ${context.protests}`);
|
||||
if (context?.militaryFlights != null) dataLines.push(`Military aircraft detected in/near country: ${context.militaryFlights}`);
|
||||
if (context?.militaryVessels != null) dataLines.push(`Military vessels detected in/near country: ${context.militaryVessels}`);
|
||||
if (context?.outages != null) dataLines.push(`Internet outages: ${context.outages}`);
|
||||
if (context?.earthquakes != null) dataLines.push(`Recent earthquakes: ${context.earthquakes}`);
|
||||
if (context?.stockIndex) dataLines.push(`Stock Market Index: ${context.stockIndex}`);
|
||||
if (context?.convergenceScore != null) {
|
||||
dataLines.push(`Signal convergence score: ${context.convergenceScore}/100 (multiple signal types detected: ${(context.signalTypes || []).join(', ')})`);
|
||||
}
|
||||
if (context?.regionalConvergence?.length > 0) {
|
||||
dataLines.push(`\nRegional convergence alerts:`);
|
||||
context.regionalConvergence.forEach(r => dataLines.push(`- ${r}`));
|
||||
}
|
||||
if (context?.headlines?.length > 0) {
|
||||
dataLines.push(`\nRecent headlines mentioning ${country} (${context.headlines.length} found):`);
|
||||
context.headlines.slice(0, 15).forEach((h, i) => dataLines.push(`${i + 1}. ${h}`));
|
||||
}
|
||||
|
||||
const dataSection = dataLines.length > 0
|
||||
? `\nCURRENT SENSOR DATA:\n${dataLines.join('\n')}`
|
||||
: '\nNo real-time sensor data available for this country.';
|
||||
|
||||
const dateStr = new Date().toISOString().split('T')[0];
|
||||
|
||||
const systemPrompt = `You are a senior intelligence analyst providing comprehensive country situation briefs. Current date: ${dateStr}. Donald Trump is the current US President (second term, inaugurated Jan 2025).
|
||||
|
||||
Write a thorough, data-driven intelligence brief for the requested country. Structure:
|
||||
|
||||
1. **Current Situation** — What is happening right now. Reference specific data: instability scores, protest counts, military presence, outages. Explain what the numbers mean in context.
|
||||
|
||||
2. **Military & Security Posture** — Analyze military activity in/near the country. What forces are present? What does the positioning suggest? What are foreign nations doing in this theater?
|
||||
|
||||
3. **Key Risk Factors** — What drives instability or stability. Connect the dots between different signals (protests + outages = potential crackdown? military buildup + diplomatic tensions = escalation risk?). Reference specific headlines.
|
||||
|
||||
4. **Regional Context** — How does this country's situation affect or relate to its neighbors and the broader region? Reference any convergence alerts.
|
||||
|
||||
5. **Outlook & Watch Items** — What to monitor in the near term. Be specific about indicators that would signal escalation or de-escalation.
|
||||
|
||||
Rules:
|
||||
- Be specific and analytical. Reference the data provided (scores, counts, headlines, convergence).
|
||||
- If data shows low activity, say so — don't manufacture threats.
|
||||
- Connect signals: explain what combinations of data points suggest.
|
||||
- 5-6 paragraphs, 300-400 words.
|
||||
- No speculation beyond what the data supports.
|
||||
- Use plain language, not jargon.
|
||||
- If military assets are 0, don't speculate about military presence — say monitoring shows no current military activity.
|
||||
- When referencing a specific headline from the numbered list, cite it as [N] where N is the headline number (e.g. "tensions escalated [3]"). Only cite headlines you directly reference.`;
|
||||
|
||||
const userPrompt = `Country: ${country} (${code})${dataSection}`;
|
||||
|
||||
const groqRes = await fetch(GROQ_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
temperature: 0.4,
|
||||
max_tokens: 900,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!groqRes.ok) {
|
||||
const errText = await groqRes.text();
|
||||
console.error('[CountryIntel] Groq error:', groqRes.status, errText);
|
||||
return new Response(JSON.stringify({ error: 'AI service error', fallback: true }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const groqData = await groqRes.json();
|
||||
const brief = groqData.choices?.[0]?.message?.content || '';
|
||||
|
||||
const result = {
|
||||
brief,
|
||||
country,
|
||||
code,
|
||||
model: MODEL,
|
||||
generatedAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
if (brief) {
|
||||
await setCachedJson(cacheKey, result, CACHE_TTL_SECONDS);
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[CountryIntel] Error:', err);
|
||||
return new Response(JSON.stringify({ error: 'Internal error' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,371 +0,0 @@
|
||||
import { strict as assert } from 'node:assert';
|
||||
import test from 'node:test';
|
||||
import handler, {
|
||||
__resetCyberThreatsState,
|
||||
__testDedupeThreats,
|
||||
__testParseFeodoRecords,
|
||||
} from './cyber-threats.js';
|
||||
|
||||
const ORIGINAL_FETCH = globalThis.fetch;
|
||||
const ORIGINAL_URLHAUS_KEY = process.env.URLHAUS_AUTH_KEY;
|
||||
const ORIGINAL_OTX_KEY = process.env.OTX_API_KEY;
|
||||
const ORIGINAL_ABUSEIPDB_KEY = process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
function makeRequest(path = '/api/cyber-threats', ip = '198.51.100.10') {
|
||||
const headers = new Headers();
|
||||
headers.set('x-forwarded-for', ip);
|
||||
return new Request(`https://worldmonitor.app${path}`, { headers });
|
||||
}
|
||||
|
||||
function jsonResponse(body, status = 200) {
|
||||
return new Response(JSON.stringify(body), {
|
||||
status,
|
||||
headers: { 'content-type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
function textResponse(body, status = 200) {
|
||||
return new Response(body, {
|
||||
status,
|
||||
headers: { 'content-type': 'text/plain' },
|
||||
});
|
||||
}
|
||||
|
||||
// Mock that handles all 5 source URLs + geo enrichment
|
||||
function createMockFetch({ feodo, urlhaus, c2intel, otx, abuseipdb, geo } = {}) {
|
||||
return async (url) => {
|
||||
const target = String(url);
|
||||
if (target.includes('feodotracker.abuse.ch') && feodo) return feodo(target);
|
||||
if (target.includes('urlhaus-api.abuse.ch') && urlhaus) return urlhaus(target);
|
||||
if (target.includes('raw.githubusercontent.com') && target.includes('C2IntelFeeds') && c2intel) return c2intel(target);
|
||||
if (target.includes('otx.alienvault.com') && otx) return otx(target);
|
||||
if (target.includes('api.abuseipdb.com') && abuseipdb) return abuseipdb(target);
|
||||
if ((target.includes('ipwho.is') || target.includes('ipapi.co')) && geo) return geo(target);
|
||||
// Default: return 404 for unconfigured sources
|
||||
return new Response('not found', { status: 404 });
|
||||
};
|
||||
}
|
||||
|
||||
test.afterEach(() => {
|
||||
globalThis.fetch = ORIGINAL_FETCH;
|
||||
process.env.URLHAUS_AUTH_KEY = ORIGINAL_URLHAUS_KEY;
|
||||
process.env.OTX_API_KEY = ORIGINAL_OTX_KEY;
|
||||
process.env.ABUSEIPDB_API_KEY = ORIGINAL_ABUSEIPDB_KEY;
|
||||
__resetCyberThreatsState();
|
||||
});
|
||||
|
||||
test('Feodo parser accepts online and recent offline entries, filters stale', () => {
|
||||
const nowMs = Date.parse('2026-02-15T12:00:00.000Z');
|
||||
const records = [
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
first_seen: '2026-02-14 10:00:00 UTC',
|
||||
last_online: '2026-02-15 10:00:00 UTC',
|
||||
malware: 'QakBot',
|
||||
},
|
||||
{
|
||||
ip_address: '5.6.7.8',
|
||||
status: 'offline',
|
||||
first_seen: '2026-02-14 10:00:00 UTC',
|
||||
last_online: '2026-02-15 10:00:00 UTC',
|
||||
malware: 'Emotet',
|
||||
},
|
||||
{
|
||||
ip_address: '9.9.9.9',
|
||||
status: 'online',
|
||||
first_seen: '2025-10-01 10:00:00 UTC',
|
||||
last_online: '2025-10-02 10:00:00 UTC',
|
||||
malware: 'generic',
|
||||
},
|
||||
{
|
||||
ip_address: '2.2.2.2',
|
||||
first_seen: '2026-02-14 10:00:00 UTC',
|
||||
last_online: '2026-02-15 10:00:00 UTC',
|
||||
malware: 'generic',
|
||||
},
|
||||
];
|
||||
|
||||
const parsed = __testParseFeodoRecords(records, { nowMs, days: 14 });
|
||||
// online + offline (recent) + no-status (recent) = 3; stale (9.9.9.9) filtered
|
||||
assert.equal(parsed.length, 3);
|
||||
assert.equal(parsed[0].indicator, '1.2.3.4');
|
||||
assert.equal(parsed[0].severity, 'critical');
|
||||
assert.equal(parsed[1].indicator, '5.6.7.8');
|
||||
assert.equal(parsed[1].severity, 'medium');
|
||||
assert.equal(parsed[0].firstSeen?.endsWith('Z'), true);
|
||||
assert.equal(parsed[0].lastSeen?.endsWith('Z'), true);
|
||||
});
|
||||
|
||||
test('dedupes by source + indicatorType + indicator', () => {
|
||||
const deduped = __testDedupeThreats([
|
||||
{
|
||||
id: 'a',
|
||||
source: 'feodo',
|
||||
type: 'c2_server',
|
||||
indicatorType: 'ip',
|
||||
indicator: '1.2.3.4',
|
||||
severity: 'high',
|
||||
tags: ['a'],
|
||||
firstSeen: '2026-02-10T00:00:00.000Z',
|
||||
lastSeen: '2026-02-11T00:00:00.000Z',
|
||||
},
|
||||
{
|
||||
id: 'b',
|
||||
source: 'feodo',
|
||||
type: 'c2_server',
|
||||
indicatorType: 'ip',
|
||||
indicator: '1.2.3.4',
|
||||
severity: 'critical',
|
||||
tags: ['b'],
|
||||
firstSeen: '2026-02-12T00:00:00.000Z',
|
||||
lastSeen: '2026-02-13T00:00:00.000Z',
|
||||
},
|
||||
{
|
||||
id: 'c',
|
||||
source: 'urlhaus',
|
||||
type: 'malicious_url',
|
||||
indicatorType: 'domain',
|
||||
indicator: 'bad.example',
|
||||
severity: 'medium',
|
||||
tags: [],
|
||||
firstSeen: '2026-02-11T00:00:00.000Z',
|
||||
lastSeen: '2026-02-11T01:00:00.000Z',
|
||||
},
|
||||
]);
|
||||
|
||||
assert.equal(deduped.length, 2);
|
||||
const feodo = deduped.find((item) => item.source === 'feodo');
|
||||
assert.equal(feodo?.severity, 'critical');
|
||||
assert.equal(feodo?.tags.includes('a'), true);
|
||||
assert.equal(feodo?.tags.includes('b'), true);
|
||||
});
|
||||
|
||||
test('API aggregates from all 5 sources', async () => {
|
||||
process.env.URLHAUS_AUTH_KEY = 'test-key';
|
||||
process.env.OTX_API_KEY = 'test-otx';
|
||||
process.env.ABUSEIPDB_API_KEY = 'test-abuse';
|
||||
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]),
|
||||
urlhaus: () => jsonResponse({
|
||||
urls: [{
|
||||
url: 'http://5.5.5.5/malware.exe',
|
||||
host: '5.5.5.5',
|
||||
url_status: 'online',
|
||||
threat: 'malware_download',
|
||||
tags: ['malware'],
|
||||
dateadded: '2026-02-14T08:00:00.000Z',
|
||||
latitude: 48.86,
|
||||
longitude: 2.35,
|
||||
country: 'FR',
|
||||
}],
|
||||
}),
|
||||
c2intel: () => textResponse(
|
||||
'#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP\n10.10.10.11,Possible Metasploit C2 IP',
|
||||
),
|
||||
otx: () => jsonResponse({
|
||||
results: [{
|
||||
indicator: '20.20.20.20',
|
||||
title: 'APT threat',
|
||||
tags: ['apt', 'c2'],
|
||||
created: '2026-02-13T00:00:00.000Z',
|
||||
modified: '2026-02-14T00:00:00.000Z',
|
||||
}],
|
||||
}),
|
||||
abuseipdb: () => jsonResponse({
|
||||
data: [{
|
||||
ipAddress: '30.30.30.30',
|
||||
abuseConfidenceScore: 98,
|
||||
lastReportedAt: '2026-02-15T06:00:00.000Z',
|
||||
countryCode: 'CN',
|
||||
latitude: 39.9,
|
||||
longitude: 116.4,
|
||||
}],
|
||||
}),
|
||||
geo: () => jsonResponse({ success: true, latitude: 40.0, longitude: -74.0, country_code: 'US' }),
|
||||
});
|
||||
|
||||
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.20'));
|
||||
assert.equal(response.status, 200);
|
||||
|
||||
const body = await response.json();
|
||||
assert.equal(body.success, true);
|
||||
assert.equal(body.sources.feodo.ok, true);
|
||||
assert.equal(body.sources.urlhaus.ok, true);
|
||||
assert.equal(body.sources.c2intel.ok, true);
|
||||
assert.equal(body.sources.otx.ok, true);
|
||||
assert.equal(body.sources.abuseipdb.ok, true);
|
||||
// 5 sources, all with coords (3 native + 3 via geo enrichment mock)
|
||||
assert.equal(body.data.length >= 5, true);
|
||||
});
|
||||
|
||||
test('API works with only free sources when keys missing', async () => {
|
||||
delete process.env.URLHAUS_AUTH_KEY;
|
||||
delete process.env.OTX_API_KEY;
|
||||
delete process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]),
|
||||
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
|
||||
});
|
||||
|
||||
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.11'));
|
||||
assert.equal(response.status, 200);
|
||||
assert.equal(response.headers.get('X-Cache'), 'MISS');
|
||||
|
||||
const body = await response.json();
|
||||
assert.equal(body.success, true);
|
||||
assert.equal(body.partial, false);
|
||||
assert.equal(body.sources.feodo.ok, true);
|
||||
assert.equal(body.sources.c2intel.ok, true);
|
||||
assert.equal(body.sources.urlhaus.ok, false);
|
||||
assert.equal(body.sources.urlhaus.reason, 'missing_auth_key');
|
||||
assert.equal(body.sources.otx.ok, false);
|
||||
assert.equal(body.sources.otx.reason, 'missing_api_key');
|
||||
assert.equal(body.sources.abuseipdb.ok, false);
|
||||
assert.equal(body.sources.abuseipdb.reason, 'missing_api_key');
|
||||
assert.equal(Array.isArray(body.data), true);
|
||||
});
|
||||
|
||||
test('API marks partial=true when URLhaus is enabled but fails', async () => {
|
||||
process.env.URLHAUS_AUTH_KEY = 'test-key';
|
||||
delete process.env.OTX_API_KEY;
|
||||
delete process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]),
|
||||
urlhaus: () => new Response('boom', { status: 500 }),
|
||||
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
|
||||
});
|
||||
|
||||
const response = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.12'));
|
||||
assert.equal(response.status, 200);
|
||||
|
||||
const body = await response.json();
|
||||
assert.equal(body.success, true);
|
||||
assert.equal(body.partial, true);
|
||||
assert.equal(body.sources.urlhaus.ok, false);
|
||||
assert.equal(body.sources.urlhaus.reason, 'urlhaus_http_500');
|
||||
});
|
||||
|
||||
test('API returns memory cache hit on repeated request', async () => {
|
||||
delete process.env.URLHAUS_AUTH_KEY;
|
||||
delete process.env.OTX_API_KEY;
|
||||
delete process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
let feodoCalls = 0;
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => {
|
||||
feodoCalls += 1;
|
||||
return jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]);
|
||||
},
|
||||
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
|
||||
});
|
||||
|
||||
const first = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.13'));
|
||||
assert.equal(first.status, 200);
|
||||
assert.equal(first.headers.get('X-Cache'), 'MISS');
|
||||
assert.equal(feodoCalls, 1);
|
||||
|
||||
globalThis.fetch = async () => {
|
||||
throw new Error('network should not be hit for memory cache');
|
||||
};
|
||||
|
||||
const second = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.13'));
|
||||
assert.equal(second.status, 200);
|
||||
assert.equal(second.headers.get('X-Cache'), 'MEMORY-HIT');
|
||||
assert.equal(feodoCalls, 1);
|
||||
});
|
||||
|
||||
test('API returns stale fallback when upstream fails after fresh cache TTL', async () => {
|
||||
delete process.env.URLHAUS_AUTH_KEY;
|
||||
delete process.env.OTX_API_KEY;
|
||||
delete process.env.ABUSEIPDB_API_KEY;
|
||||
|
||||
const baseNow = Date.parse('2026-02-15T12:00:00.000Z');
|
||||
const originalDateNow = Date.now;
|
||||
Date.now = () => baseNow;
|
||||
|
||||
try {
|
||||
globalThis.fetch = createMockFetch({
|
||||
feodo: () => jsonResponse([
|
||||
{
|
||||
ip_address: '1.2.3.4',
|
||||
status: 'online',
|
||||
last_online: '2026-02-15T10:00:00.000Z',
|
||||
first_seen: '2026-02-14T10:00:00.000Z',
|
||||
malware: 'QakBot',
|
||||
country: 'GB',
|
||||
lat: 51.5,
|
||||
lon: -0.12,
|
||||
},
|
||||
]),
|
||||
c2intel: () => textResponse('#ip,ioc\n10.10.10.10,Possible Cobaltstrike C2 IP'),
|
||||
});
|
||||
|
||||
const first = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.14'));
|
||||
assert.equal(first.status, 200);
|
||||
assert.equal(first.headers.get('X-Cache'), 'MISS');
|
||||
|
||||
Date.now = () => baseNow + (11 * 60 * 1000);
|
||||
globalThis.fetch = async () => {
|
||||
throw new Error('forced upstream failure');
|
||||
};
|
||||
|
||||
const stale = await handler(makeRequest('/api/cyber-threats?limit=100&days=14', '198.51.100.14'));
|
||||
assert.equal(stale.status, 200);
|
||||
assert.equal(stale.headers.get('X-Cache'), 'STALE');
|
||||
|
||||
const body = await stale.json();
|
||||
assert.equal(body.success, true);
|
||||
assert.equal(Array.isArray(body.data), true);
|
||||
assert.equal(body.data.length >= 1, true);
|
||||
} finally {
|
||||
Date.now = originalDateNow;
|
||||
}
|
||||
});
|
||||
@@ -1,77 +1,16 @@
|
||||
// Tech Events API - Parses Techmeme ICS feed and dev.events RSS, returns structured events
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
/**
|
||||
* Comprehensive city geocoding database (500+ cities worldwide).
|
||||
* Extracted from the legacy api/tech-events.js endpoint.
|
||||
*/
|
||||
|
||||
const ICS_URL = 'https://www.techmeme.com/newsy_events.ics';
|
||||
const DEV_EVENTS_RSS = 'https://dev.events/rss.xml';
|
||||
export interface CityCoord {
|
||||
lat: number;
|
||||
lng: number;
|
||||
country: string;
|
||||
virtual?: boolean;
|
||||
}
|
||||
|
||||
// Curated major tech events that may fall off limited RSS feeds
|
||||
// These are manually maintained for important conferences
|
||||
const CURATED_EVENTS = [
|
||||
{
|
||||
id: 'step-dubai-2026',
|
||||
title: 'STEP Dubai 2026',
|
||||
type: 'conference',
|
||||
location: 'Dubai Internet City, Dubai',
|
||||
coords: { lat: 25.0956, lng: 55.1548, country: 'UAE', original: 'Dubai Internet City, Dubai' },
|
||||
startDate: '2026-02-11',
|
||||
endDate: '2026-02-12',
|
||||
url: 'https://dubai.stepconference.com',
|
||||
source: 'curated',
|
||||
description: 'Intelligence Everywhere: The AI Economy - 8,000+ attendees, 400+ startups',
|
||||
},
|
||||
{
|
||||
id: 'gitex-global-2026',
|
||||
title: 'GITEX Global 2026',
|
||||
type: 'conference',
|
||||
location: 'Dubai World Trade Centre, Dubai',
|
||||
coords: { lat: 25.2285, lng: 55.2867, country: 'UAE', original: 'Dubai World Trade Centre, Dubai' },
|
||||
startDate: '2026-12-07',
|
||||
endDate: '2026-12-11',
|
||||
url: 'https://www.gitex.com',
|
||||
source: 'curated',
|
||||
description: 'World\'s largest tech & startup show',
|
||||
},
|
||||
{
|
||||
id: 'token2049-dubai-2026',
|
||||
title: 'TOKEN2049 Dubai 2026',
|
||||
type: 'conference',
|
||||
location: 'Dubai, UAE',
|
||||
coords: { lat: 25.2048, lng: 55.2708, country: 'UAE', original: 'Dubai, UAE' },
|
||||
startDate: '2026-04-29',
|
||||
endDate: '2026-04-30',
|
||||
url: 'https://www.token2049.com',
|
||||
source: 'curated',
|
||||
description: 'Premier crypto event in Dubai',
|
||||
},
|
||||
{
|
||||
id: 'collision-2026',
|
||||
title: 'Collision 2026',
|
||||
type: 'conference',
|
||||
location: 'Toronto, Canada',
|
||||
coords: { lat: 43.6532, lng: -79.3832, country: 'Canada', original: 'Toronto, Canada' },
|
||||
startDate: '2026-06-22',
|
||||
endDate: '2026-06-25',
|
||||
url: 'https://collisionconf.com',
|
||||
source: 'curated',
|
||||
description: 'North America\'s fastest growing tech conference',
|
||||
},
|
||||
{
|
||||
id: 'web-summit-2026',
|
||||
title: 'Web Summit 2026',
|
||||
type: 'conference',
|
||||
location: 'Lisbon, Portugal',
|
||||
coords: { lat: 38.7223, lng: -9.1393, country: 'Portugal', original: 'Lisbon, Portugal' },
|
||||
startDate: '2026-11-02',
|
||||
endDate: '2026-11-05',
|
||||
url: 'https://websummit.com',
|
||||
source: 'curated',
|
||||
description: 'The world\'s premier tech conference',
|
||||
},
|
||||
];
|
||||
|
||||
// Comprehensive city geocoding database (500+ cities worldwide)
|
||||
const CITY_COORDS = {
|
||||
export const CITY_COORDS: Record<string, CityCoord> = {
|
||||
// North America - USA
|
||||
'san francisco': { lat: 37.7749, lng: -122.4194, country: 'USA' },
|
||||
'san jose': { lat: 37.3382, lng: -121.8863, country: 'USA' },
|
||||
@@ -164,7 +103,7 @@ const CITY_COORDS = {
|
||||
'tijuana': { lat: 32.5149, lng: -117.0382, country: 'Mexico' },
|
||||
'cancun': { lat: 21.1619, lng: -86.8515, country: 'Mexico' },
|
||||
'panama city': { lat: 8.9824, lng: -79.5199, country: 'Panama' },
|
||||
'san jose': { lat: 9.9281, lng: -84.0907, country: 'Costa Rica' },
|
||||
'san jose cr': { lat: 9.9281, lng: -84.0907, country: 'Costa Rica' },
|
||||
|
||||
// South America
|
||||
'sao paulo': { lat: -23.5505, lng: -46.6333, country: 'Brazil' },
|
||||
@@ -176,9 +115,9 @@ const CITY_COORDS = {
|
||||
'buenos aires': { lat: -34.6037, lng: -58.3816, country: 'Argentina' },
|
||||
'santiago': { lat: -33.4489, lng: -70.6693, country: 'Chile' },
|
||||
'bogota': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
|
||||
'bogotá': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
|
||||
'bogot\u00e1': { lat: 4.7110, lng: -74.0721, country: 'Colombia' },
|
||||
'medellin': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
|
||||
'medellín': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
|
||||
'medell\u00edn': { lat: 6.2476, lng: -75.5658, country: 'Colombia' },
|
||||
'lima': { lat: -12.0464, lng: -77.0428, country: 'Peru' },
|
||||
'caracas': { lat: 10.4806, lng: -66.9036, country: 'Venezuela' },
|
||||
'montevideo': { lat: -34.9011, lng: -56.1645, country: 'Uruguay' },
|
||||
@@ -186,7 +125,7 @@ const CITY_COORDS = {
|
||||
|
||||
// Europe - UK & Ireland
|
||||
'london': { lat: 51.5074, lng: -0.1278, country: 'UK' },
|
||||
'cambridge': { lat: 52.2053, lng: 0.1218, country: 'UK' },
|
||||
'cambridge uk': { lat: 52.2053, lng: 0.1218, country: 'UK' },
|
||||
'oxford': { lat: 51.7520, lng: -1.2577, country: 'UK' },
|
||||
'manchester': { lat: 53.4808, lng: -2.2426, country: 'UK' },
|
||||
'birmingham': { lat: 52.4862, lng: -1.8904, country: 'UK' },
|
||||
@@ -214,12 +153,12 @@ const CITY_COORDS = {
|
||||
'monaco': { lat: 43.7384, lng: 7.4246, country: 'Monaco' },
|
||||
'berlin': { lat: 52.5200, lng: 13.4050, country: 'Germany' },
|
||||
'munich': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
|
||||
'münchen': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
|
||||
'm\u00fcnchen': { lat: 48.1351, lng: 11.5820, country: 'Germany' },
|
||||
'frankfurt': { lat: 50.1109, lng: 8.6821, country: 'Germany' },
|
||||
'hamburg': { lat: 53.5511, lng: 9.9937, country: 'Germany' },
|
||||
'cologne': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
|
||||
'köln': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
|
||||
'düsseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
|
||||
'k\u00f6ln': { lat: 50.9375, lng: 6.9603, country: 'Germany' },
|
||||
'd\u00fcsseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
|
||||
'dusseldorf': { lat: 51.2277, lng: 6.7735, country: 'Germany' },
|
||||
'stuttgart': { lat: 48.7758, lng: 9.1829, country: 'Germany' },
|
||||
'hanover': { lat: 52.3759, lng: 9.7320, country: 'Germany' },
|
||||
@@ -237,9 +176,9 @@ const CITY_COORDS = {
|
||||
'ghent': { lat: 51.0543, lng: 3.7174, country: 'Belgium' },
|
||||
'luxembourg': { lat: 49.6116, lng: 6.1319, country: 'Luxembourg' },
|
||||
'zurich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
|
||||
'zürich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
|
||||
'z\u00fcrich': { lat: 47.3769, lng: 8.5417, country: 'Switzerland' },
|
||||
'geneva': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
|
||||
'genève': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
|
||||
'gen\u00e8ve': { lat: 46.2044, lng: 6.1432, country: 'Switzerland' },
|
||||
'basel': { lat: 47.5596, lng: 7.5886, country: 'Switzerland' },
|
||||
'bern': { lat: 46.9480, lng: 7.4474, country: 'Switzerland' },
|
||||
'lausanne': { lat: 46.5197, lng: 6.6323, country: 'Switzerland' },
|
||||
@@ -257,7 +196,7 @@ const CITY_COORDS = {
|
||||
'seville': { lat: 37.3891, lng: -5.9845, country: 'Spain' },
|
||||
'sevilla': { lat: 37.3891, lng: -5.9845, country: 'Spain' },
|
||||
'malaga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
|
||||
'málaga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
|
||||
'm\u00e1laga': { lat: 36.7213, lng: -4.4214, country: 'Spain' },
|
||||
'bilbao': { lat: 43.2630, lng: -2.9350, country: 'Spain' },
|
||||
'lisbon': { lat: 38.7223, lng: -9.1393, country: 'Portugal' },
|
||||
'lisboa': { lat: 38.7223, lng: -9.1393, country: 'Portugal' },
|
||||
@@ -283,11 +222,11 @@ const CITY_COORDS = {
|
||||
// Europe - Northern
|
||||
'stockholm': { lat: 59.3293, lng: 18.0686, country: 'Sweden' },
|
||||
'gothenburg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
|
||||
'göteborg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
|
||||
'malmö': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
|
||||
'g\u00f6teborg': { lat: 57.7089, lng: 11.9746, country: 'Sweden' },
|
||||
'malm\u00f6': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
|
||||
'malmo': { lat: 55.6050, lng: 13.0038, country: 'Sweden' },
|
||||
'copenhagen': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
|
||||
'københavn': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
|
||||
'k\u00f8benhavn': { lat: 55.6761, lng: 12.5683, country: 'Denmark' },
|
||||
'aarhus': { lat: 56.1629, lng: 10.2039, country: 'Denmark' },
|
||||
'oslo': { lat: 59.9139, lng: 10.7522, country: 'Norway' },
|
||||
'bergen': { lat: 60.3913, lng: 5.3221, country: 'Norway' },
|
||||
@@ -300,16 +239,16 @@ const CITY_COORDS = {
|
||||
'warsaw': { lat: 52.2297, lng: 21.0122, country: 'Poland' },
|
||||
'warszawa': { lat: 52.2297, lng: 21.0122, country: 'Poland' },
|
||||
'krakow': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
|
||||
'kraków': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
|
||||
'krak\u00f3w': { lat: 50.0647, lng: 19.9450, country: 'Poland' },
|
||||
'wroclaw': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
|
||||
'wrocław': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
|
||||
'wroc\u0142aw': { lat: 51.1079, lng: 17.0385, country: 'Poland' },
|
||||
'gdansk': { lat: 54.3520, lng: 18.6466, country: 'Poland' },
|
||||
'prague': { lat: 50.0755, lng: 14.4378, country: 'Czech Republic' },
|
||||
'praha': { lat: 50.0755, lng: 14.4378, country: 'Czech Republic' },
|
||||
'brno': { lat: 49.1951, lng: 16.6068, country: 'Czech Republic' },
|
||||
'budapest': { lat: 47.4979, lng: 19.0402, country: 'Hungary' },
|
||||
'bucharest': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
|
||||
'bucurești': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
|
||||
'bucure\u0219ti': { lat: 44.4268, lng: 26.1025, country: 'Romania' },
|
||||
'cluj-napoca': { lat: 46.7712, lng: 23.6236, country: 'Romania' },
|
||||
'sofia': { lat: 42.6977, lng: 23.3219, country: 'Bulgaria' },
|
||||
'belgrade': { lat: 44.7866, lng: 20.4489, country: 'Serbia' },
|
||||
@@ -464,274 +403,3 @@ const CITY_COORDS = {
|
||||
'virtual': { lat: 0, lng: 0, country: 'Virtual', virtual: true },
|
||||
'hybrid': { lat: 0, lng: 0, country: 'Virtual', virtual: true },
|
||||
};
|
||||
|
||||
function normalizeLocation(location) {
|
||||
if (!location) return null;
|
||||
|
||||
// Clean up the location string
|
||||
let normalized = location.toLowerCase().trim();
|
||||
|
||||
// Remove common suffixes/prefixes
|
||||
normalized = normalized.replace(/^hybrid:\s*/i, '');
|
||||
normalized = normalized.replace(/,\s*(usa|us|uk|canada)$/i, '');
|
||||
|
||||
// Direct lookup
|
||||
if (CITY_COORDS[normalized]) {
|
||||
return { ...CITY_COORDS[normalized], original: location };
|
||||
}
|
||||
|
||||
// Try removing state/country suffix
|
||||
const parts = normalized.split(',');
|
||||
if (parts.length > 1) {
|
||||
const city = parts[0].trim();
|
||||
if (CITY_COORDS[city]) {
|
||||
return { ...CITY_COORDS[city], original: location };
|
||||
}
|
||||
}
|
||||
|
||||
// Try fuzzy match (contains)
|
||||
for (const [key, coords] of Object.entries(CITY_COORDS)) {
|
||||
if (normalized.includes(key) || key.includes(normalized)) {
|
||||
return { ...coords, original: location };
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
function parseICS(icsText) {
|
||||
const events = [];
|
||||
const eventBlocks = icsText.split('BEGIN:VEVENT').slice(1);
|
||||
|
||||
for (const block of eventBlocks) {
|
||||
const summaryMatch = block.match(/SUMMARY:(.+)/);
|
||||
const locationMatch = block.match(/LOCATION:(.+)/);
|
||||
const dtstartMatch = block.match(/DTSTART;VALUE=DATE:(\d+)/);
|
||||
const dtendMatch = block.match(/DTEND;VALUE=DATE:(\d+)/);
|
||||
const urlMatch = block.match(/URL:(.+)/);
|
||||
const uidMatch = block.match(/UID:(.+)/);
|
||||
|
||||
if (summaryMatch && dtstartMatch) {
|
||||
const summary = summaryMatch[1].trim();
|
||||
const location = locationMatch ? locationMatch[1].trim() : null;
|
||||
const startDate = dtstartMatch[1];
|
||||
const endDate = dtendMatch ? dtendMatch[1] : startDate;
|
||||
const url = urlMatch ? urlMatch[1].trim() : null;
|
||||
const uid = uidMatch ? uidMatch[1].trim() : null;
|
||||
|
||||
// Determine event type
|
||||
let type = 'other';
|
||||
if (summary.startsWith('Earnings:')) type = 'earnings';
|
||||
else if (summary.startsWith('IPO')) type = 'ipo';
|
||||
else if (location) type = 'conference';
|
||||
|
||||
// Parse coordinates if location exists
|
||||
const coords = normalizeLocation(location);
|
||||
|
||||
events.push({
|
||||
id: uid,
|
||||
title: summary,
|
||||
type,
|
||||
location: location,
|
||||
coords: coords,
|
||||
startDate: `${startDate.slice(0, 4)}-${startDate.slice(4, 6)}-${startDate.slice(6, 8)}`,
|
||||
endDate: `${endDate.slice(0, 4)}-${endDate.slice(4, 6)}-${endDate.slice(6, 8)}`,
|
||||
url: url,
|
||||
source: 'techmeme',
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return events.sort((a, b) => a.startDate.localeCompare(b.startDate));
|
||||
}
|
||||
|
||||
function parseDevEventsRSS(rssText) {
|
||||
const events = [];
|
||||
|
||||
// Simple regex-based RSS parsing for edge runtime
|
||||
const itemMatches = rssText.matchAll(/<item>([\s\S]*?)<\/item>/g);
|
||||
|
||||
for (const match of itemMatches) {
|
||||
const item = match[1];
|
||||
|
||||
const titleMatch = item.match(/<title><!\[CDATA\[(.*?)\]\]><\/title>|<title>(.*?)<\/title>/);
|
||||
const linkMatch = item.match(/<link>(.*?)<\/link>/);
|
||||
const descMatch = item.match(/<description><!\[CDATA\[(.*?)\]\]><\/description>|<description>(.*?)<\/description>/s);
|
||||
const guidMatch = item.match(/<guid[^>]*>(.*?)<\/guid>/);
|
||||
|
||||
const title = titleMatch ? (titleMatch[1] || titleMatch[2]) : null;
|
||||
const link = linkMatch ? linkMatch[1] : null;
|
||||
const description = descMatch ? (descMatch[1] || descMatch[2]) : '';
|
||||
const guid = guidMatch ? guidMatch[1] : null;
|
||||
|
||||
if (!title) continue;
|
||||
|
||||
// Parse date from description: "EventName is happening on Month Day, Year"
|
||||
const dateMatch = description.match(/on\s+(\w+\s+\d{1,2},?\s+\d{4})/i);
|
||||
let startDate = null;
|
||||
if (dateMatch) {
|
||||
const parsed = new Date(dateMatch[1]);
|
||||
if (!isNaN(parsed.getTime())) {
|
||||
startDate = parsed.toISOString().split('T')[0];
|
||||
}
|
||||
}
|
||||
|
||||
// Parse location from description: various formats
|
||||
let location = null;
|
||||
const locationMatch = description.match(/(?:in|at)\s+([A-Za-z\s]+,\s*[A-Za-z\s]+)(?:\.|$)/i) ||
|
||||
description.match(/Location:\s*([^<\n]+)/i);
|
||||
if (locationMatch) {
|
||||
location = locationMatch[1].trim();
|
||||
}
|
||||
// Check for "Online" events
|
||||
if (description.toLowerCase().includes('online')) {
|
||||
location = 'Online';
|
||||
}
|
||||
|
||||
// Skip events without valid dates or in the past
|
||||
if (!startDate) continue;
|
||||
const eventDate = new Date(startDate);
|
||||
const now = new Date();
|
||||
now.setHours(0, 0, 0, 0);
|
||||
if (eventDate < now) continue;
|
||||
|
||||
const coords = location && location !== 'Online' ? normalizeLocation(location) : null;
|
||||
if (location === 'Online') {
|
||||
// Mark as virtual
|
||||
if (coords) coords.virtual = true;
|
||||
}
|
||||
|
||||
events.push({
|
||||
id: guid || `dev-events-${title.slice(0, 20)}`,
|
||||
title: title,
|
||||
type: 'conference',
|
||||
location: location,
|
||||
coords: coords || (location === 'Online' ? { virtual: true, original: 'Online' } : null),
|
||||
startDate: startDate,
|
||||
endDate: startDate, // RSS doesn't have end date
|
||||
url: link,
|
||||
source: 'dev.events',
|
||||
});
|
||||
}
|
||||
|
||||
return events;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const type = url.searchParams.get('type'); // 'all', 'conferences', 'earnings', 'ipo'
|
||||
const mappable = url.searchParams.get('mappable') === 'true'; // Only return events with coords
|
||||
const limit = parseInt(url.searchParams.get('limit')) || 0; // Max events (0 = unlimited)
|
||||
const days = parseInt(url.searchParams.get('days')) || 0; // Events within N days (0 = unlimited)
|
||||
|
||||
try {
|
||||
// Fetch both sources in parallel
|
||||
const [icsResponse, rssResponse] = await Promise.allSettled([
|
||||
fetch(ICS_URL, {
|
||||
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0)' },
|
||||
}),
|
||||
fetch(DEV_EVENTS_RSS, {
|
||||
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0)' },
|
||||
}),
|
||||
]);
|
||||
|
||||
let events = [];
|
||||
|
||||
// Parse Techmeme ICS
|
||||
if (icsResponse.status === 'fulfilled' && icsResponse.value.ok) {
|
||||
const icsText = await icsResponse.value.text();
|
||||
events.push(...parseICS(icsText));
|
||||
} else {
|
||||
console.warn('Failed to fetch Techmeme ICS');
|
||||
}
|
||||
|
||||
// Parse dev.events RSS
|
||||
if (rssResponse.status === 'fulfilled' && rssResponse.value.ok) {
|
||||
const rssText = await rssResponse.value.text();
|
||||
const devEvents = parseDevEventsRSS(rssText);
|
||||
events.push(...devEvents);
|
||||
} else {
|
||||
console.warn('Failed to fetch dev.events RSS');
|
||||
}
|
||||
|
||||
// Add curated events (major conferences that may fall off limited RSS feeds)
|
||||
const now = new Date();
|
||||
now.setHours(0, 0, 0, 0);
|
||||
for (const curated of CURATED_EVENTS) {
|
||||
const eventDate = new Date(curated.startDate);
|
||||
if (eventDate >= now) {
|
||||
events.push(curated);
|
||||
}
|
||||
}
|
||||
|
||||
// Deduplicate by title similarity (rough match)
|
||||
const seen = new Set();
|
||||
events = events.filter(e => {
|
||||
const key = e.title.toLowerCase().replace(/[^a-z0-9]/g, '').slice(0, 30);
|
||||
if (seen.has(key)) return false;
|
||||
seen.add(key);
|
||||
return true;
|
||||
});
|
||||
|
||||
// Sort by date
|
||||
events.sort((a, b) => a.startDate.localeCompare(b.startDate));
|
||||
|
||||
// Filter by type if specified
|
||||
if (type && type !== 'all') {
|
||||
events = events.filter(e => e.type === type);
|
||||
}
|
||||
|
||||
// Filter to only mappable events if requested
|
||||
if (mappable) {
|
||||
events = events.filter(e => e.coords && !e.coords.virtual);
|
||||
}
|
||||
|
||||
// Filter by time range if specified
|
||||
if (days > 0) {
|
||||
const cutoff = new Date();
|
||||
cutoff.setDate(cutoff.getDate() + days);
|
||||
events = events.filter(e => new Date(e.startDate) <= cutoff);
|
||||
}
|
||||
|
||||
// Apply limit if specified
|
||||
if (limit > 0) {
|
||||
events = events.slice(0, limit);
|
||||
}
|
||||
|
||||
// Add metadata
|
||||
const conferences = events.filter(e => e.type === 'conference');
|
||||
const mappableCount = conferences.filter(e => e.coords && !e.coords.virtual).length;
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
success: true,
|
||||
count: events.length,
|
||||
conferenceCount: conferences.length,
|
||||
mappableCount,
|
||||
lastUpdated: new Date().toISOString(),
|
||||
events,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Tech events error:', error);
|
||||
return new Response(JSON.stringify({
|
||||
success: false,
|
||||
error: error.message,
|
||||
}), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -1,11 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler() {
|
||||
return new Response(JSON.stringify({ error: 'Not found' }), {
|
||||
status: 404,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
},
|
||||
});
|
||||
}
|
||||
+32
-1
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const RELEASES_URL = 'https://api.github.com/repos/koala73/worldmonitor/releases/latest';
|
||||
@@ -9,11 +10,38 @@ const PLATFORM_PATTERNS = {
|
||||
'macos-arm64': (name) => name.endsWith('_aarch64.dmg'),
|
||||
'macos-x64': (name) => name.endsWith('_x64.dmg') && !name.includes('setup'),
|
||||
'linux-appimage': (name) => name.endsWith('_amd64.AppImage'),
|
||||
'linux-appimage-arm64': (name) => name.endsWith('_aarch64.AppImage'),
|
||||
};
|
||||
|
||||
const VARIANT_IDENTIFIERS = {
|
||||
full: ['worldmonitor'],
|
||||
world: ['worldmonitor'],
|
||||
tech: ['techmonitor'],
|
||||
finance: ['financemonitor'],
|
||||
};
|
||||
|
||||
function canonicalAssetName(name) {
|
||||
return String(name || '').toLowerCase().replace(/[^a-z0-9]+/g, '');
|
||||
}
|
||||
|
||||
function findAssetForVariant(assets, variant, platformMatcher) {
|
||||
const identifiers = VARIANT_IDENTIFIERS[variant] ?? null;
|
||||
if (!identifiers) return null;
|
||||
|
||||
return assets.find((asset) => {
|
||||
const assetName = String(asset?.name || '');
|
||||
const normalizedAssetName = canonicalAssetName(assetName);
|
||||
const hasVariantIdentifier = identifiers.some((identifier) =>
|
||||
normalizedAssetName.includes(identifier)
|
||||
);
|
||||
return hasVariantIdentifier && platformMatcher(assetName);
|
||||
}) ?? null;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const url = new URL(req.url);
|
||||
const platform = url.searchParams.get('platform');
|
||||
const variant = (url.searchParams.get('variant') || '').toLowerCase();
|
||||
|
||||
if (!platform || !PLATFORM_PATTERNS[platform]) {
|
||||
return Response.redirect(RELEASES_PAGE, 302);
|
||||
@@ -33,7 +61,10 @@ export default async function handler(req) {
|
||||
|
||||
const release = await res.json();
|
||||
const matcher = PLATFORM_PATTERNS[platform];
|
||||
const asset = release.assets?.find((a) => matcher(a.name));
|
||||
const assets = Array.isArray(release.assets) ? release.assets : [];
|
||||
const asset = variant
|
||||
? findAssetForVariant(assets, variant, matcher)
|
||||
: assets.find((a) => matcher(String(a?.name || '')));
|
||||
|
||||
if (!asset) {
|
||||
return Response.redirect(RELEASES_PAGE, 302);
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const response = await fetch(
|
||||
'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson',
|
||||
{
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,163 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_TTL = 900;
|
||||
let cachedResponse = null;
|
||||
let cacheTimestamp = 0;
|
||||
|
||||
const ETF_LIST = [
|
||||
{ ticker: 'IBIT', issuer: 'BlackRock' },
|
||||
{ ticker: 'FBTC', issuer: 'Fidelity' },
|
||||
{ ticker: 'ARKB', issuer: 'ARK/21Shares' },
|
||||
{ ticker: 'BITB', issuer: 'Bitwise' },
|
||||
{ ticker: 'GBTC', issuer: 'Grayscale' },
|
||||
{ ticker: 'HODL', issuer: 'VanEck' },
|
||||
{ ticker: 'BRRR', issuer: 'Valkyrie' },
|
||||
{ ticker: 'EZBC', issuer: 'Franklin' },
|
||||
{ ticker: 'BTCO', issuer: 'Invesco' },
|
||||
{ ticker: 'BTCW', issuer: 'WisdomTree' },
|
||||
];
|
||||
|
||||
async function fetchChart(ticker) {
|
||||
const url = `https://query1.finance.yahoo.com/v8/finance/chart/${ticker}?range=5d&interval=1d`;
|
||||
const controller = new AbortController();
|
||||
const id = setTimeout(() => controller.abort(), 8000);
|
||||
try {
|
||||
const res = await fetch(url, { signal: controller.signal });
|
||||
if (!res.ok) return null;
|
||||
return await res.json();
|
||||
} catch {
|
||||
return null;
|
||||
} finally {
|
||||
clearTimeout(id);
|
||||
}
|
||||
}
|
||||
|
||||
function parseChartData(chart, ticker, issuer) {
|
||||
try {
|
||||
const result = chart?.chart?.result?.[0];
|
||||
if (!result) return null;
|
||||
|
||||
const quote = result.indicators?.quote?.[0];
|
||||
const closes = quote?.close || [];
|
||||
const volumes = quote?.volume || [];
|
||||
|
||||
const validCloses = closes.filter(p => p != null);
|
||||
const validVolumes = volumes.filter(v => v != null);
|
||||
|
||||
if (validCloses.length < 2) return null;
|
||||
|
||||
const latestPrice = validCloses[validCloses.length - 1];
|
||||
const prevPrice = validCloses[validCloses.length - 2];
|
||||
const priceChange = prevPrice ? ((latestPrice - prevPrice) / prevPrice * 100) : 0;
|
||||
|
||||
const latestVolume = validVolumes.length > 0 ? validVolumes[validVolumes.length - 1] : 0;
|
||||
const avgVolume = validVolumes.length > 1
|
||||
? validVolumes.slice(0, -1).reduce((a, b) => a + b, 0) / (validVolumes.length - 1)
|
||||
: latestVolume;
|
||||
|
||||
// Estimate flow direction from price change + volume
|
||||
const volumeRatio = avgVolume > 0 ? latestVolume / avgVolume : 1;
|
||||
const direction = priceChange > 0.1 ? 'inflow' : priceChange < -0.1 ? 'outflow' : 'neutral';
|
||||
const estFlowMagnitude = latestVolume * latestPrice * (priceChange > 0 ? 1 : -1) * 0.1;
|
||||
|
||||
return {
|
||||
ticker,
|
||||
issuer,
|
||||
price: +latestPrice.toFixed(2),
|
||||
priceChange: +priceChange.toFixed(2),
|
||||
volume: latestVolume,
|
||||
avgVolume: Math.round(avgVolume),
|
||||
volumeRatio: +volumeRatio.toFixed(2),
|
||||
direction,
|
||||
estFlow: Math.round(estFlowMagnitude),
|
||||
};
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function buildFallbackResult() {
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
summary: {
|
||||
etfCount: 0,
|
||||
totalVolume: 0,
|
||||
totalEstFlow: 0,
|
||||
netDirection: 'UNAVAILABLE',
|
||||
inflowCount: 0,
|
||||
outflowCount: 0,
|
||||
},
|
||||
etfs: [],
|
||||
unavailable: true,
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: cors });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
|
||||
return new Response(JSON.stringify(cachedResponse), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=1800` },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const charts = await Promise.allSettled(
|
||||
ETF_LIST.map(etf => fetchChart(etf.ticker))
|
||||
);
|
||||
|
||||
const etfs = [];
|
||||
for (let i = 0; i < ETF_LIST.length; i++) {
|
||||
const chart = charts[i].status === 'fulfilled' ? charts[i].value : null;
|
||||
if (chart) {
|
||||
const parsed = parseChartData(chart, ETF_LIST[i].ticker, ETF_LIST[i].issuer);
|
||||
if (parsed) etfs.push(parsed);
|
||||
}
|
||||
}
|
||||
|
||||
const totalVolume = etfs.reduce((sum, e) => sum + e.volume, 0);
|
||||
const totalEstFlow = etfs.reduce((sum, e) => sum + e.estFlow, 0);
|
||||
const inflowCount = etfs.filter(e => e.direction === 'inflow').length;
|
||||
const outflowCount = etfs.filter(e => e.direction === 'outflow').length;
|
||||
|
||||
const result = {
|
||||
timestamp: new Date().toISOString(),
|
||||
summary: {
|
||||
etfCount: etfs.length,
|
||||
totalVolume,
|
||||
totalEstFlow,
|
||||
netDirection: totalEstFlow > 0 ? 'NET INFLOW' : totalEstFlow < 0 ? 'NET OUTFLOW' : 'NEUTRAL',
|
||||
inflowCount,
|
||||
outflowCount,
|
||||
},
|
||||
etfs: etfs.sort((a, b) => b.volume - a.volume),
|
||||
};
|
||||
|
||||
cachedResponse = result;
|
||||
cacheTimestamp = now;
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=1800` },
|
||||
});
|
||||
} catch (err) {
|
||||
const fallback = cachedResponse || buildFallbackResult();
|
||||
cachedResponse = fallback;
|
||||
cacheTimestamp = now;
|
||||
return new Response(JSON.stringify(fallback), {
|
||||
status: 200,
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,29 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const response = await fetch('https://nasstatus.faa.gov/api/airport-status-information', {
|
||||
headers: { 'Accept': 'application/xml' },
|
||||
});
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/xml',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(`<error>${error.message}</error>`, {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/xml' },
|
||||
});
|
||||
}
|
||||
}
|
||||
-115
@@ -1,115 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const SYMBOL_PATTERN = /^[A-Za-z0-9.^]+$/;
|
||||
const MAX_SYMBOLS = 20;
|
||||
const MAX_SYMBOL_LENGTH = 10;
|
||||
|
||||
function validateSymbols(symbolsParam) {
|
||||
if (!symbolsParam) return null;
|
||||
|
||||
const symbols = symbolsParam
|
||||
.split(',')
|
||||
.map(s => s.trim().toUpperCase())
|
||||
.filter(s => s.length <= MAX_SYMBOL_LENGTH && SYMBOL_PATTERN.test(s))
|
||||
.slice(0, MAX_SYMBOLS);
|
||||
|
||||
return symbols.length > 0 ? symbols : null;
|
||||
}
|
||||
|
||||
async function fetchQuote(symbol, apiKey) {
|
||||
const url = `https://finnhub.io/api/v1/quote?symbol=${encodeURIComponent(symbol)}&token=${apiKey}`;
|
||||
const response = await fetch(url, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return { symbol, error: `HTTP ${response.status}` };
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
// Finnhub returns { c, d, dp, h, l, o, pc, t } where:
|
||||
// c = current price, d = change, dp = percent change
|
||||
// h = high, l = low, o = open, pc = previous close, t = timestamp
|
||||
if (data.c === 0 && data.h === 0 && data.l === 0) {
|
||||
return { symbol, error: 'No data available' };
|
||||
}
|
||||
|
||||
return {
|
||||
symbol,
|
||||
price: data.c,
|
||||
change: data.d,
|
||||
changePercent: data.dp,
|
||||
high: data.h,
|
||||
low: data.l,
|
||||
open: data.o,
|
||||
previousClose: data.pc,
|
||||
timestamp: data.t,
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.FINNHUB_API_KEY;
|
||||
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ quotes: [], skipped: true, reason: 'FINNHUB_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const symbols = validateSymbols(url.searchParams.get('symbols'));
|
||||
|
||||
if (!symbols) {
|
||||
return new Response(JSON.stringify({ error: 'Invalid or missing symbols parameter' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Fetch all quotes in parallel (Finnhub allows 60 req/min on free tier)
|
||||
const quotes = await Promise.all(
|
||||
symbols.map(symbol => fetchQuote(symbol, apiKey))
|
||||
);
|
||||
|
||||
return new Response(JSON.stringify({ quotes }), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,145 +0,0 @@
|
||||
/**
|
||||
* NASA FIRMS Satellite Fire Detection API
|
||||
* Proxies requests to NASA FIRMS to avoid CORS and protect API key
|
||||
* Returns parsed fire data for monitored conflict regions
|
||||
*
|
||||
* GET ?region=Ukraine&days=1 — fires for one region
|
||||
* GET ?days=1 — fires for all monitored regions
|
||||
*/
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const FIRMS_API_KEY = process.env.NASA_FIRMS_API_KEY || process.env.FIRMS_API_KEY || '';
|
||||
const FIRMS_BASE = 'https://firms.modaps.eosdis.nasa.gov/api/area/csv';
|
||||
const SOURCE = 'VIIRS_SNPP_NRT';
|
||||
|
||||
// Bounding boxes as west,south,east,north
|
||||
const MONITORED_REGIONS = {
|
||||
'Ukraine': { bbox: '22,44,40,53' },
|
||||
'Russia': { bbox: '20,50,180,82' },
|
||||
'Iran': { bbox: '44,25,63,40' },
|
||||
'Israel/Gaza': { bbox: '34,29,36,34' },
|
||||
'Syria': { bbox: '35,32,42,37' },
|
||||
'Taiwan': { bbox: '119,21,123,26' },
|
||||
'North Korea': { bbox: '124,37,131,43' },
|
||||
'Saudi Arabia': { bbox: '34,16,56,32' },
|
||||
'Turkey': { bbox: '26,36,45,42' },
|
||||
};
|
||||
|
||||
// Map VIIRS confidence letters to numeric
|
||||
function parseConfidence(c) {
|
||||
if (c === 'h') return 95;
|
||||
if (c === 'n') return 50;
|
||||
if (c === 'l') return 20;
|
||||
return parseInt(c) || 0;
|
||||
}
|
||||
|
||||
function parseCSV(csv) {
|
||||
const lines = csv.trim().split('\n');
|
||||
if (lines.length < 2) return [];
|
||||
|
||||
const headers = lines[0].split(',').map(h => h.trim());
|
||||
const results = [];
|
||||
|
||||
for (let i = 1; i < lines.length; i++) {
|
||||
const vals = lines[i].split(',').map(v => v.trim());
|
||||
if (vals.length < headers.length) continue;
|
||||
|
||||
const row = {};
|
||||
headers.forEach((h, idx) => { row[h] = vals[idx]; });
|
||||
|
||||
results.push({
|
||||
lat: parseFloat(row.latitude),
|
||||
lon: parseFloat(row.longitude),
|
||||
brightness: parseFloat(row.bright_ti4) || 0,
|
||||
scan: parseFloat(row.scan) || 0,
|
||||
track: parseFloat(row.track) || 0,
|
||||
acq_date: row.acq_date || '',
|
||||
acq_time: row.acq_time || '',
|
||||
satellite: row.satellite || '',
|
||||
confidence: parseConfidence(row.confidence),
|
||||
bright_t31: parseFloat(row.bright_ti5) || 0,
|
||||
frp: parseFloat(row.frp) || 0,
|
||||
daynight: row.daynight || '',
|
||||
});
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (request.method === 'OPTIONS') return new Response(null, { status: 204, headers: cors });
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
if (!FIRMS_API_KEY) {
|
||||
return json({ regions: {}, totalCount: 0, skipped: true, reason: 'NASA_FIRMS_API_KEY not configured', source: SOURCE, days: 0, timestamp: new Date().toISOString() });
|
||||
}
|
||||
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const regionName = searchParams.get('region');
|
||||
const days = Math.min(parseInt(searchParams.get('days')) || 1, 5);
|
||||
|
||||
const regions = regionName
|
||||
? { [regionName]: MONITORED_REGIONS[regionName] }
|
||||
: MONITORED_REGIONS;
|
||||
|
||||
if (regionName && !MONITORED_REGIONS[regionName]) {
|
||||
return json({ error: `Unknown region: ${regionName}` }, 400);
|
||||
}
|
||||
|
||||
const allFires = {};
|
||||
let totalCount = 0;
|
||||
|
||||
// Fetch regions in parallel (max 10)
|
||||
const entries = Object.entries(regions);
|
||||
const results = await Promise.allSettled(
|
||||
entries.map(async ([name, { bbox }]) => {
|
||||
const url = `${FIRMS_BASE}/${FIRMS_API_KEY}/${SOURCE}/${bbox}/${days}`;
|
||||
const res = await fetch(url, {
|
||||
headers: { 'Accept': 'text/csv' },
|
||||
});
|
||||
if (!res.ok) throw new Error(`FIRMS ${res.status} for ${name}`);
|
||||
const csv = await res.text();
|
||||
return { name, fires: parseCSV(csv) };
|
||||
})
|
||||
);
|
||||
|
||||
for (const result of results) {
|
||||
if (result.status === 'fulfilled') {
|
||||
const { name, fires } = result.value;
|
||||
allFires[name] = fires;
|
||||
totalCount += fires.length;
|
||||
} else {
|
||||
console.error('[FIRMS]', result.reason?.message);
|
||||
}
|
||||
}
|
||||
|
||||
return json({
|
||||
regions: allFires,
|
||||
totalCount,
|
||||
source: SOURCE,
|
||||
days,
|
||||
timestamp: new Date().toISOString(),
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[FIRMS] Error:', err);
|
||||
return json({ error: 'Failed to fetch fire data' }, 500);
|
||||
}
|
||||
}
|
||||
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', // 10 min cache
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,89 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const seriesId = url.searchParams.get('series_id');
|
||||
const observationStart = url.searchParams.get('observation_start');
|
||||
const observationEnd = url.searchParams.get('observation_end');
|
||||
|
||||
if (!seriesId) {
|
||||
return new Response(JSON.stringify({ error: 'Missing series_id parameter' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.FRED_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({
|
||||
observations: [],
|
||||
skipped: true,
|
||||
reason: 'FRED_API_KEY not configured',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const params = new URLSearchParams({
|
||||
series_id: seriesId,
|
||||
api_key: apiKey,
|
||||
file_type: 'json',
|
||||
sort_order: 'desc',
|
||||
limit: '10',
|
||||
});
|
||||
|
||||
if (observationStart) params.set('observation_start', observationStart);
|
||||
if (observationEnd) params.set('observation_end', observationEnd);
|
||||
|
||||
const fredUrl = `https://api.stlouisfed.org/fred/series/observations?${params}`;
|
||||
const response = await fetch(fredUrl, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
return new Response(JSON.stringify(data), {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: error.message }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const MAX_RECORDS = 20;
|
||||
const DEFAULT_RECORDS = 10;
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const query = url.searchParams.get('query');
|
||||
const maxrecords = Math.min(
|
||||
parseInt(url.searchParams.get('maxrecords') || DEFAULT_RECORDS, 10),
|
||||
MAX_RECORDS
|
||||
);
|
||||
const timespan = url.searchParams.get('timespan') || '72h';
|
||||
|
||||
if (!query || query.length < 2) {
|
||||
return new Response(JSON.stringify({ error: 'Query parameter required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const gdeltUrl = new URL('https://api.gdeltproject.org/api/v2/doc/doc');
|
||||
gdeltUrl.searchParams.set('query', query);
|
||||
gdeltUrl.searchParams.set('mode', 'artlist');
|
||||
gdeltUrl.searchParams.set('maxrecords', maxrecords.toString());
|
||||
gdeltUrl.searchParams.set('format', 'json');
|
||||
gdeltUrl.searchParams.set('sort', 'date');
|
||||
gdeltUrl.searchParams.set('timespan', timespan);
|
||||
|
||||
const response = await fetch(gdeltUrl.toString());
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`GDELT returned ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
const articles = (data.articles || []).map(article => ({
|
||||
title: article.title,
|
||||
url: article.url,
|
||||
source: article.domain || article.source?.domain,
|
||||
date: article.seendate,
|
||||
image: article.socialimage,
|
||||
language: article.language,
|
||||
tone: article.tone,
|
||||
}));
|
||||
|
||||
return new Response(JSON.stringify({ articles, query }), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: error.message, articles: [] }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,86 +0,0 @@
|
||||
// GDELT Geo API proxy with security hardening
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const ALLOWED_FORMATS = ['geojson', 'json', 'csv'];
|
||||
const MAX_RECORDS = 500;
|
||||
const MIN_RECORDS = 1;
|
||||
const ALLOWED_TIMESPANS = ['1d', '7d', '14d', '30d', '60d', '90d'];
|
||||
|
||||
function validateMaxRecords(val) {
|
||||
const num = parseInt(val, 10);
|
||||
if (isNaN(num)) return 250;
|
||||
return Math.max(MIN_RECORDS, Math.min(MAX_RECORDS, num));
|
||||
}
|
||||
|
||||
function validateFormat(val) {
|
||||
return ALLOWED_FORMATS.includes(val) ? val : 'geojson';
|
||||
}
|
||||
|
||||
function validateTimespan(val) {
|
||||
return ALLOWED_TIMESPANS.includes(val) ? val : '7d';
|
||||
}
|
||||
|
||||
function sanitizeQuery(val) {
|
||||
if (!val || typeof val !== 'string') return 'protest';
|
||||
return val.slice(0, 200).replace(/[<>\"']/g, '');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const query = sanitizeQuery(url.searchParams.get('query'));
|
||||
const format = validateFormat(url.searchParams.get('format') || 'geojson');
|
||||
const maxrecords = validateMaxRecords(url.searchParams.get('maxrecords') || '250');
|
||||
const timespan = validateTimespan(url.searchParams.get('timespan') || '7d');
|
||||
|
||||
try {
|
||||
const response = await fetch(
|
||||
`https://api.gdeltproject.org/api/v2/geo/geo?query=${encodeURIComponent(query)}&format=${format}&maxrecords=${maxrecords}×pan=${timespan}`
|
||||
);
|
||||
|
||||
if (!response.ok) {
|
||||
return new Response(JSON.stringify({ error: 'Upstream service unavailable' }), {
|
||||
status: 502,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': format === 'csv' ? 'text/csv' : 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[GDELT] Fetch error:', error.message);
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch GDELT data' }), {
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,90 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
// Fetch trending GitHub repositories
|
||||
// Uses unofficial GitHub trending scraper API
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const language = searchParams.get('language') || 'python'; // python, javascript, typescript, etc.
|
||||
const since = searchParams.get('since') || 'daily'; // daily, weekly, monthly
|
||||
const spoken_language = searchParams.get('spoken_language') || ''; // en, zh, etc.
|
||||
|
||||
// Using GitHub trending API (unofficial)
|
||||
// Alternative: https://gh-trending-api.herokuapp.com/repositories
|
||||
const baseUrl = 'https://api.gitterapp.com/repositories';
|
||||
const queryParams = new URLSearchParams({
|
||||
language: language,
|
||||
since: since,
|
||||
});
|
||||
|
||||
if (spoken_language) {
|
||||
queryParams.append('spoken_language_code', spoken_language);
|
||||
}
|
||||
|
||||
const apiUrl = `${baseUrl}?${queryParams.toString()}`;
|
||||
|
||||
const response = await fetch(apiUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'WorldMonitor/1.0 (Tech Tracker)',
|
||||
},
|
||||
signal: AbortSignal.timeout(10000), // 10 second timeout
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
// Fallback: try alternative API
|
||||
const fallbackUrl = `https://gh-trending-api.herokuapp.com/repositories/${language}?since=${since}`;
|
||||
const fallbackResponse = await fetch(fallbackUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
signal: AbortSignal.timeout(10000),
|
||||
});
|
||||
|
||||
if (!fallbackResponse.ok) {
|
||||
throw new Error(`GitHub trending API returned ${fallbackResponse.status}`);
|
||||
}
|
||||
|
||||
const data = await fallbackResponse.json();
|
||||
return new Response(JSON.stringify(data), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300', // 30 min cache
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
return new Response(JSON.stringify(data), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300', // 30 min cache
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: 'Failed to fetch GitHub trending data',
|
||||
message: error.message
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -1,296 +0,0 @@
|
||||
/**
|
||||
* Groq API Summarization Endpoint with Redis Caching
|
||||
* Uses Llama 3.1 8B Instant for high-throughput summarization
|
||||
* Free tier: 14,400 requests/day (14x more than 70B model)
|
||||
* Server-side Redis cache for cross-user deduplication
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const GROQ_API_URL = 'https://api.groq.com/openai/v1/chat/completions';
|
||||
const MODEL = 'llama-3.1-8b-instant'; // 14.4K RPD vs 1K for 70b
|
||||
const CACHE_TTL_SECONDS = 86400; // 24 hours
|
||||
|
||||
const CACHE_VERSION = 'v3';
|
||||
|
||||
function getCacheKey(headlines, mode, geoContext = '', variant = 'full', lang = 'en') {
|
||||
const sorted = headlines.slice(0, 8).sort().join('|');
|
||||
const geoHash = geoContext ? ':g' + hashString(geoContext).slice(0, 6) : '';
|
||||
const hash = hashString(`${mode}:${sorted}`);
|
||||
const normalizedVariant = typeof variant === 'string' && variant ? variant.toLowerCase() : 'full';
|
||||
const normalizedLang = typeof lang === 'string' && lang ? lang.toLowerCase() : 'en';
|
||||
|
||||
if (mode === 'translate') {
|
||||
const targetLang = normalizedVariant || normalizedLang;
|
||||
return `summary:${CACHE_VERSION}:${mode}:${targetLang}:${hash}${geoHash}`;
|
||||
}
|
||||
|
||||
return `summary:${CACHE_VERSION}:${mode}:${normalizedVariant}:${normalizedLang}:${hash}${geoHash}`;
|
||||
}
|
||||
|
||||
// Deduplicate similar headlines (same story from different sources)
|
||||
function deduplicateHeadlines(headlines) {
|
||||
const seen = new Set();
|
||||
const unique = [];
|
||||
|
||||
for (const headline of headlines) {
|
||||
// Normalize: lowercase, remove punctuation, collapse whitespace
|
||||
const normalized = headline.toLowerCase()
|
||||
.replace(/[^\w\s]/g, '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
|
||||
// Extract key words (4+ chars) for similarity check
|
||||
const words = new Set(normalized.split(' ').filter(w => w.length >= 4));
|
||||
|
||||
// Check if this headline is too similar to any we've seen
|
||||
let isDuplicate = false;
|
||||
for (const seenWords of seen) {
|
||||
const intersection = [...words].filter(w => seenWords.has(w));
|
||||
const similarity = intersection.length / Math.min(words.size, seenWords.size);
|
||||
if (similarity > 0.6) {
|
||||
isDuplicate = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!isDuplicate) {
|
||||
seen.add(words);
|
||||
unique.push(headline);
|
||||
}
|
||||
}
|
||||
|
||||
return unique;
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.GROQ_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ summary: null, fallback: true, skipped: true, reason: 'GROQ_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return new Response(JSON.stringify({ error: 'Payload too large' }), {
|
||||
status: 413,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const { headlines, mode = 'brief', geoContext = '', variant = 'full', lang = 'en' } = await request.json();
|
||||
|
||||
if (!headlines || !Array.isArray(headlines) || headlines.length === 0) {
|
||||
return new Response(JSON.stringify({ error: 'Headlines array required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Check cache first
|
||||
const cacheKey = getCacheKey(headlines, mode, geoContext, variant, lang);
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.summary) {
|
||||
console.log('[Groq] Cache hit:', cacheKey);
|
||||
return new Response(JSON.stringify({
|
||||
summary: cached.summary,
|
||||
model: cached.model || MODEL,
|
||||
provider: 'cache',
|
||||
cached: true,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Deduplicate similar headlines (same story from multiple sources)
|
||||
const uniqueHeadlines = deduplicateHeadlines(headlines.slice(0, 8));
|
||||
const headlineText = uniqueHeadlines.map((h, i) => `${i + 1}. ${h}`).join('\n');
|
||||
|
||||
let systemPrompt, userPrompt;
|
||||
|
||||
// Include intelligence synthesis context in prompt if available
|
||||
const intelSection = geoContext ? `\n\n${geoContext}` : '';
|
||||
|
||||
// Current date context for LLM (models may have outdated knowledge)
|
||||
const isTechVariant = variant === 'tech';
|
||||
const dateContext = `Current date: ${new Date().toISOString().split('T')[0]}.${isTechVariant ? '' : ' Donald Trump is the current US President (second term, inaugurated Jan 2025).'}`;
|
||||
|
||||
// Language instruction
|
||||
const langInstruction = lang && lang !== 'en' ? `\nIMPORTANT: Output the summary in ${lang.toUpperCase()} language.` : '';
|
||||
|
||||
if (mode === 'brief') {
|
||||
if (isTechVariant) {
|
||||
// Tech variant: focus on startups, AI, funding, product launches
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Summarize the key tech/startup development in 2-3 sentences.
|
||||
Rules:
|
||||
- Focus ONLY on technology, startups, AI, funding, product launches, or developer news
|
||||
- IGNORE political news, trade policy, tariffs, government actions unless directly about tech regulation
|
||||
- Lead with the company/product/technology name
|
||||
- Start directly: "OpenAI announced...", "A new $50M Series B...", "GitHub released..."
|
||||
- No bullet points, no meta-commentary${langInstruction}`;
|
||||
} else {
|
||||
// Full variant: geopolitical focus
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Summarize the key development in 2-3 sentences.
|
||||
Rules:
|
||||
- Lead with WHAT happened and WHERE - be specific
|
||||
- NEVER start with "Breaking news", "Good evening", "Tonight", or TV-style openings
|
||||
- Start directly with the subject: "Iran's regime...", "The US Treasury...", "Protests in..."
|
||||
- CRITICAL FOCAL POINTS are the main actors - mention them by name
|
||||
- If focal points show news + signals convergence, that's the lead
|
||||
- No bullet points, no meta-commentary${langInstruction}`;
|
||||
}
|
||||
userPrompt = `Summarize the top story:\n${headlineText}${intelSection}`;
|
||||
} else if (mode === 'analysis') {
|
||||
if (isTechVariant) {
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Analyze the tech/startup trend in 2-3 sentences.
|
||||
Rules:
|
||||
- Focus ONLY on technology implications: funding trends, AI developments, market shifts, product strategy
|
||||
- IGNORE political implications, trade wars, government unless directly about tech policy
|
||||
- Lead with the insight for tech industry
|
||||
- Connect to startup ecosystem, VC trends, or technical implications`;
|
||||
} else {
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Provide analysis in 2-3 sentences. Be direct and specific.
|
||||
Rules:
|
||||
- Lead with the insight - what's significant and why
|
||||
- NEVER start with "Breaking news", "Tonight", "The key/dominant narrative is"
|
||||
- Start with substance: "Iran faces...", "The escalation in...", "Multiple signals suggest..."
|
||||
- CRITICAL FOCAL POINTS are your main actors - explain WHY they matter
|
||||
- If focal points show news-signal correlation, flag as escalation
|
||||
- Connect dots, be specific about implications`;
|
||||
}
|
||||
userPrompt = isTechVariant
|
||||
? `What's the key tech trend or development?\n${headlineText}${intelSection}`
|
||||
: `What's the key pattern or risk?\n${headlineText}${intelSection}`;
|
||||
} else if (mode === 'translate') {
|
||||
const targetLang = variant; // In translate mode, variant param holds the target language code (e.g., 'fr', 'es')
|
||||
systemPrompt = `You are a professional news translator. Translate the following news headlines/summaries into ${targetLang}.
|
||||
Rules:
|
||||
- Maintain the original tone and journalistic style.
|
||||
- Do NOT add any conversational filler (e.g., "Here is the translation").
|
||||
- Output ONLY the translated text.
|
||||
- If the text is already in ${targetLang}, return it as is.`;
|
||||
userPrompt = `Translate to ${targetLang}:\n${headlines[0]}`;
|
||||
} else {
|
||||
systemPrompt = isTechVariant
|
||||
? `${dateContext}\n\nSynthesize tech news in 2 sentences. Focus on startups, AI, funding, products. Ignore politics unless directly about tech regulation.${langInstruction}`
|
||||
: `${dateContext}\n\nSynthesize in 2 sentences max. Lead with substance. NEVER start with "Breaking news" or "Tonight" - just state the insight directly. CRITICAL focal points with news-signal convergence are significant.${langInstruction}`;
|
||||
userPrompt = `Key takeaway:\n${headlineText}${intelSection}`;
|
||||
}
|
||||
|
||||
const response = await fetch(GROQ_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
temperature: 0.3,
|
||||
max_tokens: 150,
|
||||
top_p: 0.9,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.error('[Groq] API error:', response.status, errorText);
|
||||
|
||||
// Return fallback signal for rate limiting
|
||||
if (response.status === 429) {
|
||||
return new Response(JSON.stringify({ error: 'Rate limited', fallback: true }), {
|
||||
status: 429,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({ error: 'Groq API error', fallback: true }), {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const summary = data.choices?.[0]?.message?.content?.trim();
|
||||
|
||||
if (!summary) {
|
||||
return new Response(JSON.stringify({ error: 'Empty response', fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Store in cache
|
||||
await setCachedJson(cacheKey, {
|
||||
summary,
|
||||
model: MODEL,
|
||||
timestamp: Date.now(),
|
||||
}, CACHE_TTL_SECONDS);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
summary,
|
||||
model: MODEL,
|
||||
provider: 'groq',
|
||||
cached: false,
|
||||
tokens: data.usage?.total_tokens || 0,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
|
||||
},
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('[Groq] Error:', error.name, error.message, error.stack?.split('\n')[1]);
|
||||
return new Response(JSON.stringify({
|
||||
error: error.message,
|
||||
errorType: error.name,
|
||||
fallback: true
|
||||
}), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,98 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
// Fetch Hacker News front page stories
|
||||
// Uses official HackerNews Firebase API
|
||||
const ALLOWED_STORY_TYPES = new Set(['top', 'new', 'best', 'ask', 'show', 'job']);
|
||||
const DEFAULT_LIMIT = 30;
|
||||
const MAX_LIMIT = 60;
|
||||
const MAX_CONCURRENCY = 10;
|
||||
|
||||
function parseLimit(rawLimit) {
|
||||
const parsed = Number.parseInt(rawLimit || '', 10);
|
||||
if (!Number.isFinite(parsed)) return DEFAULT_LIMIT;
|
||||
return Math.max(1, Math.min(MAX_LIMIT, parsed));
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const requestedType = searchParams.get('type') || 'top';
|
||||
const storyType = ALLOWED_STORY_TYPES.has(requestedType) ? requestedType : 'top';
|
||||
const limit = parseLimit(searchParams.get('limit'));
|
||||
|
||||
// HackerNews official Firebase API
|
||||
const storiesUrl = `https://hacker-news.firebaseio.com/v0/${storyType}stories.json`;
|
||||
|
||||
// Fetch story IDs
|
||||
const storiesResponse = await fetch(storiesUrl, {
|
||||
signal: AbortSignal.timeout(10000),
|
||||
});
|
||||
|
||||
if (!storiesResponse.ok) {
|
||||
throw new Error(`HackerNews API returned ${storiesResponse.status}`);
|
||||
}
|
||||
|
||||
const storyIds = await storiesResponse.json();
|
||||
if (!Array.isArray(storyIds)) {
|
||||
throw new Error('HackerNews API returned unexpected payload');
|
||||
}
|
||||
const limitedIds = storyIds.slice(0, limit);
|
||||
|
||||
// Fetch story details in bounded batches to avoid unbounded fan-out.
|
||||
const stories = [];
|
||||
for (let i = 0; i < limitedIds.length; i += MAX_CONCURRENCY) {
|
||||
const batchIds = limitedIds.slice(i, i + MAX_CONCURRENCY);
|
||||
const storyPromises = batchIds.map(async (id) => {
|
||||
const storyUrl = `https://hacker-news.firebaseio.com/v0/item/${id}.json`;
|
||||
try {
|
||||
const response = await fetch(storyUrl, {
|
||||
signal: AbortSignal.timeout(5000),
|
||||
});
|
||||
if (response.ok) {
|
||||
return await response.json();
|
||||
}
|
||||
return null;
|
||||
} catch (error) {
|
||||
console.error(`Failed to fetch story ${id}:`, error);
|
||||
return null;
|
||||
}
|
||||
});
|
||||
const batchResults = await Promise.all(storyPromises);
|
||||
stories.push(...batchResults.filter((story) => story !== null));
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
type: storyType,
|
||||
stories: stories,
|
||||
total: stories.length,
|
||||
timestamp: new Date().toISOString()
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', // 5 min cache
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: 'Failed to fetch Hacker News data',
|
||||
message: error.message
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
},
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
-148
@@ -1,148 +0,0 @@
|
||||
// HDX HAPI (Humanitarian API) proxy
|
||||
// Returns aggregated conflict event counts per country
|
||||
// Source: ACLED data aggregated monthly by HDX
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_KEY = 'hapi:conflict-events:v2';
|
||||
const CACHE_TTL_SECONDS = 6 * 60 * 60; // 6 hours
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const RESPONSE_CACHE_CONTROL = 'public, max-age=1800';
|
||||
|
||||
// In-memory fallback when Redis is unavailable.
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(
|
||||
data &&
|
||||
typeof data === 'object' &&
|
||||
Array.isArray(data.countries)
|
||||
);
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/hapi', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/hapi', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const appId = btoa('worldmonitor:monitor@worldmonitor.app');
|
||||
const response = await fetch(
|
||||
`https://hapi.humdata.org/api/v2/coordination-context/conflict-events?output_format=json&limit=1000&offset=0&app_identifier=${appId}`,
|
||||
{
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`HAPI API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const rawData = await response.json();
|
||||
const records = rawData.data || [];
|
||||
|
||||
// Each record is (country, event_type, month) — aggregate across event types per country
|
||||
// Keep only the most recent month per country
|
||||
const byCountry = {};
|
||||
for (const r of records) {
|
||||
const iso3 = r.location_code || '';
|
||||
if (!iso3) continue;
|
||||
|
||||
const month = r.reference_period_start || '';
|
||||
const eventType = (r.event_type || '').toLowerCase();
|
||||
const events = r.events || 0;
|
||||
const fatalities = r.fatalities || 0;
|
||||
|
||||
if (!byCountry[iso3]) {
|
||||
byCountry[iso3] = { iso3, locationName: r.location_name || '', month, eventsTotal: 0, eventsPoliticalViolence: 0, eventsCivilianTargeting: 0, eventsDemonstrations: 0, fatalitiesTotalPoliticalViolence: 0, fatalitiesTotalCivilianTargeting: 0 };
|
||||
}
|
||||
|
||||
const c = byCountry[iso3];
|
||||
if (month > c.month) {
|
||||
// Newer month — reset
|
||||
c.month = month;
|
||||
c.eventsTotal = 0; c.eventsPoliticalViolence = 0; c.eventsCivilianTargeting = 0; c.eventsDemonstrations = 0; c.fatalitiesTotalPoliticalViolence = 0; c.fatalitiesTotalCivilianTargeting = 0;
|
||||
}
|
||||
if (month === c.month) {
|
||||
c.eventsTotal += events;
|
||||
if (eventType.includes('political_violence')) { c.eventsPoliticalViolence += events; c.fatalitiesTotalPoliticalViolence += fatalities; }
|
||||
if (eventType.includes('civilian_targeting')) { c.eventsCivilianTargeting += events; c.fatalitiesTotalCivilianTargeting += fatalities; }
|
||||
if (eventType.includes('demonstration')) { c.eventsDemonstrations += events; }
|
||||
}
|
||||
}
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: Object.keys(byCountry).length,
|
||||
countries: Object.values(byCountry),
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/hapi', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'MISS',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/hapi', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'X-Cache': 'STALE',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/hapi', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, countries: [] }, {
|
||||
status: 500,
|
||||
headers: { ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
import { strict as assert } from 'node:assert';
|
||||
import test from 'node:test';
|
||||
import { XMLLoader } from '@loaders.gl/xml';
|
||||
import { WMSCapabilitiesLoader, WMSErrorLoader, _WMSFeatureInfoLoader } from '@loaders.gl/wms';
|
||||
|
||||
const WMS_CAPABILITIES_XML = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<WMS_Capabilities version="1.3.0">
|
||||
<Service>
|
||||
<Name>WMS</Name>
|
||||
<Title>Test Service</Title>
|
||||
<KeywordList>
|
||||
<Keyword>alerts</Keyword>
|
||||
<Keyword>world</Keyword>
|
||||
</KeywordList>
|
||||
</Service>
|
||||
<Capability>
|
||||
<Request>
|
||||
<GetMap>
|
||||
<Format>image/png</Format>
|
||||
<Format>image/jpeg</Format>
|
||||
</GetMap>
|
||||
</Request>
|
||||
<Exception>
|
||||
<Format>application/vnd.ogc.se_xml</Format>
|
||||
</Exception>
|
||||
<Layer>
|
||||
<Title>Root Layer</Title>
|
||||
<CRS>EPSG:4326</CRS>
|
||||
<EX_GeographicBoundingBox>
|
||||
<westBoundLongitude>-180</westBoundLongitude>
|
||||
<eastBoundLongitude>180</eastBoundLongitude>
|
||||
<southBoundLatitude>-90</southBoundLatitude>
|
||||
<northBoundLatitude>90</northBoundLatitude>
|
||||
</EX_GeographicBoundingBox>
|
||||
<Layer queryable="1">
|
||||
<Name>alerts</Name>
|
||||
<Title>Alerts</Title>
|
||||
<BoundingBox CRS="EPSG:4326" minx="-10" miny="-20" maxx="30" maxy="40" />
|
||||
<Dimension name="time" units="ISO8601" default="2024-01-01" nearestValue="1">
|
||||
2024-01-01/2024-12-31/P1D
|
||||
</Dimension>
|
||||
</Layer>
|
||||
</Layer>
|
||||
</Capability>
|
||||
</WMS_Capabilities>`;
|
||||
|
||||
test('XMLLoader keeps namespace stripping + array paths stable', () => {
|
||||
const xml = '<root><ns:Child attr="x">ok</ns:Child><ns:Child attr="y">yo</ns:Child></root>';
|
||||
const parsed = XMLLoader.parseTextSync(xml, {
|
||||
xml: {
|
||||
removeNSPrefix: true,
|
||||
arrayPaths: ['root.Child'],
|
||||
},
|
||||
});
|
||||
|
||||
assert.deepEqual(parsed, {
|
||||
root: {
|
||||
Child: [
|
||||
{ value: 'ok', attr: 'x' },
|
||||
{ value: 'yo', attr: 'y' },
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test('WMSCapabilitiesLoader parses core typed fields from XML capabilities', () => {
|
||||
const parsed = WMSCapabilitiesLoader.parseTextSync(WMS_CAPABILITIES_XML);
|
||||
|
||||
assert.equal(parsed.version, '1.3.0');
|
||||
assert.equal(parsed.name, 'WMS');
|
||||
assert.deepEqual(parsed.requests.GetMap.mimeTypes, ['image/png', 'image/jpeg']);
|
||||
|
||||
assert.equal(parsed.layers.length, 1);
|
||||
const rootLayer = parsed.layers[0];
|
||||
assert.deepEqual(rootLayer.geographicBoundingBox, [[-180, -90], [180, 90]]);
|
||||
|
||||
const alertsLayer = rootLayer.layers[0];
|
||||
assert.equal(alertsLayer.name, 'alerts');
|
||||
assert.equal(alertsLayer.queryable, true);
|
||||
assert.deepEqual(alertsLayer.boundingBoxes[0], {
|
||||
crs: 'EPSG:4326',
|
||||
boundingBox: [[-10, -20], [30, 40]],
|
||||
});
|
||||
assert.deepEqual(alertsLayer.dimensions[0], {
|
||||
name: 'time',
|
||||
units: 'ISO8601',
|
||||
extent: '2024-01-01/2024-12-31/P1D',
|
||||
defaultValue: '2024-01-01',
|
||||
nearestValue: true,
|
||||
});
|
||||
});
|
||||
|
||||
test('WMSErrorLoader extracts namespaced error text and honors throw options', () => {
|
||||
const namespacedErrorXml =
|
||||
'<?xml version="1.0"?><ogc:ServiceExceptionReport><ogc:ServiceException code="LayerNotDefined">Bad layer</ogc:ServiceException></ogc:ServiceExceptionReport>';
|
||||
|
||||
const defaultMessage = WMSErrorLoader.parseTextSync(namespacedErrorXml);
|
||||
assert.equal(defaultMessage, 'WMS Service error: Bad layer');
|
||||
|
||||
const minimalMessage = WMSErrorLoader.parseTextSync(namespacedErrorXml, {
|
||||
wms: { minimalErrors: true },
|
||||
});
|
||||
assert.equal(minimalMessage, 'Bad layer');
|
||||
|
||||
assert.throws(
|
||||
() => WMSErrorLoader.parseTextSync(namespacedErrorXml, { wms: { throwOnError: true } }),
|
||||
/WMS Service error: Bad layer/
|
||||
);
|
||||
});
|
||||
|
||||
test('WMS feature info parsing remains stable for single and repeated FIELDS nodes', () => {
|
||||
const singleFieldsXml = '<?xml version="1.0"?><FeatureInfoResponse><FIELDS id="1" label="one"/></FeatureInfoResponse>';
|
||||
const manyFieldsXml = '<?xml version="1.0"?><FeatureInfoResponse><FIELDS id="1"/><FIELDS id="2"/></FeatureInfoResponse>';
|
||||
|
||||
const single = _WMSFeatureInfoLoader.parseTextSync(singleFieldsXml);
|
||||
const many = _WMSFeatureInfoLoader.parseTextSync(manyFieldsXml);
|
||||
|
||||
assert.equal(single.features.length, 1);
|
||||
assert.deepEqual(single.features[0]?.attributes, { id: '1', label: 'one' });
|
||||
assert.equal(many.features.length, 2);
|
||||
assert.equal(many.features[0]?.attributes?.id, '1');
|
||||
assert.equal(many.features[1]?.attributes?.id, '2');
|
||||
});
|
||||
@@ -1,284 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_TTL = 300;
|
||||
let cachedResponse = null;
|
||||
let cacheTimestamp = 0;
|
||||
|
||||
async function fetchJSON(url, timeout = 8000) {
|
||||
const controller = new AbortController();
|
||||
const id = setTimeout(() => controller.abort(), timeout);
|
||||
try {
|
||||
const res = await fetch(url, { signal: controller.signal });
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
return await res.json();
|
||||
} finally {
|
||||
clearTimeout(id);
|
||||
}
|
||||
}
|
||||
|
||||
function rateOfChange(prices, days) {
|
||||
if (!prices || prices.length < days + 1) return null;
|
||||
const recent = prices[prices.length - 1];
|
||||
const past = prices[prices.length - 1 - days];
|
||||
if (!past || past === 0) return null;
|
||||
return ((recent - past) / past) * 100;
|
||||
}
|
||||
|
||||
function sma(prices, period) {
|
||||
if (!prices || prices.length < period) return null;
|
||||
const slice = prices.slice(-period);
|
||||
return slice.reduce((a, b) => a + b, 0) / period;
|
||||
}
|
||||
|
||||
function extractClosePrices(chart) {
|
||||
try {
|
||||
const result = chart?.chart?.result?.[0];
|
||||
return result?.indicators?.quote?.[0]?.close?.filter(p => p != null) || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function extractVolumes(chart) {
|
||||
try {
|
||||
const result = chart?.chart?.result?.[0];
|
||||
return result?.indicators?.quote?.[0]?.volume?.filter(v => v != null) || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function extractAlignedPriceVolume(chart) {
|
||||
try {
|
||||
const result = chart?.chart?.result?.[0];
|
||||
const closes = result?.indicators?.quote?.[0]?.close || [];
|
||||
const volumes = result?.indicators?.quote?.[0]?.volume || [];
|
||||
const pairs = [];
|
||||
for (let i = 0; i < closes.length; i++) {
|
||||
if (closes[i] != null && volumes[i] != null) {
|
||||
pairs.push({ price: closes[i], volume: volumes[i] });
|
||||
}
|
||||
}
|
||||
return pairs;
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function buildFallbackResult() {
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
verdict: 'UNKNOWN',
|
||||
bullishCount: 0,
|
||||
totalCount: 0,
|
||||
signals: {
|
||||
liquidity: { status: 'UNKNOWN', value: null, sparkline: [] },
|
||||
flowStructure: { status: 'UNKNOWN', btcReturn5: null, qqqReturn5: null },
|
||||
macroRegime: { status: 'UNKNOWN', qqqRoc20: null, xlpRoc20: null },
|
||||
technicalTrend: {
|
||||
status: 'UNKNOWN',
|
||||
btcPrice: null,
|
||||
sma50: null,
|
||||
sma200: null,
|
||||
vwap30d: null,
|
||||
mayerMultiple: null,
|
||||
sparkline: [],
|
||||
},
|
||||
hashRate: { status: 'UNKNOWN', change30d: null },
|
||||
miningCost: { status: 'UNKNOWN' },
|
||||
fearGreed: { status: 'UNKNOWN', value: null, history: [] },
|
||||
},
|
||||
meta: { qqqSparkline: [] },
|
||||
unavailable: true,
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: cors });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
|
||||
return new Response(JSON.stringify(cachedResponse), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=600` },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const yahooBase = 'https://query1.finance.yahoo.com/v8/finance/chart';
|
||||
const [jpyChart, btcChart, qqqChart, xlpChart, fearGreed, mempoolHash] = await Promise.allSettled([
|
||||
fetchJSON(`${yahooBase}/JPY=X?range=1y&interval=1d`),
|
||||
fetchJSON(`${yahooBase}/BTC-USD?range=1y&interval=1d`),
|
||||
fetchJSON(`${yahooBase}/QQQ?range=1y&interval=1d`),
|
||||
fetchJSON(`${yahooBase}/XLP?range=1y&interval=1d`),
|
||||
fetchJSON('https://api.alternative.me/fng/?limit=30&format=json'),
|
||||
fetchJSON('https://mempool.space/api/v1/mining/hashrate/1m'),
|
||||
]);
|
||||
|
||||
const jpyPrices = jpyChart.status === 'fulfilled' ? extractClosePrices(jpyChart.value) : [];
|
||||
const btcPrices = btcChart.status === 'fulfilled' ? extractClosePrices(btcChart.value) : [];
|
||||
const btcVolumes = btcChart.status === 'fulfilled' ? extractVolumes(btcChart.value) : [];
|
||||
const btcAligned = btcChart.status === 'fulfilled' ? extractAlignedPriceVolume(btcChart.value) : [];
|
||||
const qqqPrices = qqqChart.status === 'fulfilled' ? extractClosePrices(qqqChart.value) : [];
|
||||
const xlpPrices = xlpChart.status === 'fulfilled' ? extractClosePrices(xlpChart.value) : [];
|
||||
|
||||
// 1. Liquidity Signal (JPY 30d ROC)
|
||||
const jpyRoc30 = rateOfChange(jpyPrices, 30);
|
||||
const liquidityStatus = jpyRoc30 !== null
|
||||
? (jpyRoc30 < -2 ? 'SQUEEZE' : 'NORMAL')
|
||||
: 'UNKNOWN';
|
||||
|
||||
// 2. Flow Structure (BTC vs QQQ 5d return)
|
||||
const btcReturn5 = rateOfChange(btcPrices, 5);
|
||||
const qqqReturn5 = rateOfChange(qqqPrices, 5);
|
||||
let flowStatus = 'UNKNOWN';
|
||||
if (btcReturn5 !== null && qqqReturn5 !== null) {
|
||||
const gap = btcReturn5 - qqqReturn5;
|
||||
flowStatus = Math.abs(gap) > 5 ? 'PASSIVE GAP' : 'ALIGNED';
|
||||
}
|
||||
|
||||
// 3. Macro Regime (QQQ/XLP 20d ROC)
|
||||
const qqqRoc20 = rateOfChange(qqqPrices, 20);
|
||||
const xlpRoc20 = rateOfChange(xlpPrices, 20);
|
||||
let regimeStatus = 'UNKNOWN';
|
||||
if (qqqRoc20 !== null && xlpRoc20 !== null) {
|
||||
regimeStatus = qqqRoc20 > xlpRoc20 ? 'RISK-ON' : 'DEFENSIVE';
|
||||
}
|
||||
|
||||
// 4. Technical Trend (BTC vs SMA50 + VWAP)
|
||||
const btcSma50 = sma(btcPrices, 50);
|
||||
const btcSma200 = sma(btcPrices, 200);
|
||||
const btcCurrent = btcPrices.length > 0 ? btcPrices[btcPrices.length - 1] : null;
|
||||
|
||||
// Compute VWAP from aligned price/volume pairs (30d)
|
||||
let btcVwap = null;
|
||||
if (btcAligned.length >= 30) {
|
||||
const last30 = btcAligned.slice(-30);
|
||||
let sumPV = 0, sumV = 0;
|
||||
for (const { price, volume } of last30) {
|
||||
sumPV += price * volume;
|
||||
sumV += volume;
|
||||
}
|
||||
if (sumV > 0) btcVwap = +(sumPV / sumV).toFixed(0);
|
||||
}
|
||||
|
||||
let trendStatus = 'UNKNOWN';
|
||||
let mayerMultiple = null;
|
||||
if (btcCurrent && btcSma50) {
|
||||
const aboveSma = btcCurrent > btcSma50 * 1.02;
|
||||
const belowSma = btcCurrent < btcSma50 * 0.98;
|
||||
const aboveVwap = btcVwap ? btcCurrent > btcVwap : null;
|
||||
if (aboveSma && aboveVwap !== false) trendStatus = 'BULLISH';
|
||||
else if (belowSma && aboveVwap !== true) trendStatus = 'BEARISH';
|
||||
else trendStatus = 'NEUTRAL';
|
||||
}
|
||||
if (btcCurrent && btcSma200) {
|
||||
mayerMultiple = +(btcCurrent / btcSma200).toFixed(2);
|
||||
}
|
||||
|
||||
// 5. Hash Rate
|
||||
let hashStatus = 'UNKNOWN';
|
||||
let hashChange = null;
|
||||
if (mempoolHash.status === 'fulfilled') {
|
||||
const hr = mempoolHash.value?.hashrates || mempoolHash.value;
|
||||
if (Array.isArray(hr) && hr.length >= 2) {
|
||||
const recent = hr[hr.length - 1]?.avgHashrate || hr[hr.length - 1];
|
||||
const older = hr[0]?.avgHashrate || hr[0];
|
||||
if (recent && older && older > 0) {
|
||||
hashChange = +((recent - older) / older * 100).toFixed(1);
|
||||
hashStatus = hashChange > 3 ? 'GROWING' : hashChange < -3 ? 'DECLINING' : 'STABLE';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 6. Mining Cost (hashrate-based model)
|
||||
let miningStatus = 'UNKNOWN';
|
||||
if (btcCurrent && hashChange !== null) {
|
||||
miningStatus = btcCurrent > 60000 ? 'PROFITABLE' : btcCurrent > 40000 ? 'TIGHT' : 'SQUEEZE';
|
||||
}
|
||||
|
||||
// 7. Fear & Greed
|
||||
let fgValue = null;
|
||||
let fgLabel = 'UNKNOWN';
|
||||
let fgHistory = [];
|
||||
if (fearGreed.status === 'fulfilled' && fearGreed.value?.data) {
|
||||
const data = fearGreed.value.data;
|
||||
const parsed = parseInt(data[0]?.value, 10);
|
||||
fgValue = Number.isFinite(parsed) ? parsed : null;
|
||||
fgLabel = data[0]?.value_classification || 'UNKNOWN';
|
||||
fgHistory = data.slice(0, 30).map(d => ({
|
||||
value: parseInt(d.value, 10),
|
||||
date: new Date(parseInt(d.timestamp, 10) * 1000).toISOString().slice(0, 10),
|
||||
})).reverse();
|
||||
}
|
||||
|
||||
// Sparkline data
|
||||
const btcSparkline = btcPrices.slice(-30);
|
||||
const qqqSparkline = qqqPrices.slice(-30);
|
||||
const jpySparkline = jpyPrices.slice(-30);
|
||||
|
||||
// Overall Verdict
|
||||
let bullishCount = 0;
|
||||
let totalCount = 0;
|
||||
const signals = [
|
||||
{ name: 'Liquidity', status: liquidityStatus, bullish: liquidityStatus === 'NORMAL' },
|
||||
{ name: 'Flow Structure', status: flowStatus, bullish: flowStatus === 'ALIGNED' },
|
||||
{ name: 'Macro Regime', status: regimeStatus, bullish: regimeStatus === 'RISK-ON' },
|
||||
{ name: 'Technical Trend', status: trendStatus, bullish: trendStatus === 'BULLISH' },
|
||||
{ name: 'Hash Rate', status: hashStatus, bullish: hashStatus === 'GROWING' },
|
||||
{ name: 'Mining Cost', status: miningStatus, bullish: miningStatus === 'PROFITABLE' },
|
||||
{ name: 'Fear & Greed', status: fgLabel, bullish: fgValue !== null && fgValue > 50 },
|
||||
];
|
||||
|
||||
for (const s of signals) {
|
||||
if (s.status !== 'UNKNOWN') {
|
||||
totalCount++;
|
||||
if (s.bullish) bullishCount++;
|
||||
}
|
||||
}
|
||||
|
||||
const verdict = totalCount === 0 ? 'UNKNOWN' : (bullishCount / totalCount >= 0.57 ? 'BUY' : 'CASH');
|
||||
|
||||
const result = {
|
||||
timestamp: new Date().toISOString(),
|
||||
verdict,
|
||||
bullishCount,
|
||||
totalCount,
|
||||
signals: {
|
||||
liquidity: { status: liquidityStatus, value: jpyRoc30 !== null ? +jpyRoc30.toFixed(2) : null, sparkline: jpySparkline },
|
||||
flowStructure: { status: flowStatus, btcReturn5: btcReturn5 !== null ? +btcReturn5.toFixed(2) : null, qqqReturn5: qqqReturn5 !== null ? +qqqReturn5.toFixed(2) : null },
|
||||
macroRegime: { status: regimeStatus, qqqRoc20: qqqRoc20 !== null ? +qqqRoc20.toFixed(2) : null, xlpRoc20: xlpRoc20 !== null ? +xlpRoc20.toFixed(2) : null },
|
||||
technicalTrend: { status: trendStatus, btcPrice: btcCurrent, sma50: btcSma50 ? +btcSma50.toFixed(0) : null, sma200: btcSma200 ? +btcSma200.toFixed(0) : null, vwap30d: btcVwap, mayerMultiple, sparkline: btcSparkline },
|
||||
hashRate: { status: hashStatus, change30d: hashChange },
|
||||
miningCost: { status: miningStatus },
|
||||
fearGreed: { status: fgLabel, value: fgValue, history: fgHistory },
|
||||
},
|
||||
meta: { qqqSparkline },
|
||||
};
|
||||
|
||||
cachedResponse = result;
|
||||
cacheTimestamp = now;
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=600` },
|
||||
});
|
||||
} catch (err) {
|
||||
const fallback = cachedResponse || buildFallbackResult();
|
||||
cachedResponse = fallback;
|
||||
cacheTimestamp = now;
|
||||
return new Response(JSON.stringify(fallback), {
|
||||
status: 200,
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const response = await fetch(
|
||||
'https://msi.nga.mil/api/publications/broadcast-warn?output=json&status=A'
|
||||
);
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json', ...cors, 'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60' },
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: error.message }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
+7
-1
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
/**
|
||||
* Dynamic OG Image Generator for Story Sharing
|
||||
* Returns an SVG image (1200x630) — rich intelligence card for social previews.
|
||||
@@ -24,12 +25,17 @@ const LEVEL_LABELS = {
|
||||
low: 'LOW RISK',
|
||||
};
|
||||
|
||||
function normalizeLevel(rawLevel) {
|
||||
const level = String(rawLevel || '').toLowerCase();
|
||||
return Object.prototype.hasOwnProperty.call(LEVEL_COLORS, level) ? level : 'normal';
|
||||
}
|
||||
|
||||
export default function handler(req, res) {
|
||||
const url = new URL(req.url, `https://${req.headers.host}`);
|
||||
const countryCode = (url.searchParams.get('c') || '').toUpperCase();
|
||||
const type = url.searchParams.get('t') || 'ciianalysis';
|
||||
const score = url.searchParams.get('s');
|
||||
const level = url.searchParams.get('l') || 'normal';
|
||||
const level = normalizeLevel(url.searchParams.get('l'));
|
||||
|
||||
const countryName = COUNTRY_NAMES[countryCode] || countryCode || 'Global';
|
||||
const levelColor = LEVEL_COLORS[level] || '#eab308';
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
import { strict as assert } from 'node:assert';
|
||||
import test from 'node:test';
|
||||
import handler from './og-story.js';
|
||||
|
||||
function renderOgStory(query = '') {
|
||||
const req = {
|
||||
url: `https://worldmonitor.app/api/og-story${query ? `?${query}` : ''}`,
|
||||
headers: { host: 'worldmonitor.app' },
|
||||
};
|
||||
|
||||
let statusCode = 0;
|
||||
let body = '';
|
||||
const headers = {};
|
||||
|
||||
const res = {
|
||||
setHeader(name, value) {
|
||||
headers[String(name).toLowerCase()] = String(value);
|
||||
},
|
||||
status(code) {
|
||||
statusCode = code;
|
||||
return this;
|
||||
},
|
||||
send(payload) {
|
||||
body = String(payload);
|
||||
},
|
||||
};
|
||||
|
||||
handler(req, res);
|
||||
return { statusCode, body, headers };
|
||||
}
|
||||
|
||||
test('normalizes unsupported level values to prevent SVG script injection', () => {
|
||||
const injectedLevel = encodeURIComponent('</text><script>alert(1)</script><text>');
|
||||
const response = renderOgStory(`c=US&s=50&l=${injectedLevel}`);
|
||||
|
||||
assert.equal(response.statusCode, 200);
|
||||
assert.equal(/<script/i.test(response.body), false);
|
||||
assert.match(response.body, />NORMAL<\/text>/);
|
||||
});
|
||||
|
||||
test('uses a known level when it is allowlisted', () => {
|
||||
const response = renderOgStory('c=US&s=88&l=critical');
|
||||
|
||||
assert.equal(response.statusCode, 200);
|
||||
assert.match(response.body, />CRITICAL<\/text>/);
|
||||
assert.match(response.body, /#ef4444/);
|
||||
});
|
||||
|
||||
@@ -1,294 +0,0 @@
|
||||
/**
|
||||
* OpenRouter API Summarization Endpoint with Redis Caching
|
||||
* Fallback when Groq is rate-limited
|
||||
* Uses OpenRouter auto-routed free model
|
||||
* Free tier: 50 requests/day (20/min)
|
||||
* Server-side Redis cache for cross-user deduplication
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson, hashString } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const OPENROUTER_API_URL = 'https://openrouter.ai/api/v1/chat/completions';
|
||||
const MODEL = 'openrouter/free';
|
||||
const CACHE_TTL_SECONDS = 86400; // 24 hours
|
||||
|
||||
const CACHE_VERSION = 'v3';
|
||||
|
||||
function getCacheKey(headlines, mode, geoContext = '', variant = 'full', lang = 'en') {
|
||||
const sorted = headlines.slice(0, 8).sort().join('|');
|
||||
const geoHash = geoContext ? ':g' + hashString(geoContext).slice(0, 6) : '';
|
||||
const hash = hashString(`${mode}:${sorted}`);
|
||||
const normalizedVariant = typeof variant === 'string' && variant ? variant.toLowerCase() : 'full';
|
||||
const normalizedLang = typeof lang === 'string' && lang ? lang.toLowerCase() : 'en';
|
||||
|
||||
if (mode === 'translate') {
|
||||
const targetLang = normalizedVariant || normalizedLang;
|
||||
return `summary:${CACHE_VERSION}:${mode}:${targetLang}:${hash}${geoHash}`;
|
||||
}
|
||||
|
||||
return `summary:${CACHE_VERSION}:${mode}:${normalizedVariant}:${normalizedLang}:${hash}${geoHash}`;
|
||||
}
|
||||
|
||||
// Deduplicate similar headlines (same story from different sources)
|
||||
function deduplicateHeadlines(headlines) {
|
||||
const seen = new Set();
|
||||
const unique = [];
|
||||
|
||||
for (const headline of headlines) {
|
||||
// Normalize: lowercase, remove punctuation, collapse whitespace
|
||||
const normalized = headline.toLowerCase()
|
||||
.replace(/[^\w\s]/g, '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
|
||||
// Extract key words (4+ chars) for similarity check
|
||||
const words = new Set(normalized.split(' ').filter(w => w.length >= 4));
|
||||
|
||||
// Check if this headline is too similar to any we've seen
|
||||
let isDuplicate = false;
|
||||
for (const seenWords of seen) {
|
||||
const intersection = [...words].filter(w => seenWords.has(w));
|
||||
const similarity = intersection.length / Math.min(words.size, seenWords.size);
|
||||
if (similarity > 0.6) {
|
||||
isDuplicate = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (!isDuplicate) {
|
||||
seen.add(words);
|
||||
unique.push(headline);
|
||||
}
|
||||
}
|
||||
|
||||
return unique;
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const apiKey = process.env.OPENROUTER_API_KEY;
|
||||
if (!apiKey) {
|
||||
return new Response(JSON.stringify({ summary: null, fallback: true, skipped: true, reason: 'OPENROUTER_API_KEY not configured' }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return new Response(JSON.stringify({ error: 'Payload too large' }), {
|
||||
status: 413,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const { headlines, mode = 'brief', geoContext = '', variant = 'full', lang = 'en' } = await request.json();
|
||||
|
||||
if (!headlines || !Array.isArray(headlines) || headlines.length === 0) {
|
||||
return new Response(JSON.stringify({ error: 'Headlines array required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Check cache first (shared with Groq endpoint)
|
||||
const cacheKey = getCacheKey(headlines, mode, geoContext, variant, lang);
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.summary) {
|
||||
console.log('[OpenRouter] Cache hit:', cacheKey);
|
||||
return new Response(JSON.stringify({
|
||||
summary: cached.summary,
|
||||
model: cached.model || MODEL,
|
||||
provider: 'cache',
|
||||
cached: true,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Deduplicate similar headlines (same story from different sources)
|
||||
const uniqueHeadlines = deduplicateHeadlines(headlines.slice(0, 8));
|
||||
const headlineText = uniqueHeadlines.map((h, i) => `${i + 1}. ${h}`).join('\n');
|
||||
|
||||
let systemPrompt, userPrompt;
|
||||
|
||||
// Include intelligence synthesis context in prompt if available
|
||||
const intelSection = geoContext ? `\n\n${geoContext}` : '';
|
||||
|
||||
// Current date context for LLM (models may have outdated knowledge)
|
||||
const isTechVariant = variant === 'tech';
|
||||
const dateContext = `Current date: ${new Date().toISOString().split('T')[0]}.${isTechVariant ? '' : ' Donald Trump is the current US President (second term, inaugurated Jan 2025).'}`;
|
||||
|
||||
// Language instruction
|
||||
const langInstruction = lang && lang !== 'en' ? `\nIMPORTANT: Output the summary in ${lang.toUpperCase()} language.` : '';
|
||||
|
||||
if (mode === 'brief') {
|
||||
if (isTechVariant) {
|
||||
// Tech variant: focus on startups, AI, funding, product launches
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Summarize the key tech/startup development in 2-3 sentences.
|
||||
Rules:
|
||||
- Focus ONLY on technology, startups, AI, funding, product launches, or developer news
|
||||
- IGNORE political news, trade policy, tariffs, government actions unless directly about tech regulation
|
||||
- Lead with the company/product/technology name
|
||||
- Start directly: "OpenAI announced...", "A new $50M Series B...", "GitHub released..."
|
||||
- No bullet points, no meta-commentary${langInstruction}`;
|
||||
} else {
|
||||
// Full variant: geopolitical focus
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Summarize the key development in 2-3 sentences.
|
||||
Rules:
|
||||
- Lead with WHAT happened and WHERE - be specific
|
||||
- NEVER start with "Breaking news", "Good evening", "Tonight", or TV-style openings
|
||||
- Start directly with the subject: "Iran's regime...", "The US Treasury...", "Protests in..."
|
||||
- CRITICAL FOCAL POINTS are the main actors - mention them by name
|
||||
- If focal points show news + signals convergence, that's the lead
|
||||
- No bullet points, no meta-commentary${langInstruction}`;
|
||||
}
|
||||
userPrompt = `Summarize the top story:\n${headlineText}${intelSection}`;
|
||||
} else if (mode === 'analysis') {
|
||||
if (isTechVariant) {
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Analyze the tech/startup trend in 2-3 sentences.
|
||||
Rules:
|
||||
- Focus ONLY on technology implications: funding trends, AI developments, market shifts, product strategy
|
||||
- IGNORE political implications, trade wars, government unless directly about tech policy
|
||||
- Lead with the insight for tech industry
|
||||
- Connect to startup ecosystem, VC trends, or technical implications`;
|
||||
} else {
|
||||
systemPrompt = `${dateContext}
|
||||
|
||||
Provide analysis in 2-3 sentences. Be direct and specific.
|
||||
Rules:
|
||||
- Lead with the insight - what's significant and why
|
||||
- NEVER start with "Breaking news", "Tonight", "The key/dominant narrative is"
|
||||
- Start with substance: "Iran faces...", "The escalation in...", "Multiple signals suggest..."
|
||||
- CRITICAL FOCAL POINTS are your main actors - explain WHY they matter
|
||||
- If focal points show news-signal correlation, flag as escalation
|
||||
- Connect dots, be specific about implications`;
|
||||
}
|
||||
userPrompt = isTechVariant
|
||||
? `What's the key tech trend or development?\n${headlineText}${intelSection}`
|
||||
: `What's the key pattern or risk?\n${headlineText}${intelSection}`;
|
||||
} else if (mode === 'translate') {
|
||||
const targetLang = variant; // In translate mode, variant param holds the target language code
|
||||
systemPrompt = `You are a professional news translator. Translate the following news headlines/summaries into ${targetLang}.
|
||||
Rules:
|
||||
- Maintain the original tone and journalistic style.
|
||||
- Do NOT add any conversational filler.
|
||||
- Output ONLY the translated text.`;
|
||||
userPrompt = `Translate to ${targetLang}:\n${headlines[0]}`;
|
||||
} else {
|
||||
systemPrompt = isTechVariant
|
||||
? `${dateContext}\n\nSynthesize tech news in 2 sentences. Focus on startups, AI, funding, products. Ignore politics unless directly about tech regulation.${langInstruction}`
|
||||
: `${dateContext}\n\nSynthesize in 2 sentences max. Lead with substance. NEVER start with "Breaking news" or "Tonight" - just state the insight directly. CRITICAL focal points with news-signal convergence are significant.${langInstruction}`;
|
||||
userPrompt = `Key takeaway:\n${headlineText}${intelSection}`;
|
||||
}
|
||||
|
||||
const response = await fetch(OPENROUTER_API_URL, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${apiKey}`,
|
||||
'Content-Type': 'application/json',
|
||||
'HTTP-Referer': 'https://worldmonitor.app',
|
||||
'X-Title': 'WorldMonitor',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
model: MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: systemPrompt },
|
||||
{ role: 'user', content: userPrompt },
|
||||
],
|
||||
temperature: 0.3,
|
||||
max_tokens: 150,
|
||||
top_p: 0.9,
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.error('[OpenRouter] API error:', response.status, errorText);
|
||||
|
||||
// Return fallback signal for rate limiting
|
||||
if (response.status === 429) {
|
||||
return new Response(JSON.stringify({ error: 'Rate limited', fallback: true }), {
|
||||
status: 429,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({ error: 'OpenRouter API error', fallback: true }), {
|
||||
status: response.status,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const summary = data.choices?.[0]?.message?.content?.trim();
|
||||
|
||||
if (!summary) {
|
||||
return new Response(JSON.stringify({ error: 'Empty response', fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
// Store in cache (shared with Groq endpoint)
|
||||
await setCachedJson(cacheKey, {
|
||||
summary,
|
||||
model: MODEL,
|
||||
timestamp: Date.now(),
|
||||
}, CACHE_TTL_SECONDS);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
summary,
|
||||
model: MODEL,
|
||||
provider: 'openrouter',
|
||||
cached: false,
|
||||
tokens: data.usage?.total_tokens || 0,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=1800, s-maxage=1800, stale-while-revalidate=300',
|
||||
},
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('[OpenRouter] Error:', error);
|
||||
return new Response(JSON.stringify({ error: error.message, fallback: true }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
+72
-55
@@ -1,73 +1,90 @@
|
||||
// OpenSky Network API proxy - v3
|
||||
// Note: OpenSky seems to block some cloud provider IPs
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
function getRelayBaseUrl() {
|
||||
const relayUrl = process.env.WS_RELAY_URL;
|
||||
if (!relayUrl) return null;
|
||||
return relayUrl.replace('wss://', 'https://').replace('ws://', 'http://').replace(/\/$/, '');
|
||||
}
|
||||
|
||||
function getRelayHeaders(baseHeaders = {}) {
|
||||
const headers = { ...baseHeaders };
|
||||
const relaySecret = process.env.RELAY_SHARED_SECRET || '';
|
||||
if (relaySecret) {
|
||||
const relayHeader = (process.env.RELAY_AUTH_HEADER || 'x-relay-key').toLowerCase();
|
||||
headers[relayHeader] = relaySecret;
|
||||
headers.Authorization = `Bearer ${relaySecret}`;
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
return headers;
|
||||
}
|
||||
|
||||
// Build OpenSky API URL with bounding box params
|
||||
const params = new URLSearchParams();
|
||||
['lamin', 'lomin', 'lamax', 'lomax'].forEach(key => {
|
||||
const val = url.searchParams.get(key);
|
||||
if (val) params.set(key, val);
|
||||
});
|
||||
async function fetchWithTimeout(url, options, timeoutMs = 20000) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
const openskyUrl = `https://opensky-network.org/api/states/all${params.toString() ? '?' + params.toString() : ''}`;
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const relayBaseUrl = getRelayBaseUrl();
|
||||
if (!relayBaseUrl) {
|
||||
return new Response(JSON.stringify({ error: 'WS_RELAY_URL is not configured' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Try fetching with different headers to avoid blocks
|
||||
const response = await fetch(openskyUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
'Accept-Encoding': 'gzip, deflate, br',
|
||||
'Connection': 'keep-alive',
|
||||
},
|
||||
const requestUrl = new URL(req.url);
|
||||
const relayUrl = `${relayBaseUrl}/opensky${requestUrl.search || ''}`;
|
||||
const response = await fetchWithTimeout(relayUrl, {
|
||||
headers: getRelayHeaders({ Accept: 'application/json' }),
|
||||
});
|
||||
|
||||
if (response.status === 429) {
|
||||
return Response.json({ error: 'Rate limited', time: Date.now(), states: null }, {
|
||||
status: 429,
|
||||
headers: cors,
|
||||
});
|
||||
}
|
||||
const body = await response.text();
|
||||
const headers = {
|
||||
'Content-Type': response.headers.get('content-type') || 'application/json',
|
||||
'Cache-Control': 'public, s-maxage=120, stale-while-revalidate=60',
|
||||
...corsHeaders,
|
||||
};
|
||||
const xCache = response.headers.get('x-cache');
|
||||
if (xCache) headers['X-Cache'] = xCache;
|
||||
|
||||
// Check if response is OK
|
||||
if (!response.ok) {
|
||||
const text = await response.text();
|
||||
return Response.json({
|
||||
error: `OpenSky HTTP ${response.status}: ${text.substring(0, 200)}`,
|
||||
time: Date.now(),
|
||||
states: null
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: cors,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
return Response.json(data, {
|
||||
return new Response(body, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
},
|
||||
headers,
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${error.name} - ${error.message}`,
|
||||
time: Date.now(),
|
||||
states: null
|
||||
}, {
|
||||
status: 500,
|
||||
headers: cors,
|
||||
const isTimeout = error?.name === 'AbortError';
|
||||
return new Response(JSON.stringify({
|
||||
error: isTimeout ? 'Relay timeout' : 'Relay request failed',
|
||||
details: error?.message || String(error),
|
||||
}), {
|
||||
status: isTimeout ? 504 : 502,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
try {
|
||||
const response = await fetch('https://www.pizzint.watch/api/dashboard-data', {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'WorldMonitor/1.0',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Upstream returned ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch PizzINT data', details: error.message }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,46 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const pairs = url.searchParams.get('pairs') || 'usa_russia,russia_ukraine,usa_china,china_taiwan,usa_iran,usa_venezuela';
|
||||
const dateStart = url.searchParams.get('dateStart');
|
||||
const dateEnd = url.searchParams.get('dateEnd');
|
||||
const method = url.searchParams.get('method') || 'gpr';
|
||||
|
||||
let targetUrl = `https://www.pizzint.watch/api/gdelt/batch?pairs=${encodeURIComponent(pairs)}&method=${method}`;
|
||||
if (dateStart) targetUrl += `&dateStart=${dateStart}`;
|
||||
if (dateEnd) targetUrl += `&dateEnd=${dateEnd}`;
|
||||
|
||||
try {
|
||||
const response = await fetch(targetUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'WorldMonitor/1.0',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Upstream returned ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch GDELT data', details: error.message }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
+69
-89
@@ -2,107 +2,87 @@ import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const GAMMA_BASE = 'https://gamma-api.polymarket.com';
|
||||
|
||||
const ALLOWED_ORDER = ['volume', 'liquidity', 'startDate', 'endDate', 'spread'];
|
||||
const MAX_LIMIT = 100;
|
||||
const MIN_LIMIT = 1;
|
||||
|
||||
function validateBoolean(val, defaultVal) {
|
||||
if (val === 'true' || val === 'false') return val;
|
||||
return defaultVal;
|
||||
function getRelayBaseUrl() {
|
||||
const relayUrl = process.env.WS_RELAY_URL;
|
||||
if (!relayUrl) return null;
|
||||
return relayUrl.replace('wss://', 'https://').replace('ws://', 'http://').replace(/\/$/, '');
|
||||
}
|
||||
|
||||
function validateLimit(val) {
|
||||
const num = parseInt(val, 10);
|
||||
if (isNaN(num)) return 50;
|
||||
return Math.max(MIN_LIMIT, Math.min(MAX_LIMIT, num));
|
||||
function getRelayHeaders(baseHeaders = {}) {
|
||||
const headers = { ...baseHeaders };
|
||||
const relaySecret = process.env.RELAY_SHARED_SECRET || '';
|
||||
if (relaySecret) {
|
||||
const relayHeader = (process.env.RELAY_AUTH_HEADER || 'x-relay-key').toLowerCase();
|
||||
headers[relayHeader] = relaySecret;
|
||||
headers.Authorization = `Bearer ${relaySecret}`;
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
function validateOrder(val) {
|
||||
return ALLOWED_ORDER.includes(val) ? val : 'volume';
|
||||
}
|
||||
|
||||
function sanitizeTagSlug(val) {
|
||||
if (!val) return null;
|
||||
return val.replace(/[^a-z0-9-]/gi, '').slice(0, 100) || null;
|
||||
}
|
||||
|
||||
async function tryFetch(url, timeoutMs = 8000) {
|
||||
async function fetchWithTimeout(url, options, timeoutMs = 15000) {
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timer);
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}`);
|
||||
}
|
||||
return await response.text();
|
||||
} catch (err) {
|
||||
clearTimeout(timer);
|
||||
throw err;
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
function buildUrl(base, endpoint, params) {
|
||||
if (endpoint === 'events') {
|
||||
return `${base}/events?${params}`;
|
||||
}
|
||||
return `${base}/markets?${params}`;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const endpoint = url.searchParams.get('endpoint') || 'markets';
|
||||
|
||||
const closed = validateBoolean(url.searchParams.get('closed'), 'false');
|
||||
const order = validateOrder(url.searchParams.get('order'));
|
||||
const ascending = validateBoolean(url.searchParams.get('ascending'), 'false');
|
||||
const limit = validateLimit(url.searchParams.get('limit'));
|
||||
|
||||
const params = new URLSearchParams({
|
||||
closed,
|
||||
order,
|
||||
ascending,
|
||||
limit: String(limit),
|
||||
});
|
||||
|
||||
if (endpoint === 'events') {
|
||||
const tag = sanitizeTagSlug(url.searchParams.get('tag'));
|
||||
if (tag) params.set('tag_slug', tag);
|
||||
}
|
||||
|
||||
// Gamma API is behind Cloudflare which blocks server-side TLS connections
|
||||
// (JA3 fingerprint detection). Only browser-originated requests succeed.
|
||||
// We still try in case Cloudflare policy changes, but gracefully return empty on failure.
|
||||
try {
|
||||
const data = await tryFetch(buildUrl(GAMMA_BASE, endpoint, params));
|
||||
return new Response(data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=120, s-maxage=120, stale-while-revalidate=60',
|
||||
'X-Polymarket-Source': 'gamma',
|
||||
},
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
} catch (err) {
|
||||
// Expected: Cloudflare blocks non-browser TLS connections
|
||||
return new Response(JSON.stringify([]), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'X-Polymarket-Error': err.message,
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
}
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
if (req.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
const relayBaseUrl = getRelayBaseUrl();
|
||||
if (!relayBaseUrl) {
|
||||
return new Response(JSON.stringify({ error: 'WS_RELAY_URL is not configured' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const requestUrl = new URL(req.url);
|
||||
const relayUrl = `${relayBaseUrl}/polymarket${requestUrl.search || ''}`;
|
||||
const response = await fetchWithTimeout(relayUrl, {
|
||||
headers: getRelayHeaders({ Accept: 'application/json' }),
|
||||
}, 15000);
|
||||
|
||||
const body = await response.text();
|
||||
const headers = {
|
||||
'Content-Type': response.headers.get('content-type') || 'application/json',
|
||||
'Cache-Control': response.headers.get('cache-control') || 'no-cache',
|
||||
...corsHeaders,
|
||||
};
|
||||
|
||||
return new Response(body, {
|
||||
status: response.status,
|
||||
headers,
|
||||
});
|
||||
} catch (error) {
|
||||
const isTimeout = error?.name === 'AbortError';
|
||||
return new Response(JSON.stringify({
|
||||
error: isTimeout ? 'Relay timeout' : 'Relay request failed',
|
||||
details: error?.message || String(error),
|
||||
}), {
|
||||
status: isTimeout ? 504 : 502,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { ConvexHttpClient } from 'convex/browser';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const EMAIL_RE = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
|
||||
const MAX_EMAIL_LENGTH = 320;
|
||||
|
||||
const rateLimitMap = new Map();
|
||||
const RATE_LIMIT = 5;
|
||||
const RATE_WINDOW_MS = 60 * 60 * 1000;
|
||||
|
||||
function isRateLimited(ip) {
|
||||
const now = Date.now();
|
||||
const entry = rateLimitMap.get(ip);
|
||||
if (!entry || now - entry.windowStart > RATE_WINDOW_MS) {
|
||||
rateLimitMap.set(ip, { windowStart: now, count: 1 });
|
||||
return false;
|
||||
}
|
||||
entry.count += 1;
|
||||
return entry.count > RATE_LIMIT;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
const cors = getCorsHeaders(req, 'POST, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
|
||||
if (req.method !== 'POST') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
const ip = req.headers.get('x-forwarded-for')?.split(',')[0]?.trim() || 'unknown';
|
||||
if (isRateLimited(ip)) {
|
||||
return new Response(JSON.stringify({ error: 'Too many requests' }), {
|
||||
status: 429,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
let body;
|
||||
try {
|
||||
body = await req.json();
|
||||
} catch {
|
||||
return new Response(JSON.stringify({ error: 'Invalid JSON' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
const { email, source, appVersion } = body;
|
||||
if (!email || typeof email !== 'string' || email.length > MAX_EMAIL_LENGTH || !EMAIL_RE.test(email)) {
|
||||
return new Response(JSON.stringify({ error: 'Invalid email address' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
const convexUrl = process.env.CONVEX_URL;
|
||||
if (!convexUrl) {
|
||||
return new Response(JSON.stringify({ error: 'Registration service unavailable' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const client = new ConvexHttpClient(convexUrl);
|
||||
const result = await client.mutation('registerInterest:register', {
|
||||
email,
|
||||
source: source || 'unknown',
|
||||
appVersion: appVersion || 'unknown',
|
||||
});
|
||||
return new Response(JSON.stringify(result), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[register-interest] Convex error:', err);
|
||||
return new Response(JSON.stringify({ error: 'Registration failed' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,355 +0,0 @@
|
||||
/**
|
||||
* Risk Scores API - Cached CII and Strategic Risk computation
|
||||
* Eliminates 15-minute "learning mode" for users by pre-computing scores
|
||||
* Uses Upstash Redis for cross-user caching (10-minute TTL)
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const CACHE_TTL_SECONDS = 600; // 10 minutes
|
||||
const STALE_CACHE_TTL_SECONDS = 3600; // 1 hour - serve stale when API fails
|
||||
const CACHE_KEY = 'risk:scores:v2';
|
||||
const STALE_CACHE_KEY = 'risk:scores:stale:v2';
|
||||
|
||||
// Tier 1 countries for CII
|
||||
const TIER1_COUNTRIES = {
|
||||
US: 'United States', RU: 'Russia', CN: 'China', UA: 'Ukraine', IR: 'Iran',
|
||||
IL: 'Israel', TW: 'Taiwan', KP: 'North Korea', SA: 'Saudi Arabia', TR: 'Turkey',
|
||||
PL: 'Poland', DE: 'Germany', FR: 'France', GB: 'United Kingdom', IN: 'India',
|
||||
PK: 'Pakistan', SY: 'Syria', YE: 'Yemen', MM: 'Myanmar', VE: 'Venezuela',
|
||||
};
|
||||
|
||||
// Baseline geopolitical risk (0-50)
|
||||
const BASELINE_RISK = {
|
||||
US: 5, RU: 35, CN: 25, UA: 50, IR: 40, IL: 45, TW: 30, KP: 45,
|
||||
SA: 20, TR: 25, PL: 10, DE: 5, FR: 10, GB: 5, IN: 20, PK: 35,
|
||||
SY: 50, YE: 50, MM: 45, VE: 40,
|
||||
};
|
||||
|
||||
// Event significance multipliers
|
||||
const EVENT_MULTIPLIER = {
|
||||
US: 0.3, RU: 2.0, CN: 2.5, UA: 0.8, IR: 2.0, IL: 0.7, TW: 1.5, KP: 3.0,
|
||||
SA: 2.0, TR: 1.2, PL: 0.8, DE: 0.5, FR: 0.6, GB: 0.5, IN: 0.8, PK: 1.5,
|
||||
SY: 0.7, YE: 0.7, MM: 1.8, VE: 1.8,
|
||||
};
|
||||
|
||||
// Country keywords for matching
|
||||
const COUNTRY_KEYWORDS = {
|
||||
US: ['united states', 'usa', 'america', 'washington', 'biden', 'trump', 'pentagon'],
|
||||
RU: ['russia', 'moscow', 'kremlin', 'putin'],
|
||||
CN: ['china', 'beijing', 'xi jinping', 'prc'],
|
||||
UA: ['ukraine', 'kyiv', 'zelensky', 'donbas'],
|
||||
IR: ['iran', 'tehran', 'khamenei', 'irgc'],
|
||||
IL: ['israel', 'tel aviv', 'netanyahu', 'idf', 'gaza'],
|
||||
TW: ['taiwan', 'taipei'],
|
||||
KP: ['north korea', 'pyongyang', 'kim jong'],
|
||||
SA: ['saudi arabia', 'riyadh'],
|
||||
TR: ['turkey', 'ankara', 'erdogan'],
|
||||
PL: ['poland', 'warsaw'],
|
||||
DE: ['germany', 'berlin'],
|
||||
FR: ['france', 'paris', 'macron'],
|
||||
GB: ['britain', 'uk', 'london'],
|
||||
IN: ['india', 'delhi', 'modi'],
|
||||
PK: ['pakistan', 'islamabad'],
|
||||
SY: ['syria', 'damascus'],
|
||||
YE: ['yemen', 'sanaa', 'houthi'],
|
||||
MM: ['myanmar', 'burma'],
|
||||
VE: ['venezuela', 'caracas', 'maduro'],
|
||||
};
|
||||
|
||||
function normalizeCountryName(text) {
|
||||
const lower = text.toLowerCase();
|
||||
for (const [code, keywords] of Object.entries(COUNTRY_KEYWORDS)) {
|
||||
if (keywords.some(kw => lower.includes(kw))) return code;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function getScoreLevel(score) {
|
||||
if (score >= 70) return 'critical';
|
||||
if (score >= 55) return 'high';
|
||||
if (score >= 40) return 'elevated';
|
||||
if (score >= 25) return 'normal';
|
||||
return 'low';
|
||||
}
|
||||
|
||||
async function fetchACLEDProtests() {
|
||||
try {
|
||||
// Fetch recent protests from ACLED (last 7 days)
|
||||
const endDate = new Date().toISOString().split('T')[0];
|
||||
const startDate = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString().split('T')[0];
|
||||
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 15000); // 15s timeout
|
||||
|
||||
// ACLED API now requires authentication - new endpoint as of Jan 2026
|
||||
const token = process.env.ACLED_ACCESS_TOKEN;
|
||||
const headers = { 'Accept': 'application/json' };
|
||||
if (token) {
|
||||
headers['Authorization'] = `Bearer ${token}`;
|
||||
}
|
||||
|
||||
// Updated endpoint: acleddata.com/api/ instead of api.acleddata.com
|
||||
const response = await fetch(
|
||||
`https://acleddata.com/api/acled/read?_format=json&event_type=Protests&event_type=Riots&event_date=${startDate}|${endDate}&event_date_where=BETWEEN&limit=500`,
|
||||
{
|
||||
headers,
|
||||
signal: controller.signal,
|
||||
}
|
||||
);
|
||||
|
||||
clearTimeout(timeoutId);
|
||||
|
||||
if (!response.ok) {
|
||||
const text = await response.text().catch(() => '');
|
||||
console.warn('[RiskScores] ACLED fetch failed:', response.status, text.slice(0, 200));
|
||||
// Check for auth errors specifically
|
||||
if (response.status === 401 || response.status === 403) {
|
||||
throw new Error('ACLED API requires valid authentication token');
|
||||
}
|
||||
throw new Error(`ACLED API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
// Check for API-level error in response
|
||||
if (data.message) {
|
||||
console.warn('[RiskScores] ACLED API returned message:', data.message);
|
||||
throw new Error(data.message);
|
||||
}
|
||||
if (data.error || data.success === false) {
|
||||
console.warn('[RiskScores] ACLED API returned error:', data.error || 'unknown');
|
||||
throw new Error(data.error || 'ACLED API error');
|
||||
}
|
||||
|
||||
return data.data || [];
|
||||
} catch (error) {
|
||||
console.warn('[RiskScores] ACLED error:', error.message);
|
||||
throw error; // Re-throw to trigger stale cache fallback
|
||||
}
|
||||
}
|
||||
|
||||
function computeCIIScores(protests) {
|
||||
const countryEvents = new Map();
|
||||
|
||||
// Count events per country
|
||||
for (const event of protests) {
|
||||
const country = event.country;
|
||||
const code = normalizeCountryName(country);
|
||||
if (code && TIER1_COUNTRIES[code]) {
|
||||
const count = countryEvents.get(code) || { protests: 0, riots: 0 };
|
||||
if (event.event_type === 'Riots') {
|
||||
count.riots++;
|
||||
} else {
|
||||
count.protests++;
|
||||
}
|
||||
countryEvents.set(code, count);
|
||||
}
|
||||
}
|
||||
|
||||
// Compute scores for all Tier 1 countries
|
||||
const scores = [];
|
||||
const now = new Date();
|
||||
|
||||
for (const [code, name] of Object.entries(TIER1_COUNTRIES)) {
|
||||
const events = countryEvents.get(code) || { protests: 0, riots: 0 };
|
||||
const baseline = BASELINE_RISK[code] || 20;
|
||||
const multiplier = EVENT_MULTIPLIER[code] || 1.0;
|
||||
|
||||
// Unrest component: protests + riots (riots weighted 2x)
|
||||
const unrestRaw = (events.protests + events.riots * 2) * multiplier;
|
||||
const unrest = Math.min(100, Math.round(unrestRaw * 2));
|
||||
|
||||
// Security component: baseline + riot contribution
|
||||
const security = Math.min(100, baseline + events.riots * multiplier * 5);
|
||||
|
||||
// Information component: based on event count (proxy for news coverage)
|
||||
const totalEvents = events.protests + events.riots;
|
||||
const information = Math.min(100, totalEvents * multiplier * 3);
|
||||
|
||||
// Composite score: weighted average + baseline
|
||||
const composite = Math.min(100, Math.round(
|
||||
baseline +
|
||||
(unrest * 0.4 + security * 0.35 + information * 0.25) * 0.5
|
||||
));
|
||||
|
||||
scores.push({
|
||||
code,
|
||||
name,
|
||||
score: composite,
|
||||
level: getScoreLevel(composite),
|
||||
trend: 'stable', // Would need historical data for real trend
|
||||
change24h: 0,
|
||||
components: { unrest, security, information },
|
||||
lastUpdated: now.toISOString(),
|
||||
});
|
||||
}
|
||||
|
||||
// Sort by score descending
|
||||
scores.sort((a, b) => b.score - a.score);
|
||||
return scores;
|
||||
}
|
||||
|
||||
function computeStrategicRisk(ciiScores) {
|
||||
// Top 5 CII scores weighted average
|
||||
const top5 = ciiScores.slice(0, 5);
|
||||
const weights = top5.map((_, i) => 1 - (i * 0.15)); // [1.0, 0.85, 0.70, 0.55, 0.40]
|
||||
const totalWeight = weights.reduce((sum, w) => sum + w, 0); // 3.5
|
||||
const weightedSum = top5.reduce((sum, s, i) => sum + s.score * weights[i], 0);
|
||||
const ciiComponent = weightedSum / totalWeight;
|
||||
|
||||
// Overall strategic risk
|
||||
const overallScore = Math.round(ciiComponent * 0.7 + 15); // 30% baseline
|
||||
|
||||
return {
|
||||
score: Math.min(100, overallScore),
|
||||
level: getScoreLevel(overallScore),
|
||||
trend: 'stable',
|
||||
lastUpdated: new Date().toISOString(),
|
||||
contributors: top5.map(s => ({
|
||||
country: s.name,
|
||||
code: s.code,
|
||||
score: s.score,
|
||||
level: s.level,
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'GET, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (request.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (!process.env.ACLED_ACCESS_TOKEN) {
|
||||
const baselineScores = computeCIIScores([]);
|
||||
const baselineStrategic = computeStrategicRisk(baselineScores);
|
||||
return new Response(JSON.stringify({
|
||||
cii: baselineScores,
|
||||
strategicRisk: baselineStrategic,
|
||||
protestCount: 0,
|
||||
computedAt: new Date().toISOString(),
|
||||
baseline: true,
|
||||
error: 'ACLED token not configured - showing baseline risk assessments',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Check cache first
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (cached && typeof cached === 'object') {
|
||||
console.log('[RiskScores] Cache hit');
|
||||
return new Response(JSON.stringify({
|
||||
...cached,
|
||||
cached: true,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Fetch ACLED protests
|
||||
console.log('[RiskScores] Computing scores...');
|
||||
const protests = await fetchACLEDProtests();
|
||||
|
||||
// Compute CII scores
|
||||
const ciiScores = computeCIIScores(protests);
|
||||
|
||||
// Compute strategic risk
|
||||
const strategicRisk = computeStrategicRisk(ciiScores);
|
||||
|
||||
const result = {
|
||||
cii: ciiScores,
|
||||
strategicRisk,
|
||||
protestCount: protests.length,
|
||||
computedAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
// Cache (both regular and stale backup)
|
||||
await Promise.all([
|
||||
setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS),
|
||||
setCachedJson(STALE_CACHE_KEY, result, STALE_CACHE_TTL_SECONDS),
|
||||
]);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
...result,
|
||||
cached: false,
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
},
|
||||
});
|
||||
|
||||
} catch (error) {
|
||||
console.error('[RiskScores] Error:', error);
|
||||
|
||||
// Try to return stale cached data
|
||||
const stale = await getCachedJson(STALE_CACHE_KEY);
|
||||
if (stale && typeof stale === 'object') {
|
||||
console.log('[RiskScores] Returning stale cache due to error');
|
||||
return new Response(JSON.stringify({
|
||||
...stale,
|
||||
cached: true,
|
||||
stale: true,
|
||||
error: 'Using cached data - ACLED temporarily unavailable',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Final fallback: return baseline scores without unrest data
|
||||
console.log('[RiskScores] Returning baseline scores (no ACLED data)');
|
||||
const baselineScores = computeCIIScores([]); // Empty protests = baseline only
|
||||
const baselineStrategic = computeStrategicRisk(baselineScores);
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
cii: baselineScores,
|
||||
strategicRisk: baselineStrategic,
|
||||
protestCount: 0,
|
||||
computedAt: new Date().toISOString(),
|
||||
baseline: true,
|
||||
error: 'ACLED unavailable - showing baseline risk assessments',
|
||||
}), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
+162
-35
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
@@ -14,6 +15,35 @@ async function fetchWithTimeout(url, options, timeoutMs = 15000) {
|
||||
}
|
||||
}
|
||||
|
||||
function getRelayBaseUrl() {
|
||||
const relayUrl = process.env.WS_RELAY_URL || '';
|
||||
if (!relayUrl) return '';
|
||||
return relayUrl.replace('wss://', 'https://').replace('ws://', 'http://').replace(/\/$/, '');
|
||||
}
|
||||
|
||||
function getRelayHeaders(baseHeaders = {}) {
|
||||
const headers = { ...baseHeaders };
|
||||
const relaySecret = process.env.RELAY_SHARED_SECRET || '';
|
||||
if (relaySecret) {
|
||||
const relayHeader = (process.env.RELAY_AUTH_HEADER || 'x-relay-key').toLowerCase();
|
||||
headers[relayHeader] = relaySecret;
|
||||
headers.Authorization = `Bearer ${relaySecret}`;
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
async function fetchViaRailway(feedUrl, timeoutMs) {
|
||||
const relayBaseUrl = getRelayBaseUrl();
|
||||
if (!relayBaseUrl) return null;
|
||||
const relayUrl = `${relayBaseUrl}/rss?url=${encodeURIComponent(feedUrl)}`;
|
||||
return fetchWithTimeout(relayUrl, {
|
||||
headers: getRelayHeaders({
|
||||
'Accept': 'application/rss+xml, application/xml, text/xml, */*',
|
||||
'User-Agent': 'WorldMonitor-RSS-Proxy/1.0',
|
||||
}),
|
||||
}, timeoutMs);
|
||||
}
|
||||
|
||||
// Allowed RSS feed domains for security
|
||||
const ALLOWED_DOMAINS = [
|
||||
'feeds.bbci.co.uk',
|
||||
@@ -21,6 +51,7 @@ const ALLOWED_DOMAINS = [
|
||||
'feeds.npr.org',
|
||||
'news.google.com',
|
||||
'www.aljazeera.com',
|
||||
'www.aljazeera.net',
|
||||
'rss.cnn.com',
|
||||
'hnrss.org',
|
||||
'feeds.arstechnica.com',
|
||||
@@ -63,6 +94,11 @@ const ALLOWED_DOMAINS = [
|
||||
'www.brookings.edu',
|
||||
'layoffs.fyi',
|
||||
'www.defensenews.com',
|
||||
'www.militarytimes.com',
|
||||
'taskandpurpose.com',
|
||||
'news.usni.org',
|
||||
'www.oryxspioenkop.com',
|
||||
'www.gov.uk',
|
||||
'www.foreignaffairs.com',
|
||||
'www.atlanticcouncil.org',
|
||||
// Tech variant domains
|
||||
@@ -70,6 +106,7 @@ const ALLOWED_DOMAINS = [
|
||||
'www.techmeme.com',
|
||||
'www.darkreading.com',
|
||||
'www.schneier.com',
|
||||
'www.ransomware.live',
|
||||
'rss.politico.com',
|
||||
'www.anandtech.com',
|
||||
'www.tomshardware.com',
|
||||
@@ -133,12 +170,35 @@ const ALLOWED_DOMAINS = [
|
||||
// International News Sources
|
||||
'www.france24.com',
|
||||
'www.euronews.com',
|
||||
'de.euronews.com',
|
||||
'es.euronews.com',
|
||||
'fr.euronews.com',
|
||||
'it.euronews.com',
|
||||
'pt.euronews.com',
|
||||
'ru.euronews.com',
|
||||
'www.lemonde.fr',
|
||||
'rss.dw.com',
|
||||
'www.bild.de',
|
||||
'www.africanews.com',
|
||||
'fr.africanews.com',
|
||||
// Nigeria
|
||||
'www.premiumtimesng.com',
|
||||
'www.vanguardngr.com',
|
||||
'www.channelstv.com',
|
||||
'dailytrust.com',
|
||||
'www.thisdaylive.com',
|
||||
// Greek
|
||||
'www.naftemporiki.gr',
|
||||
'www.in.gr',
|
||||
'www.iefimerida.gr',
|
||||
'www.lasillavacia.com',
|
||||
'www.channelnewsasia.com',
|
||||
'japantoday.com',
|
||||
'www.thehindu.com',
|
||||
'indianexpress.com',
|
||||
'www.indianewsnetwork.com',
|
||||
'www.twz.com',
|
||||
'gcaptain.com',
|
||||
// International Organizations
|
||||
'news.un.org',
|
||||
'www.iaea.org',
|
||||
@@ -171,12 +231,74 @@ const ALLOWED_DOMAINS = [
|
||||
'www.fao.org',
|
||||
'worldbank.org',
|
||||
'www.imf.org',
|
||||
// International news (various languages)
|
||||
'www.bbc.com',
|
||||
'www.spiegel.de',
|
||||
'www.tagesschau.de',
|
||||
'newsfeed.zeit.de',
|
||||
'feeds.elpais.com',
|
||||
'e00-elmundo.uecdn.es',
|
||||
'www.repubblica.it',
|
||||
'www.ansa.it',
|
||||
'xml2.corriereobjects.it',
|
||||
'feeds.nos.nl',
|
||||
'www.nrc.nl',
|
||||
'www.telegraaf.nl',
|
||||
'www.dn.se',
|
||||
'www.svd.se',
|
||||
'www.svt.se',
|
||||
'www.asahi.com',
|
||||
'www.clarin.com',
|
||||
'oglobo.globo.com',
|
||||
'feeds.folha.uol.com.br',
|
||||
'www.eltiempo.com',
|
||||
'www.eluniversal.com.mx',
|
||||
'www.jeuneafrique.com',
|
||||
'www.lorientlejour.com',
|
||||
// Regional locale feeds (tr, pl, ru, th, vi, pt)
|
||||
'www.hurriyet.com.tr',
|
||||
'tvn24.pl',
|
||||
'www.polsatnews.pl',
|
||||
'www.rp.pl',
|
||||
'meduza.io',
|
||||
'novayagazeta.eu',
|
||||
'www.bangkokpost.com',
|
||||
'vnexpress.net',
|
||||
'www.abc.net.au',
|
||||
'islandtimes.org',
|
||||
'www.brasilparalelo.com.br',
|
||||
// Mexico & LatAm Security
|
||||
'mexiconewsdaily.com',
|
||||
'animalpolitico.com',
|
||||
'www.proceso.com.mx',
|
||||
'www.milenio.com',
|
||||
'insightcrime.org',
|
||||
// Additional
|
||||
'news.ycombinator.com',
|
||||
// Finance variant
|
||||
'seekingalpha.com',
|
||||
'www.coindesk.com',
|
||||
'cointelegraph.com',
|
||||
// Happy variant — positive news sources
|
||||
'www.goodnewsnetwork.org',
|
||||
'www.positive.news',
|
||||
'reasonstobecheerful.world',
|
||||
'www.optimistdaily.com',
|
||||
'www.upworthy.com',
|
||||
'www.dailygood.org',
|
||||
'www.goodgoodgood.co',
|
||||
'www.good.is',
|
||||
'www.sunnyskyz.com',
|
||||
'thebetterindia.com',
|
||||
'singularityhub.com',
|
||||
'humanprogress.org',
|
||||
'greatergood.berkeley.edu',
|
||||
'www.onlygoodnewsdaily.com',
|
||||
'www.sciencedaily.com',
|
||||
'feeds.nature.com',
|
||||
'www.nature.com',
|
||||
'www.livescience.com',
|
||||
'www.newscientist.com',
|
||||
];
|
||||
|
||||
export default async function handler(req) {
|
||||
@@ -200,8 +322,11 @@ export default async function handler(req) {
|
||||
try {
|
||||
const parsedUrl = new URL(feedUrl);
|
||||
|
||||
// Security: Check if domain is allowed
|
||||
if (!ALLOWED_DOMAINS.includes(parsedUrl.hostname)) {
|
||||
// Security: Check if domain is allowed (normalize www prefix)
|
||||
const hostname = parsedUrl.hostname;
|
||||
const bare = hostname.replace(/^www\./, '');
|
||||
const withWww = hostname.startsWith('www.') ? hostname : `www.${hostname}`;
|
||||
if (!ALLOWED_DOMAINS.includes(hostname) && !ALLOWED_DOMAINS.includes(bare) && !ALLOWED_DOMAINS.includes(withWww)) {
|
||||
return new Response(JSON.stringify({ error: 'Domain not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
@@ -212,57 +337,59 @@ export default async function handler(req) {
|
||||
const isGoogleNews = feedUrl.includes('news.google.com');
|
||||
const timeout = isGoogleNews ? 20000 : 12000;
|
||||
|
||||
const response = await fetchWithTimeout(feedUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'Accept': 'application/rss+xml, application/xml, text/xml, */*',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
},
|
||||
redirect: 'manual',
|
||||
}, timeout);
|
||||
const fetchDirect = async () => {
|
||||
const response = await fetchWithTimeout(feedUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'Accept': 'application/rss+xml, application/xml, text/xml, */*',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
},
|
||||
redirect: 'manual',
|
||||
}, timeout);
|
||||
|
||||
if (response.status >= 300 && response.status < 400) {
|
||||
const location = response.headers.get('location');
|
||||
if (location) {
|
||||
try {
|
||||
if (response.status >= 300 && response.status < 400) {
|
||||
const location = response.headers.get('location');
|
||||
if (location) {
|
||||
const redirectUrl = new URL(location, feedUrl);
|
||||
if (!ALLOWED_DOMAINS.includes(redirectUrl.hostname)) {
|
||||
return new Response(JSON.stringify({ error: 'Redirect to disallowed domain' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
throw new Error('Redirect to disallowed domain');
|
||||
}
|
||||
const redirectResponse = await fetchWithTimeout(redirectUrl.href, {
|
||||
return fetchWithTimeout(redirectUrl.href, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36',
|
||||
'Accept': 'application/rss+xml, application/xml, text/xml, */*',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
},
|
||||
}, timeout);
|
||||
const data = await redirectResponse.text();
|
||||
return new Response(data, {
|
||||
status: redirectResponse.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/xml',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
} catch {
|
||||
return new Response(JSON.stringify({ error: 'Invalid redirect' }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json', ...corsHeaders },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return response;
|
||||
};
|
||||
|
||||
let response;
|
||||
let usedRelay = false;
|
||||
try {
|
||||
response = await fetchDirect();
|
||||
} catch (directError) {
|
||||
response = await fetchViaRailway(feedUrl, timeout);
|
||||
usedRelay = !!response;
|
||||
if (!response) throw directError;
|
||||
}
|
||||
|
||||
if (!response.ok && !usedRelay) {
|
||||
const relayResponse = await fetchViaRailway(feedUrl, timeout);
|
||||
if (relayResponse && relayResponse.ok) {
|
||||
response = relayResponse;
|
||||
}
|
||||
}
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/xml',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60',
|
||||
'Content-Type': response.headers.get('content-type') || 'application/xml',
|
||||
'Cache-Control': response.headers.get('cache-control') || 'public, max-age=600, s-maxage=600, stale-while-revalidate=300',
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,296 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
// Major tech services and their status page endpoints
|
||||
// Most use Statuspage.io which has a standard /api/v2/status.json endpoint
|
||||
const SERVICES = [
|
||||
// Cloud Providers
|
||||
{ id: 'aws', name: 'AWS', statusPage: 'https://health.aws.amazon.com/health/status', customParser: 'aws', category: 'cloud' },
|
||||
{ id: 'azure', name: 'Azure', statusPage: 'https://azure.status.microsoft/en-us/status/feed/', customParser: 'rss', category: 'cloud' },
|
||||
{ id: 'gcp', name: 'Google Cloud', statusPage: 'https://status.cloud.google.com/incidents.json', customParser: 'gcp', category: 'cloud' },
|
||||
{ id: 'cloudflare', name: 'Cloudflare', statusPage: 'https://www.cloudflarestatus.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'vercel', name: 'Vercel', statusPage: 'https://www.vercel-status.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'netlify', name: 'Netlify', statusPage: 'https://www.netlifystatus.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'digitalocean', name: 'DigitalOcean', statusPage: 'https://status.digitalocean.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'render', name: 'Render', statusPage: 'https://status.render.com/api/v2/status.json', category: 'cloud' },
|
||||
{ id: 'railway', name: 'Railway', statusPage: 'https://railway.instatus.com/summary.json', customParser: 'instatus', category: 'cloud' },
|
||||
|
||||
// Developer Tools
|
||||
{ id: 'github', name: 'GitHub', statusPage: 'https://www.githubstatus.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'gitlab', name: 'GitLab', statusPage: 'https://status.gitlab.com/1.0/status/5b36dc6502d06804c08349f7', customParser: 'statusio', category: 'dev' },
|
||||
{ id: 'npm', name: 'npm', statusPage: 'https://status.npmjs.org/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'docker', name: 'Docker Hub', statusPage: 'https://www.dockerstatus.com/1.0/status/533c6539221ae15e3f000031', customParser: 'statusio', category: 'dev' },
|
||||
{ id: 'bitbucket', name: 'Bitbucket', statusPage: 'https://bitbucket.status.atlassian.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'circleci', name: 'CircleCI', statusPage: 'https://status.circleci.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'jira', name: 'Jira', statusPage: 'https://jira-software.status.atlassian.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'confluence', name: 'Confluence', statusPage: 'https://confluence.status.atlassian.com/api/v2/status.json', category: 'dev' },
|
||||
{ id: 'linear', name: 'Linear', statusPage: 'https://linearstatus.com/api/v2/status.json', customParser: 'incidentio', category: 'dev' },
|
||||
|
||||
// Communication
|
||||
{ id: 'slack', name: 'Slack', statusPage: 'https://slack-status.com/api/v2.0.0/current', customParser: 'slack', category: 'comm' },
|
||||
{ id: 'discord', name: 'Discord', statusPage: 'https://discordstatus.com/api/v2/status.json', category: 'comm' },
|
||||
{ id: 'zoom', name: 'Zoom', statusPage: 'https://www.zoomstatus.com/api/v2/status.json', category: 'comm' },
|
||||
{ id: 'notion', name: 'Notion', statusPage: 'https://www.notion-status.com/api/v2/status.json', category: 'comm' },
|
||||
|
||||
// AI Services (incident.io powered)
|
||||
{ id: 'openai', name: 'OpenAI', statusPage: 'https://status.openai.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
|
||||
{ id: 'anthropic', name: 'Anthropic', statusPage: 'https://status.claude.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
|
||||
{ id: 'replicate', name: 'Replicate', statusPage: 'https://www.replicatestatus.com/api/v2/status.json', customParser: 'incidentio', category: 'ai' },
|
||||
|
||||
// SaaS
|
||||
{ id: 'stripe', name: 'Stripe', statusPage: 'https://status.stripe.com/current', customParser: 'stripe', category: 'saas' },
|
||||
{ id: 'twilio', name: 'Twilio', statusPage: 'https://status.twilio.com/api/v2/status.json', category: 'saas' },
|
||||
{ id: 'datadog', name: 'Datadog', statusPage: 'https://status.datadoghq.com/api/v2/status.json', category: 'saas' },
|
||||
{ id: 'sentry', name: 'Sentry', statusPage: 'https://status.sentry.io/api/v2/status.json', category: 'saas' },
|
||||
{ id: 'supabase', name: 'Supabase', statusPage: 'https://status.supabase.com/api/v2/status.json', category: 'saas' },
|
||||
];
|
||||
|
||||
// Statuspage.io API returns status like: none, minor, major, critical
|
||||
function normalizeStatus(indicator) {
|
||||
if (!indicator) return 'unknown';
|
||||
const val = indicator.toLowerCase();
|
||||
// Check for operational indicators
|
||||
if (val === 'none' || val === 'operational' || val.includes('all systems operational')) {
|
||||
return 'operational';
|
||||
}
|
||||
// Check for degraded indicators
|
||||
if (val === 'minor' || val === 'degraded_performance' || val === 'partial_outage' || val.includes('degraded')) {
|
||||
return 'degraded';
|
||||
}
|
||||
// Check for outage indicators
|
||||
if (val === 'major' || val === 'major_outage' || val === 'critical' || val.includes('outage')) {
|
||||
return 'outage';
|
||||
}
|
||||
return 'unknown';
|
||||
}
|
||||
|
||||
async function checkStatusPage(service) {
|
||||
if (!service.statusPage) {
|
||||
return { ...service, status: 'unknown', description: 'No API available' };
|
||||
}
|
||||
|
||||
try {
|
||||
// Use browser-like headers to avoid being blocked
|
||||
const headers = {
|
||||
'Accept': service.customParser === 'rss' ? 'application/xml, text/xml' : 'application/json, text/plain, */*',
|
||||
'Accept-Language': 'en-US,en;q=0.9',
|
||||
'Cache-Control': 'no-cache',
|
||||
};
|
||||
// Don't send User-Agent for incident.io - they may block bots
|
||||
if (service.customParser !== 'incidentio') {
|
||||
headers['User-Agent'] = 'Mozilla/5.0 (compatible; WorldMonitor/1.0)';
|
||||
}
|
||||
|
||||
const response = await fetch(service.statusPage, {
|
||||
headers,
|
||||
signal: AbortSignal.timeout(10000),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return { ...service, status: 'unknown', description: `HTTP ${response.status}` };
|
||||
}
|
||||
|
||||
// Handle custom parsers
|
||||
if (service.customParser === 'gcp') {
|
||||
const data = await response.json();
|
||||
// GCP incidents.json returns array of incidents
|
||||
const activeIncidents = Array.isArray(data) ? data.filter(i =>
|
||||
i.end === undefined || new Date(i.end) > new Date()
|
||||
) : [];
|
||||
if (activeIncidents.length === 0) {
|
||||
return { ...service, status: 'operational', description: 'All services operational' };
|
||||
}
|
||||
const severity = activeIncidents.some(i => i.severity === 'high') ? 'outage' : 'degraded';
|
||||
return { ...service, status: severity, description: `${activeIncidents.length} active incident(s)` };
|
||||
}
|
||||
|
||||
if (service.customParser === 'aws') {
|
||||
// AWS status page is complex HTML - assume operational if reachable
|
||||
return { ...service, status: 'operational', description: 'Status page reachable' };
|
||||
}
|
||||
|
||||
if (service.customParser === 'rss') {
|
||||
// Azure RSS feed - check if there are recent items (incidents)
|
||||
const text = await response.text();
|
||||
const hasRecentIncident = text.includes('<item>') &&
|
||||
(text.includes('degradation') || text.includes('outage') || text.includes('incident'));
|
||||
return {
|
||||
...service,
|
||||
status: hasRecentIncident ? 'degraded' : 'operational',
|
||||
description: hasRecentIncident ? 'Recent incidents reported' : 'No recent incidents'
|
||||
};
|
||||
}
|
||||
|
||||
if (service.customParser === 'instatus') {
|
||||
// Instatus format (Railway, etc.)
|
||||
const data = await response.json();
|
||||
const pageStatus = data.page?.status;
|
||||
if (pageStatus === 'UP') {
|
||||
return { ...service, status: 'operational', description: 'All systems operational' };
|
||||
} else if (pageStatus === 'HASISSUES') {
|
||||
return { ...service, status: 'degraded', description: 'Some issues reported' };
|
||||
} else {
|
||||
return { ...service, status: 'unknown', description: pageStatus || 'Unknown' };
|
||||
}
|
||||
}
|
||||
|
||||
if (service.customParser === 'statusio') {
|
||||
// Status.io format (GitLab, Docker Hub)
|
||||
const data = await response.json();
|
||||
const overall = data.result?.status_overall;
|
||||
const statusCode = overall?.status_code;
|
||||
if (statusCode === 100) {
|
||||
return { ...service, status: 'operational', description: overall.status || 'All systems operational' };
|
||||
} else if (statusCode >= 300 && statusCode < 500) {
|
||||
return { ...service, status: 'degraded', description: overall.status || 'Degraded performance' };
|
||||
} else if (statusCode >= 500) {
|
||||
return { ...service, status: 'outage', description: overall.status || 'Service disruption' };
|
||||
}
|
||||
return { ...service, status: 'unknown', description: overall?.status || 'Unknown status' };
|
||||
}
|
||||
|
||||
if (service.customParser === 'slack') {
|
||||
// Slack custom API format
|
||||
const data = await response.json();
|
||||
if (data.status === 'ok') {
|
||||
return { ...service, status: 'operational', description: 'All systems operational' };
|
||||
} else if (data.status === 'active' || data.active_incidents?.length > 0) {
|
||||
const count = data.active_incidents?.length || 1;
|
||||
return { ...service, status: 'degraded', description: `${count} active incident(s)` };
|
||||
}
|
||||
return { ...service, status: 'unknown', description: data.status || 'Unknown' };
|
||||
}
|
||||
|
||||
if (service.customParser === 'stripe') {
|
||||
// Stripe custom API format at /current
|
||||
const data = await response.json();
|
||||
if (data.largestatus === 'up') {
|
||||
return { ...service, status: 'operational', description: data.message || 'All systems operational' };
|
||||
} else if (data.largestatus === 'degraded') {
|
||||
return { ...service, status: 'degraded', description: data.message || 'Degraded performance' };
|
||||
} else if (data.largestatus === 'down') {
|
||||
return { ...service, status: 'outage', description: data.message || 'Service disruption' };
|
||||
}
|
||||
return { ...service, status: 'unknown', description: data.message || 'Unknown' };
|
||||
}
|
||||
|
||||
if (service.customParser === 'incidentio') {
|
||||
// incident.io status pages (OpenAI, Linear, Replicate, Anthropic)
|
||||
const text = await response.text();
|
||||
// Check for HTML response (blocked)
|
||||
if (text.startsWith('<!') || text.startsWith('<html')) {
|
||||
// Try parsing HTML for status - incident.io pages have status in HTML
|
||||
const operationalMatch = text.match(/All Systems Operational|fully operational|no issues/i);
|
||||
if (operationalMatch) {
|
||||
return { ...service, status: 'operational', description: 'All systems operational' };
|
||||
}
|
||||
const degradedMatch = text.match(/degraded|partial outage|experiencing issues/i);
|
||||
if (degradedMatch) {
|
||||
return { ...service, status: 'degraded', description: 'Some issues reported' };
|
||||
}
|
||||
return { ...service, status: 'unknown', description: 'Could not parse status' };
|
||||
}
|
||||
// Parse JSON response
|
||||
try {
|
||||
const data = JSON.parse(text);
|
||||
const indicator = data.status?.indicator || '';
|
||||
const description = data.status?.description || '';
|
||||
if (indicator === 'none' || description.toLowerCase().includes('operational')) {
|
||||
return { ...service, status: 'operational', description: description || 'All systems operational' };
|
||||
} else if (indicator === 'minor' || indicator === 'maintenance') {
|
||||
return { ...service, status: 'degraded', description: description || 'Minor issues' };
|
||||
} else if (indicator === 'major' || indicator === 'critical') {
|
||||
return { ...service, status: 'outage', description: description || 'Major outage' };
|
||||
}
|
||||
return { ...service, status: 'operational', description: description || 'Status OK' };
|
||||
} catch {
|
||||
return { ...service, status: 'unknown', description: 'Invalid response' };
|
||||
}
|
||||
}
|
||||
|
||||
const text = await response.text();
|
||||
|
||||
// Check if we got HTML instead of JSON (blocked/redirected)
|
||||
if (text.startsWith('<!') || text.startsWith('<html')) {
|
||||
return { ...service, status: 'unknown', description: 'Blocked by service' };
|
||||
}
|
||||
|
||||
let data;
|
||||
try {
|
||||
data = JSON.parse(text);
|
||||
} catch {
|
||||
return { ...service, status: 'unknown', description: 'Invalid JSON response' };
|
||||
}
|
||||
|
||||
// Handle different API formats
|
||||
let status, description;
|
||||
|
||||
if (data.status?.indicator !== undefined) {
|
||||
// Standard Statuspage.io format
|
||||
status = normalizeStatus(data.status.indicator);
|
||||
description = data.status.description || '';
|
||||
} else if (data.status?.status) {
|
||||
// Slack format
|
||||
status = data.status.status === 'ok' ? 'operational' : 'degraded';
|
||||
description = data.status.description || '';
|
||||
} else if (data.page && data.status) {
|
||||
// Alternative Statuspage format - check if status object exists
|
||||
status = normalizeStatus(data.status.indicator || data.status.description);
|
||||
description = data.status.description || 'Status available';
|
||||
} else {
|
||||
status = 'unknown';
|
||||
description = 'Unknown format';
|
||||
}
|
||||
|
||||
return { ...service, status, description };
|
||||
} catch (error) {
|
||||
return { ...service, status: 'unknown', description: error.message || 'Request failed' };
|
||||
}
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const category = url.searchParams.get('category'); // cloud, dev, comm, ai, saas, or all
|
||||
|
||||
let servicesToCheck = SERVICES;
|
||||
if (category && category !== 'all') {
|
||||
servicesToCheck = SERVICES.filter(s => s.category === category);
|
||||
}
|
||||
|
||||
// Check all services in parallel
|
||||
const results = await Promise.all(servicesToCheck.map(checkStatusPage));
|
||||
|
||||
// Sort by status (outages first, then degraded, then operational)
|
||||
const statusOrder = { outage: 0, degraded: 1, unknown: 2, operational: 3 };
|
||||
results.sort((a, b) => statusOrder[a.status] - statusOrder[b.status]);
|
||||
|
||||
const summary = {
|
||||
operational: results.filter(r => r.status === 'operational').length,
|
||||
degraded: results.filter(r => r.status === 'degraded').length,
|
||||
outage: results.filter(r => r.status === 'outage').length,
|
||||
unknown: results.filter(r => r.status === 'unknown').length,
|
||||
};
|
||||
|
||||
return new Response(JSON.stringify({
|
||||
success: true,
|
||||
timestamp: new Date().toISOString(),
|
||||
summary,
|
||||
services: results.map(r => ({
|
||||
id: r.id,
|
||||
name: r.name,
|
||||
category: r.category,
|
||||
status: r.status,
|
||||
description: r.description,
|
||||
})),
|
||||
}), {
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30', // 1 min cache
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,130 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_TTL = 120;
|
||||
let cachedResponse = null;
|
||||
let cacheTimestamp = 0;
|
||||
|
||||
const DEFAULT_COINS = 'tether,usd-coin,dai,first-digital-usd,ethena-usde';
|
||||
|
||||
function buildFallbackResult() {
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
summary: {
|
||||
totalMarketCap: 0,
|
||||
totalVolume24h: 0,
|
||||
coinCount: 0,
|
||||
depeggedCount: 0,
|
||||
healthStatus: 'UNAVAILABLE',
|
||||
},
|
||||
stablecoins: [],
|
||||
unavailable: true,
|
||||
};
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: cors });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Forbidden' }), { status: 403, headers: { ...cors, 'Content-Type': 'application/json' } });
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
if (cachedResponse && now - cacheTimestamp < CACHE_TTL * 1000) {
|
||||
return new Response(JSON.stringify(cachedResponse), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=300` },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const rawCoins = url.searchParams.get('coins') || DEFAULT_COINS;
|
||||
const coins = rawCoins.split(',').filter(c => /^[a-z0-9-]+$/.test(c)).join(',') || DEFAULT_COINS;
|
||||
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const id = setTimeout(() => controller.abort(), 10000);
|
||||
|
||||
const apiUrl = `https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&ids=${coins}&order=market_cap_desc&sparkline=false&price_change_percentage=7d`;
|
||||
const res = await fetch(apiUrl, {
|
||||
signal: controller.signal,
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
clearTimeout(id);
|
||||
|
||||
if (res.status === 429) {
|
||||
if (cachedResponse) {
|
||||
return new Response(JSON.stringify(cachedResponse), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
|
||||
});
|
||||
}
|
||||
return new Response(JSON.stringify({ error: 'Rate limited', timestamp: new Date().toISOString() }), {
|
||||
status: 429,
|
||||
headers: { ...cors, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (!res.ok) throw new Error(`CoinGecko HTTP ${res.status}`);
|
||||
|
||||
const data = await res.json();
|
||||
|
||||
const stablecoins = data.map(coin => {
|
||||
const price = coin.current_price || 0;
|
||||
const deviation = Math.abs(price - 1.0);
|
||||
let pegStatus;
|
||||
if (deviation <= 0.005) pegStatus = 'ON PEG';
|
||||
else if (deviation <= 0.01) pegStatus = 'SLIGHT DEPEG';
|
||||
else pegStatus = 'DEPEGGED';
|
||||
|
||||
return {
|
||||
id: coin.id,
|
||||
symbol: (coin.symbol || '').toUpperCase(),
|
||||
name: coin.name,
|
||||
price,
|
||||
deviation: +(deviation * 100).toFixed(3),
|
||||
pegStatus,
|
||||
marketCap: coin.market_cap || 0,
|
||||
volume24h: coin.total_volume || 0,
|
||||
change24h: coin.price_change_percentage_24h || 0,
|
||||
change7d: coin.price_change_percentage_7d_in_currency || 0,
|
||||
image: coin.image,
|
||||
};
|
||||
});
|
||||
|
||||
const totalMarketCap = stablecoins.reduce((sum, c) => sum + c.marketCap, 0);
|
||||
const totalVolume24h = stablecoins.reduce((sum, c) => sum + c.volume24h, 0);
|
||||
const depeggedCount = stablecoins.filter(c => c.pegStatus === 'DEPEGGED').length;
|
||||
|
||||
const result = {
|
||||
timestamp: new Date().toISOString(),
|
||||
summary: {
|
||||
totalMarketCap,
|
||||
totalVolume24h,
|
||||
coinCount: stablecoins.length,
|
||||
depeggedCount,
|
||||
healthStatus: depeggedCount === 0 ? 'HEALTHY' : depeggedCount === 1 ? 'CAUTION' : 'WARNING',
|
||||
},
|
||||
stablecoins,
|
||||
};
|
||||
|
||||
cachedResponse = result;
|
||||
cacheTimestamp = now;
|
||||
|
||||
return new Response(JSON.stringify(result), {
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': `public, s-maxage=${CACHE_TTL}, stale-while-revalidate=300` },
|
||||
});
|
||||
} catch (err) {
|
||||
const fallback = cachedResponse || buildFallbackResult();
|
||||
cachedResponse = fallback;
|
||||
cacheTimestamp = now;
|
||||
return new Response(JSON.stringify(fallback), {
|
||||
status: 200,
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=30, s-maxage=60, stale-while-revalidate=30' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,172 +0,0 @@
|
||||
/**
|
||||
* Stock Market Index Endpoint
|
||||
* Fetches weekly % change for a country's primary stock index via Yahoo Finance
|
||||
* Redis cached (1h TTL)
|
||||
*/
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const CACHE_TTL_SECONDS = 3600; // 1 hour
|
||||
const CACHE_VERSION = 'stock-v1';
|
||||
|
||||
const COUNTRY_INDEX = {
|
||||
US: { symbol: '^GSPC', name: 'S&P 500' },
|
||||
GB: { symbol: '^FTSE', name: 'FTSE 100' },
|
||||
DE: { symbol: '^GDAXI', name: 'DAX' },
|
||||
FR: { symbol: '^FCHI', name: 'CAC 40' },
|
||||
JP: { symbol: '^N225', name: 'Nikkei 225' },
|
||||
CN: { symbol: '000001.SS', name: 'SSE Composite' },
|
||||
HK: { symbol: '^HSI', name: 'Hang Seng' },
|
||||
IN: { symbol: '^BSESN', name: 'BSE Sensex' },
|
||||
KR: { symbol: '^KS11', name: 'KOSPI' },
|
||||
TW: { symbol: '^TWII', name: 'TAIEX' },
|
||||
AU: { symbol: '^AXJO', name: 'ASX 200' },
|
||||
BR: { symbol: '^BVSP', name: 'Bovespa' },
|
||||
CA: { symbol: '^GSPTSE', name: 'TSX Composite' },
|
||||
MX: { symbol: '^MXX', name: 'IPC Mexico' },
|
||||
AR: { symbol: '^MERV', name: 'MERVAL' },
|
||||
RU: { symbol: 'IMOEX.ME', name: 'MOEX' },
|
||||
ZA: { symbol: '^J203.JO', name: 'JSE All Share' },
|
||||
SA: { symbol: '^TASI.SR', name: 'Tadawul' },
|
||||
AE: { symbol: 'DFMGI.AE', name: 'DFM General' },
|
||||
IL: { symbol: '^TA125.TA', name: 'TA-125' },
|
||||
TR: { symbol: 'XU100.IS', name: 'BIST 100' },
|
||||
PL: { symbol: '^WIG20', name: 'WIG 20' },
|
||||
NL: { symbol: '^AEX', name: 'AEX' },
|
||||
CH: { symbol: '^SSMI', name: 'SMI' },
|
||||
ES: { symbol: '^IBEX', name: 'IBEX 35' },
|
||||
IT: { symbol: 'FTSEMIB.MI', name: 'FTSE MIB' },
|
||||
SE: { symbol: '^OMX', name: 'OMX Stockholm 30' },
|
||||
NO: { symbol: '^OSEAX', name: 'Oslo All Share' },
|
||||
SG: { symbol: '^STI', name: 'STI' },
|
||||
TH: { symbol: '^SET.BK', name: 'SET' },
|
||||
MY: { symbol: '^KLSE', name: 'KLCI' },
|
||||
ID: { symbol: '^JKSE', name: 'Jakarta Composite' },
|
||||
PH: { symbol: 'PSEI.PS', name: 'PSEi' },
|
||||
NZ: { symbol: '^NZ50', name: 'NZX 50' },
|
||||
EG: { symbol: '^EGX30.CA', name: 'EGX 30' },
|
||||
CL: { symbol: '^IPSA', name: 'IPSA' },
|
||||
PE: { symbol: '^SPBLPGPT', name: 'S&P Lima' },
|
||||
AT: { symbol: '^ATX', name: 'ATX' },
|
||||
BE: { symbol: '^BFX', name: 'BEL 20' },
|
||||
FI: { symbol: '^OMXH25', name: 'OMX Helsinki 25' },
|
||||
DK: { symbol: '^OMXC25', name: 'OMX Copenhagen 25' },
|
||||
IE: { symbol: '^ISEQ', name: 'ISEQ Overall' },
|
||||
PT: { symbol: '^PSI20', name: 'PSI 20' },
|
||||
CZ: { symbol: '^PX', name: 'PX Prague' },
|
||||
HU: { symbol: '^BUX', name: 'BUX' },
|
||||
};
|
||||
|
||||
export default async function handler(request) {
|
||||
const cors = getCorsHeaders(request);
|
||||
if (request.method === 'OPTIONS') return new Response(null, { status: 204, headers: cors });
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
if (request.method !== 'GET') {
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const code = (url.searchParams.get('code') || '').toUpperCase();
|
||||
|
||||
if (!code) {
|
||||
return new Response(JSON.stringify({ error: 'code parameter required' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const index = COUNTRY_INDEX[code];
|
||||
if (!index) {
|
||||
return new Response(JSON.stringify({ error: 'No stock index for country', code, available: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const cacheKey = `${CACHE_VERSION}:${code}`;
|
||||
const cached = await getCachedJson(cacheKey);
|
||||
if (cached && typeof cached === 'object' && cached.indexName) {
|
||||
return new Response(JSON.stringify({ ...cached, cached: true }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const encodedSymbol = encodeURIComponent(index.symbol);
|
||||
// Use 1mo range to handle markets with different trading weeks (e.g. Sun-Thu Middle East)
|
||||
const yahooUrl = `https://query1.finance.yahoo.com/v8/finance/chart/${encodedSymbol}?range=1mo&interval=1d`;
|
||||
|
||||
const res = await fetch(yahooUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)',
|
||||
},
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
console.error('[StockIndex] Yahoo error:', res.status, index.symbol);
|
||||
return new Response(JSON.stringify({ error: 'Upstream error', available: false }), {
|
||||
status: 502,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const data = await res.json();
|
||||
const result = data?.chart?.result?.[0];
|
||||
if (!result) {
|
||||
return new Response(JSON.stringify({ error: 'No data', available: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
const allCloses = result.indicators?.quote?.[0]?.close?.filter(v => v != null);
|
||||
if (!allCloses || allCloses.length < 2) {
|
||||
return new Response(JSON.stringify({ error: 'Insufficient data', available: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
// Take last ~5 trading days worth of data
|
||||
const closes = allCloses.slice(-6);
|
||||
const latest = closes[closes.length - 1];
|
||||
const oldest = closes[0];
|
||||
const weekChange = ((latest - oldest) / oldest) * 100;
|
||||
const meta = result.meta || {};
|
||||
|
||||
const payload = {
|
||||
available: true,
|
||||
code,
|
||||
symbol: index.symbol,
|
||||
indexName: index.name,
|
||||
price: latest.toFixed(2),
|
||||
weekChangePercent: weekChange.toFixed(2),
|
||||
currency: meta.currency || 'USD',
|
||||
fetchedAt: new Date().toISOString(),
|
||||
};
|
||||
|
||||
await setCachedJson(cacheKey, payload, CACHE_TTL_SECONDS);
|
||||
|
||||
return new Response(JSON.stringify(payload), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[StockIndex] Error:', err);
|
||||
return new Response(JSON.stringify({ error: 'Internal error', available: false }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
/**
|
||||
* Story Page for Social Crawlers
|
||||
* Returns HTML with proper og:image and twitter:card meta tags.
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
// Telegram feed proxy (web)
|
||||
// Fetches Telegram Early Signals from the Railway relay (stateful MTProto lives there).
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
async function fetchWithTimeout(url, options, timeoutMs = 25000) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
try {
|
||||
return await fetch(url, { ...options, signal: controller.signal });
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: cors });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
|
||||
let relay = process.env.WS_RELAY_URL;
|
||||
if (!relay) {
|
||||
return new Response(JSON.stringify({ error: 'WS_RELAY_URL not configured' }), {
|
||||
status: 503,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
// Guard: WS_RELAY_URL should be HTTP(S) for server-side fetches.
|
||||
// If someone accidentally sets a ws:// or wss:// URL, normalize it.
|
||||
if (relay.startsWith('wss://')) relay = relay.replace('wss://', 'https://');
|
||||
if (relay.startsWith('ws://')) relay = relay.replace('ws://', 'http://');
|
||||
|
||||
const url = new URL(req.url);
|
||||
const limit = Math.max(1, Math.min(200, parseInt(url.searchParams.get('limit') || '50', 10) || 50));
|
||||
const topic = (url.searchParams.get('topic') || '').trim();
|
||||
const channel = (url.searchParams.get('channel') || '').trim();
|
||||
|
||||
const relayUrl = new URL('/telegram/feed', relay);
|
||||
relayUrl.searchParams.set('limit', String(limit));
|
||||
if (topic) relayUrl.searchParams.set('topic', topic);
|
||||
if (channel) relayUrl.searchParams.set('channel', channel);
|
||||
|
||||
try {
|
||||
const res = await fetchWithTimeout(relayUrl.toString(), {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
}, 25000);
|
||||
|
||||
const text = await res.text();
|
||||
return new Response(text, {
|
||||
status: res.status,
|
||||
headers: {
|
||||
'Content-Type': res.headers.get('content-type') || 'application/json',
|
||||
// Short cache. Telegram is near-real-time.
|
||||
'Cache-Control': 'public, max-age=10',
|
||||
...cors,
|
||||
},
|
||||
});
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
const isAbort = err && (err.name === 'AbortError' || /aborted/i.test(msg));
|
||||
return new Response(JSON.stringify({
|
||||
error: isAbort ? 'Telegram relay request timed out' : 'Telegram relay fetch failed',
|
||||
}), {
|
||||
status: isAbort ? 504 : 502,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,176 +0,0 @@
|
||||
/**
|
||||
* Temporal Baseline Anomaly Detection API
|
||||
* Stores and queries activity baselines using Welford's online algorithm
|
||||
* Backed by Upstash Redis for cross-user persistence
|
||||
*
|
||||
* GET ?type=military_flights®ion=global&count=47 — check anomaly
|
||||
* POST { updates: [{ type, region, count }] } — batch update baselines
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson, mget } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const BASELINE_TTL = 7776000; // 90 days in seconds
|
||||
const MIN_SAMPLES = 10;
|
||||
const Z_THRESHOLD_LOW = 1.5;
|
||||
const Z_THRESHOLD_MEDIUM = 2.0;
|
||||
const Z_THRESHOLD_HIGH = 3.0;
|
||||
|
||||
const VALID_TYPES = ['military_flights', 'vessels', 'protests', 'news', 'ais_gaps', 'satellite_fires'];
|
||||
|
||||
function makeKey(type, region, weekday, month) {
|
||||
return `baseline:${type}:${region}:${weekday}:${month}`;
|
||||
}
|
||||
|
||||
function getSeverity(zScore) {
|
||||
if (zScore >= Z_THRESHOLD_HIGH) return 'critical';
|
||||
if (zScore >= Z_THRESHOLD_MEDIUM) return 'high';
|
||||
if (zScore >= Z_THRESHOLD_LOW) return 'medium';
|
||||
return 'normal';
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
const corsHeaders = getCorsHeaders(request, 'GET, POST, OPTIONS');
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), {
|
||||
status: 403,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
if (request.method === 'GET') {
|
||||
return await handleGet(request);
|
||||
} else if (request.method === 'POST') {
|
||||
return await handlePost(request);
|
||||
}
|
||||
return new Response(JSON.stringify({ error: 'Method not allowed' }), {
|
||||
status: 405,
|
||||
headers: { ...corsHeaders, 'Content-Type': 'application/json' },
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('[TemporalBaseline] Error:', err);
|
||||
return new Response(JSON.stringify({ error: 'Internal error' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async function handleGet(request) {
|
||||
const { searchParams } = new URL(request.url);
|
||||
const type = searchParams.get('type');
|
||||
const region = searchParams.get('region') || 'global';
|
||||
const count = parseFloat(searchParams.get('count'));
|
||||
|
||||
if (!type || !VALID_TYPES.includes(type) || isNaN(count)) {
|
||||
return json({ error: 'Missing or invalid params: type, count required' }, 400);
|
||||
}
|
||||
|
||||
const now = new Date();
|
||||
const weekday = now.getUTCDay();
|
||||
const month = now.getUTCMonth() + 1;
|
||||
const key = makeKey(type, region, weekday, month);
|
||||
|
||||
const baseline = await getCachedJson(key);
|
||||
|
||||
if (!baseline || baseline.sampleCount < MIN_SAMPLES) {
|
||||
return json({
|
||||
anomaly: null,
|
||||
learning: true,
|
||||
sampleCount: baseline?.sampleCount || 0,
|
||||
samplesNeeded: MIN_SAMPLES,
|
||||
});
|
||||
}
|
||||
|
||||
const variance = Math.max(0, baseline.m2 / (baseline.sampleCount - 1));
|
||||
const stdDev = Math.sqrt(variance);
|
||||
const zScore = stdDev > 0 ? Math.abs((count - baseline.mean) / stdDev) : 0;
|
||||
const severity = getSeverity(zScore);
|
||||
const multiplier = baseline.mean > 0
|
||||
? Math.round((count / baseline.mean) * 100) / 100
|
||||
: count > 0 ? 999 : 1;
|
||||
|
||||
return json({
|
||||
anomaly: zScore >= Z_THRESHOLD_LOW ? {
|
||||
zScore: Math.round(zScore * 100) / 100,
|
||||
severity,
|
||||
multiplier,
|
||||
} : null,
|
||||
baseline: {
|
||||
mean: Math.round(baseline.mean * 100) / 100,
|
||||
stdDev: Math.round(stdDev * 100) / 100,
|
||||
sampleCount: baseline.sampleCount,
|
||||
},
|
||||
learning: false,
|
||||
});
|
||||
}
|
||||
|
||||
async function handlePost(request) {
|
||||
const contentLength = parseInt(request.headers.get('content-length') || '0', 10);
|
||||
if (contentLength > 51200) {
|
||||
return json({ error: 'Payload too large' }, 413);
|
||||
}
|
||||
|
||||
const body = await request.json();
|
||||
const updates = body?.updates;
|
||||
|
||||
if (!Array.isArray(updates) || updates.length === 0) {
|
||||
return json({ error: 'Body must have updates array' }, 400);
|
||||
}
|
||||
|
||||
const batch = updates.slice(0, 20);
|
||||
const now = new Date();
|
||||
const weekday = now.getUTCDay();
|
||||
const month = now.getUTCMonth() + 1;
|
||||
|
||||
const keys = batch.map(u => makeKey(u.type, u.region || 'global', weekday, month));
|
||||
const existing = await mget(...keys);
|
||||
|
||||
const writes = [];
|
||||
|
||||
for (let i = 0; i < batch.length; i++) {
|
||||
const { type, region = 'global', count } = batch[i];
|
||||
if (!VALID_TYPES.includes(type) || typeof count !== 'number' || isNaN(count)) continue;
|
||||
|
||||
const prev = existing[i] || { mean: 0, m2: 0, sampleCount: 0 };
|
||||
|
||||
const n = prev.sampleCount + 1;
|
||||
const delta = count - prev.mean;
|
||||
const newMean = prev.mean + delta / n;
|
||||
const delta2 = count - newMean;
|
||||
const newM2 = prev.m2 + delta * delta2;
|
||||
|
||||
writes.push(setCachedJson(keys[i], {
|
||||
mean: newMean,
|
||||
m2: newM2,
|
||||
sampleCount: n,
|
||||
lastUpdated: now.toISOString(),
|
||||
}, BASELINE_TTL));
|
||||
}
|
||||
|
||||
if (writes.length > 0) {
|
||||
await Promise.all(writes);
|
||||
}
|
||||
|
||||
return json({ updated: writes.length });
|
||||
}
|
||||
|
||||
function json(data, status = 200) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'no-store',
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -1,611 +0,0 @@
|
||||
/**
|
||||
* Theater Posture API - Aggregates military aircraft by theater
|
||||
* Caches results in Upstash Redis for cross-user efficiency
|
||||
* TTL: 5 minutes (matches OpenSky refresh rate)
|
||||
*/
|
||||
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const CACHE_TTL_SECONDS = 300; // 5 minutes
|
||||
const STALE_CACHE_TTL_SECONDS = 86400; // 24 hours - serve stale data when API is down
|
||||
const BACKUP_CACHE_TTL_SECONDS = 604800; // 7 days - last resort backup
|
||||
const CACHE_KEY = 'theater-posture:v4';
|
||||
const STALE_CACHE_KEY = 'theater-posture:stale:v4';
|
||||
const BACKUP_CACHE_KEY = 'theater-posture:backup:v4';
|
||||
|
||||
// Theater definitions (matches client-side POSTURE_THEATERS)
|
||||
const POSTURE_THEATERS = [
|
||||
{
|
||||
id: 'iran-theater',
|
||||
name: 'Iran Theater',
|
||||
shortName: 'IRAN',
|
||||
targetNation: 'Iran',
|
||||
bounds: { north: 42, south: 20, east: 65, west: 30 },
|
||||
thresholds: { elevated: 8, critical: 20 },
|
||||
strikeIndicators: { minTankers: 2, minAwacs: 1, minFighters: 5 },
|
||||
},
|
||||
{
|
||||
id: 'taiwan-theater',
|
||||
name: 'Taiwan Strait',
|
||||
shortName: 'TAIWAN',
|
||||
targetNation: 'Taiwan',
|
||||
bounds: { north: 30, south: 18, east: 130, west: 115 },
|
||||
thresholds: { elevated: 6, critical: 15 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 4 },
|
||||
},
|
||||
{
|
||||
id: 'baltic-theater',
|
||||
name: 'Baltic Theater',
|
||||
shortName: 'BALTIC',
|
||||
targetNation: null,
|
||||
bounds: { north: 65, south: 52, east: 32, west: 10 },
|
||||
thresholds: { elevated: 5, critical: 12 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'blacksea-theater',
|
||||
name: 'Black Sea',
|
||||
shortName: 'BLACK SEA',
|
||||
targetNation: null,
|
||||
bounds: { north: 48, south: 40, east: 42, west: 26 },
|
||||
thresholds: { elevated: 4, critical: 10 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'korea-theater',
|
||||
name: 'Korean Peninsula',
|
||||
shortName: 'KOREA',
|
||||
targetNation: 'North Korea',
|
||||
bounds: { north: 43, south: 33, east: 132, west: 124 },
|
||||
thresholds: { elevated: 5, critical: 12 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'south-china-sea',
|
||||
name: 'South China Sea',
|
||||
shortName: 'SCS',
|
||||
targetNation: null,
|
||||
bounds: { north: 25, south: 5, east: 121, west: 105 },
|
||||
thresholds: { elevated: 6, critical: 15 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 4 },
|
||||
},
|
||||
{
|
||||
id: 'east-med-theater',
|
||||
name: 'Eastern Mediterranean',
|
||||
shortName: 'E.MED',
|
||||
targetNation: null,
|
||||
bounds: { north: 37, south: 33, east: 37, west: 25 },
|
||||
thresholds: { elevated: 4, critical: 10 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'israel-gaza-theater',
|
||||
name: 'Israel/Gaza',
|
||||
shortName: 'GAZA',
|
||||
targetNation: 'Gaza',
|
||||
bounds: { north: 33, south: 29, east: 36, west: 33 },
|
||||
thresholds: { elevated: 3, critical: 8 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
{
|
||||
id: 'yemen-redsea-theater',
|
||||
name: 'Yemen/Red Sea',
|
||||
shortName: 'RED SEA',
|
||||
targetNation: 'Yemen',
|
||||
bounds: { north: 22, south: 11, east: 54, west: 32 },
|
||||
thresholds: { elevated: 4, critical: 10 },
|
||||
strikeIndicators: { minTankers: 1, minAwacs: 1, minFighters: 3 },
|
||||
},
|
||||
];
|
||||
|
||||
// Military hex database from ADS-B Exchange (updated daily at adsbexchange.com)
|
||||
// Contains ~20k verified military aircraft hex IDs
|
||||
import { MILITARY_HEX_LIST } from './data/military-hex-db.js';
|
||||
|
||||
// Create Set for O(1) lookup
|
||||
const MILITARY_HEX_SET = new Set(MILITARY_HEX_LIST.map(h => h.toLowerCase()));
|
||||
console.log(`[TheaterPosture] Loaded ${MILITARY_HEX_SET.size} military hex IDs from ADS-B Exchange`);
|
||||
|
||||
// Check if ICAO hex is in military database
|
||||
function isMilitaryHex(hexId) {
|
||||
if (!hexId) return false;
|
||||
// Handle both string and number, remove ~ prefix if present
|
||||
const cleanHex = String(hexId).replace(/^~/, '').toLowerCase();
|
||||
return MILITARY_HEX_SET.has(cleanHex);
|
||||
}
|
||||
|
||||
// Military callsign prefixes for identification
|
||||
const MILITARY_PREFIXES = [
|
||||
// US Military
|
||||
'RCH', 'REACH', 'MOOSE', 'EVAC', 'DUSTOFF', 'PEDRO', // Transport/medevac
|
||||
'DUKE', 'HAVOC', 'KNIFE', 'WARHAWK', 'VIPER', 'RAGE', 'FURY', // Fighters
|
||||
'SHELL', 'TEXACO', 'ARCO', 'ESSO', 'PETRO', // Tankers
|
||||
'SENTRY', 'AWACS', 'MAGIC', 'DISCO', 'DARKSTAR', // AWACS/ISR
|
||||
'COBRA', 'PYTHON', 'RAPTOR', 'EAGLE', 'HAWK', 'TALON', // Various
|
||||
'BOXER', 'OMNI', 'TOPCAT', 'SKULL', 'REAPER', 'HUNTER', // More callsigns
|
||||
'ARMY', 'NAVY', 'USAF', 'USMC', 'USCG', // Service prefixes
|
||||
'AE', 'CNV', 'PAT', 'SAM', 'EXEC', // Special missions
|
||||
'OPS', 'CTF', 'TF', // Operations/Task Force
|
||||
// NATO
|
||||
'NATO', 'GAF', 'RRF', 'RAF', 'FAF', 'IAF', 'RNLAF', 'BAF', 'DAF', 'HAF', 'PAF',
|
||||
'SWORD', 'LANCE', 'ARROW', 'SPARTAN', // NATO tactical
|
||||
// Middle East (avoid UAE - conflicts with Emirates airline)
|
||||
'RSAF', 'EMIRI', 'UAEAF', 'KAF', 'QAF', 'BAHAF', 'OMAAF', // Gulf states
|
||||
'IRIAF', 'IRG', 'IRGC', // Iran (IAF already in NATO section covers Israel)
|
||||
'TAF', 'TUAF', // Turkey
|
||||
// Russia
|
||||
'RSD', 'RF', 'RFF', 'VKS',
|
||||
// China (NOTE: CCA is Air China airline, not military)
|
||||
'CHN', 'PLAAF', 'PLA',
|
||||
];
|
||||
|
||||
// Airline ICAO codes to exclude from military detection (Set for O(1) lookup)
|
||||
const AIRLINE_CODES = new Set([
|
||||
// Middle East
|
||||
'SVA', 'QTR', 'THY', 'UAE', 'ETD', 'GFA', 'MEA', 'RJA', 'KAC', 'ELY',
|
||||
'IAW', 'IRA', 'MSR', 'SYR', 'PGT', 'AXB', 'FDB', 'KNE', 'FAD', 'ADY', 'OMA',
|
||||
'ABQ', 'ABY', 'NIA', 'FJA', 'SWR', 'HZA', 'OMS', 'EGF', 'NOS', 'SXD',
|
||||
// Europe
|
||||
'BAW', 'AFR', 'DLH', 'KLM', 'AUA', 'SAS', 'FIN', 'LOT', 'AZA', 'TAP', 'IBE',
|
||||
'VLG', 'RYR', 'EZY', 'WZZ', 'NOZ', 'BEL', 'AEE', 'ROT',
|
||||
// Asia
|
||||
'AIC', 'CPA', 'SIA', 'MAS', 'THA', 'VNM', 'JAL', 'ANA', 'KAL', 'AAR', 'EVA',
|
||||
'CAL', 'CCA', 'CES', 'CSN', 'HDA', 'CHH', 'CXA', 'GIA', 'PAL', 'SLK',
|
||||
// Americas
|
||||
'AAL', 'DAL', 'UAL', 'SWA', 'JBU', 'FFT', 'ASA', 'NKS', 'WJA', 'ACA',
|
||||
// Cargo
|
||||
'FDX', 'UPS', 'GTI', 'ABW', 'CLX', 'MPH',
|
||||
// Generic
|
||||
'AIR', 'SKY', 'JET',
|
||||
]);
|
||||
|
||||
// Aircraft type detection from callsign patterns
|
||||
function detectAircraftType(callsign) {
|
||||
if (!callsign) return 'unknown';
|
||||
const cs = callsign.toUpperCase().trim();
|
||||
|
||||
// Tankers
|
||||
if (/^(SHELL|TEXACO|ARCO|ESSO|PETRO)/.test(cs)) return 'tanker';
|
||||
if (/^(KC|STRAT)/.test(cs)) return 'tanker';
|
||||
|
||||
// AWACS
|
||||
if (/^(SENTRY|AWACS|MAGIC|DISCO|DARKSTAR)/.test(cs)) return 'awacs';
|
||||
if (/^(E3|E8|E6)/.test(cs)) return 'awacs';
|
||||
|
||||
// Transport
|
||||
if (/^(RCH|REACH|MOOSE|EVAC|DUSTOFF)/.test(cs)) return 'transport';
|
||||
if (/^(C17|C5|C130|C40)/.test(cs)) return 'transport';
|
||||
|
||||
// Reconnaissance
|
||||
if (/^(HOMER|OLIVE|JAKE|PSEUDO|GORDO)/.test(cs)) return 'reconnaissance';
|
||||
if (/^(RC|U2|SR)/.test(cs)) return 'reconnaissance';
|
||||
|
||||
// Drones/UAVs
|
||||
if (/^(RQ|MQ|REAPER|PREDATOR|GLOBAL)/.test(cs)) return 'drone';
|
||||
|
||||
// Bombers
|
||||
if (/^(DEATH|BONE|DOOM)/.test(cs)) return 'bomber';
|
||||
if (/^(B52|B1|B2)/.test(cs)) return 'bomber';
|
||||
|
||||
// Default to unknown for unrecognized military aircraft
|
||||
return 'unknown';
|
||||
}
|
||||
|
||||
// Check if callsign is military
|
||||
function isMilitaryCallsign(callsign) {
|
||||
if (!callsign) return false;
|
||||
const cs = callsign.toUpperCase().trim();
|
||||
|
||||
// Check prefixes
|
||||
for (const prefix of MILITARY_PREFIXES) {
|
||||
if (cs.startsWith(prefix)) return true;
|
||||
}
|
||||
|
||||
// Check patterns - tactical callsigns (word + small number)
|
||||
// DUKE01, VIPER12, RAGE1 but NOT airline codes like PGT5873, IAW9011
|
||||
if (/^[A-Z]{4,}\d{1,3}$/.test(cs)) return true;
|
||||
|
||||
// Short tactical: 3 letters + 1-2 digits (but exclude common airlines)
|
||||
// This catches OPS4, CTF01, TF12 but blocks SVA12, QTR76, etc.
|
||||
if (/^[A-Z]{3}\d{1,2}$/.test(cs)) {
|
||||
const prefix = cs.slice(0, 3);
|
||||
if (!AIRLINE_CODES.has(prefix)) return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
// Fetch military flights from OpenSky
|
||||
async function fetchMilitaryFlights() {
|
||||
const isSidecar = (process.env.LOCAL_API_MODE || '').includes('sidecar');
|
||||
// Desktop sidecar: fetch directly from OpenSky (single user, no rate limit concern)
|
||||
// Cloud: use Railway relay to avoid OpenSky rate limits across many users
|
||||
const baseUrl = isSidecar
|
||||
? 'https://opensky-network.org/api/states/all'
|
||||
: (process.env.WS_RELAY_URL ? process.env.WS_RELAY_URL + '/opensky' : null);
|
||||
|
||||
if (!baseUrl) return [];
|
||||
|
||||
// Fetch global data with 20s timeout (Edge has 25s limit)
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 20000);
|
||||
|
||||
try {
|
||||
console.log('[TheaterPosture] Fetching from:', baseUrl);
|
||||
const response = await fetch(baseUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'Mozilla/5.0 WorldMonitor/1.0',
|
||||
},
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`OpenSky API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
if (!data.states) return [];
|
||||
|
||||
// Filter and transform to military flights
|
||||
const flights = [];
|
||||
for (const state of data.states) {
|
||||
const [icao24, callsign, , , , lon, lat, altitude, onGround, velocity, heading] = state;
|
||||
|
||||
// Skip if no position
|
||||
if (lat == null || lon == null) continue;
|
||||
|
||||
// Skip if on ground
|
||||
if (onGround) continue;
|
||||
|
||||
// Check if military (by callsign OR hex range)
|
||||
const isMilitary = isMilitaryCallsign(callsign) || isMilitaryHex(icao24);
|
||||
if (!isMilitary) continue;
|
||||
|
||||
flights.push({
|
||||
id: icao24,
|
||||
callsign: callsign?.trim() || '',
|
||||
lat,
|
||||
lon,
|
||||
altitude: altitude || 0,
|
||||
heading: heading || 0,
|
||||
speed: velocity || 0,
|
||||
aircraftType: detectAircraftType(callsign),
|
||||
operator: 'unknown',
|
||||
militaryHex: isMilitaryHex(icao24),
|
||||
});
|
||||
}
|
||||
|
||||
return flights;
|
||||
} catch (err) {
|
||||
if (err.name === 'AbortError') {
|
||||
throw new Error('OpenSky API timeout - try again');
|
||||
}
|
||||
throw err;
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch military flights from Wingbits (fallback when OpenSky fails)
|
||||
async function fetchMilitaryFlightsFromWingbits() {
|
||||
const apiKey = process.env.WINGBITS_API_KEY;
|
||||
if (!apiKey) {
|
||||
console.log('[TheaterPosture] Wingbits not configured, skipping fallback');
|
||||
return null;
|
||||
}
|
||||
|
||||
console.log('[TheaterPosture] Trying Wingbits fallback...');
|
||||
|
||||
// Build batch request for all theaters
|
||||
const areas = POSTURE_THEATERS.map(theater => ({
|
||||
alias: theater.id,
|
||||
by: 'box',
|
||||
la: (theater.bounds.north + theater.bounds.south) / 2,
|
||||
lo: (theater.bounds.east + theater.bounds.west) / 2,
|
||||
w: Math.abs(theater.bounds.east - theater.bounds.west) * 60, // degrees to nm
|
||||
h: Math.abs(theater.bounds.north - theater.bounds.south) * 60,
|
||||
unit: 'nm',
|
||||
}));
|
||||
|
||||
const controller = new AbortController();
|
||||
const timeoutId = setTimeout(() => controller.abort(), 15000);
|
||||
|
||||
try {
|
||||
const response = await fetch('https://customer-api.wingbits.com/v1/flights', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify(areas),
|
||||
signal: controller.signal,
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
console.warn('[TheaterPosture] Wingbits API error:', response.status);
|
||||
return null;
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
console.log('[TheaterPosture] Wingbits returned', data.length, 'theater results');
|
||||
|
||||
// Transform Wingbits data to our format
|
||||
// Wingbits uses short field names: h=icao24, f=flight, la=lat, lo=lon, ab=alt, th=heading, gs=speed
|
||||
const flights = [];
|
||||
const seenIds = new Set();
|
||||
|
||||
for (const areaResult of data) {
|
||||
// Batch response: each area result has flights in various possible formats
|
||||
const areaFlights = areaResult.flights || areaResult.data || areaResult || [];
|
||||
const flightList = Array.isArray(areaFlights) ? areaFlights : [];
|
||||
|
||||
for (const f of flightList) {
|
||||
// Get icao24 - Wingbits uses 'h' for hex ID
|
||||
const icao24 = f.h || f.icao24 || f.id;
|
||||
if (!icao24) continue;
|
||||
|
||||
// Skip duplicates (aircraft may appear in multiple theaters)
|
||||
if (seenIds.has(icao24)) continue;
|
||||
seenIds.add(icao24);
|
||||
|
||||
// Get callsign - Wingbits uses 'f' for flight
|
||||
const callsign = f.f || f.callsign || f.flight || '';
|
||||
|
||||
// Skip if not military (by callsign OR hex range)
|
||||
const isMilitary = isMilitaryCallsign(callsign) || isMilitaryHex(icao24);
|
||||
if (!isMilitary) continue;
|
||||
|
||||
flights.push({
|
||||
id: icao24,
|
||||
callsign: callsign.trim(),
|
||||
lat: f.la || f.latitude || f.lat,
|
||||
lon: f.lo || f.longitude || f.lon || f.lng,
|
||||
altitude: f.ab || f.altitude || f.alt || 0,
|
||||
heading: f.th || f.heading || f.track || 0,
|
||||
speed: f.gs || f.groundSpeed || f.speed || f.velocity || 0,
|
||||
aircraftType: detectAircraftType(callsign),
|
||||
operator: f.operator || 'unknown',
|
||||
source: 'wingbits',
|
||||
militaryHex: isMilitaryHex(icao24),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
console.log('[TheaterPosture] Wingbits: found', flights.length, 'military flights');
|
||||
return flights;
|
||||
} catch (err) {
|
||||
console.error('[TheaterPosture] Wingbits fetch error:', err.message);
|
||||
return null;
|
||||
} finally {
|
||||
clearTimeout(timeoutId);
|
||||
}
|
||||
}
|
||||
|
||||
// Calculate theater postures
|
||||
function calculatePostures(flights) {
|
||||
const summaries = [];
|
||||
|
||||
for (const theater of POSTURE_THEATERS) {
|
||||
// Filter flights within theater bounds
|
||||
const theaterFlights = flights.filter(f =>
|
||||
f.lat >= theater.bounds.south &&
|
||||
f.lat <= theater.bounds.north &&
|
||||
f.lon >= theater.bounds.west &&
|
||||
f.lon <= theater.bounds.east
|
||||
);
|
||||
|
||||
// Count by type
|
||||
const byType = {
|
||||
fighters: theaterFlights.filter(f => f.aircraftType === 'fighter').length,
|
||||
tankers: theaterFlights.filter(f => f.aircraftType === 'tanker').length,
|
||||
awacs: theaterFlights.filter(f => f.aircraftType === 'awacs').length,
|
||||
reconnaissance: theaterFlights.filter(f => f.aircraftType === 'reconnaissance').length,
|
||||
transport: theaterFlights.filter(f => f.aircraftType === 'transport').length,
|
||||
bombers: theaterFlights.filter(f => f.aircraftType === 'bomber').length,
|
||||
drones: theaterFlights.filter(f => f.aircraftType === 'drone').length,
|
||||
unknown: theaterFlights.filter(f => f.aircraftType === 'unknown').length,
|
||||
};
|
||||
|
||||
const total = Object.values(byType).reduce((a, b) => a + b, 0);
|
||||
|
||||
// Determine posture level
|
||||
const postureLevel = total >= theater.thresholds.critical ? 'critical' :
|
||||
total >= theater.thresholds.elevated ? 'elevated' : 'normal';
|
||||
|
||||
// Check strike capability
|
||||
const strikeCapable =
|
||||
byType.tankers >= theater.strikeIndicators.minTankers &&
|
||||
byType.awacs >= theater.strikeIndicators.minAwacs &&
|
||||
byType.fighters >= theater.strikeIndicators.minFighters;
|
||||
|
||||
// Build summary string
|
||||
const parts = [];
|
||||
if (byType.fighters > 0) parts.push(`${byType.fighters} fighters`);
|
||||
if (byType.tankers > 0) parts.push(`${byType.tankers} tankers`);
|
||||
if (byType.awacs > 0) parts.push(`${byType.awacs} AWACS`);
|
||||
if (byType.reconnaissance > 0) parts.push(`${byType.reconnaissance} recon`);
|
||||
if (byType.bombers > 0) parts.push(`${byType.bombers} bombers`);
|
||||
if (byType.transport > 0) parts.push(`${byType.transport} transport`);
|
||||
if (byType.drones > 0) parts.push(`${byType.drones} drones`);
|
||||
if (byType.unknown > 0) parts.push(`${byType.unknown} other`);
|
||||
const summary = parts.join(', ') || 'No military aircraft';
|
||||
|
||||
// Build headline
|
||||
const headline = postureLevel === 'critical'
|
||||
? `Critical military buildup - ${theater.name}`
|
||||
: postureLevel === 'elevated'
|
||||
? `Elevated military activity - ${theater.name}`
|
||||
: `Normal activity - ${theater.name}`;
|
||||
|
||||
// Build byOperator map for aircraft
|
||||
const byOperator = {};
|
||||
for (const f of theaterFlights) {
|
||||
const op = f.operator || 'unknown';
|
||||
byOperator[op] = (byOperator[op] || 0) + 1;
|
||||
}
|
||||
|
||||
summaries.push({
|
||||
theaterId: theater.id,
|
||||
theaterName: theater.name,
|
||||
shortName: theater.shortName,
|
||||
targetNation: theater.targetNation,
|
||||
// Aircraft
|
||||
fighters: byType.fighters,
|
||||
tankers: byType.tankers,
|
||||
awacs: byType.awacs,
|
||||
reconnaissance: byType.reconnaissance,
|
||||
transport: byType.transport,
|
||||
bombers: byType.bombers,
|
||||
drones: byType.drones,
|
||||
unknown: byType.unknown,
|
||||
totalAircraft: total,
|
||||
// Vessels (populated client-side)
|
||||
destroyers: 0,
|
||||
frigates: 0,
|
||||
carriers: 0,
|
||||
submarines: 0,
|
||||
patrol: 0,
|
||||
auxiliaryVessels: 0,
|
||||
totalVessels: 0,
|
||||
// By operator (aircraft + vessels added client-side)
|
||||
byOperator,
|
||||
// Metadata
|
||||
postureLevel,
|
||||
strikeCapable,
|
||||
trend: 'stable',
|
||||
changePercent: 0,
|
||||
summary,
|
||||
headline,
|
||||
centerLat: (theater.bounds.north + theater.bounds.south) / 2,
|
||||
centerLon: (theater.bounds.east + theater.bounds.west) / 2,
|
||||
bounds: theater.bounds,
|
||||
});
|
||||
}
|
||||
|
||||
return summaries;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
try {
|
||||
// Try to get from cache first
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (cached) {
|
||||
console.log('[TheaterPosture] Cache hit');
|
||||
return Response.json({
|
||||
...cached,
|
||||
cached: true,
|
||||
}, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// Fetch and calculate - try OpenSky first, then Wingbits fallback
|
||||
console.log('[TheaterPosture] Fetching fresh data...');
|
||||
let flights;
|
||||
let source = 'opensky';
|
||||
|
||||
try {
|
||||
flights = await fetchMilitaryFlights();
|
||||
} catch (openskyError) {
|
||||
console.warn('[TheaterPosture] OpenSky failed:', openskyError.message);
|
||||
console.log('[TheaterPosture] Trying Wingbits fallback...');
|
||||
|
||||
flights = await fetchMilitaryFlightsFromWingbits();
|
||||
if (flights && flights.length > 0) {
|
||||
source = 'wingbits';
|
||||
console.log('[TheaterPosture] Wingbits fallback succeeded:', flights.length, 'flights');
|
||||
} else {
|
||||
// Both failed, re-throw OpenSky error to trigger cache fallback
|
||||
throw openskyError;
|
||||
}
|
||||
}
|
||||
|
||||
const postures = calculatePostures(flights);
|
||||
|
||||
const result = {
|
||||
postures,
|
||||
totalFlights: flights.length,
|
||||
timestamp: new Date().toISOString(),
|
||||
cached: false,
|
||||
source, // 'opensky' or 'wingbits'
|
||||
};
|
||||
|
||||
// Cache the result (regular, stale, and long-term backup)
|
||||
await Promise.all([
|
||||
setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS),
|
||||
setCachedJson(STALE_CACHE_KEY, result, STALE_CACHE_TTL_SECONDS),
|
||||
setCachedJson(BACKUP_CACHE_KEY, result, BACKUP_CACHE_TTL_SECONDS),
|
||||
]);
|
||||
|
||||
return Response.json(result, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.warn('[TheaterPosture] Error:', error.message);
|
||||
|
||||
// Try to return cached data when API fails (stale first, then backup)
|
||||
const stale = await getCachedJson(STALE_CACHE_KEY);
|
||||
if (stale) {
|
||||
console.log('[TheaterPosture] Returning stale cached data (24h) due to API error');
|
||||
return Response.json({
|
||||
...stale,
|
||||
cached: true,
|
||||
stale: true,
|
||||
error: 'Using cached data - live feed temporarily unavailable',
|
||||
}, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
const backup = await getCachedJson(BACKUP_CACHE_KEY);
|
||||
if (backup) {
|
||||
console.log('[TheaterPosture] Returning backup cached data (7d) due to API error');
|
||||
return Response.json({
|
||||
...backup,
|
||||
cached: true,
|
||||
stale: true,
|
||||
error: 'Using backup data - live feed temporarily unavailable',
|
||||
}, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
// No cached data available - return error
|
||||
return Response.json({
|
||||
error: error.message,
|
||||
postures: [],
|
||||
timestamp: new Date().toISOString(),
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,237 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'ucdp:gedevents:v2';
|
||||
const CACHE_TTL_SECONDS = 6 * 60 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const UCDP_PAGE_SIZE = 1000;
|
||||
const MAX_PAGES = 12;
|
||||
const TRAILING_WINDOW_MS = 365 * 24 * 60 * 60 * 1000;
|
||||
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: 15,
|
||||
windowMs: 60 * 1000,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(data && typeof data === 'object' && Array.isArray(data.data));
|
||||
}
|
||||
|
||||
const VIOLENCE_TYPE_MAP = {
|
||||
1: 'state-based',
|
||||
2: 'non-state',
|
||||
3: 'one-sided',
|
||||
};
|
||||
|
||||
function parseDateMs(value) {
|
||||
if (!value) return NaN;
|
||||
return Date.parse(String(value));
|
||||
}
|
||||
|
||||
function getMaxDateMs(events) {
|
||||
let maxMs = NaN;
|
||||
for (const event of events) {
|
||||
const ms = parseDateMs(event?.date_start);
|
||||
if (!Number.isFinite(ms)) continue;
|
||||
if (!Number.isFinite(maxMs) || ms > maxMs) {
|
||||
maxMs = ms;
|
||||
}
|
||||
}
|
||||
return maxMs;
|
||||
}
|
||||
|
||||
function buildVersionCandidates() {
|
||||
const year = new Date().getFullYear() - 2000;
|
||||
return Array.from(new Set([
|
||||
`${year}.1`,
|
||||
`${year - 1}.1`,
|
||||
'25.1',
|
||||
'24.1',
|
||||
]));
|
||||
}
|
||||
|
||||
async function fetchGedPage(version, page) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), 8000);
|
||||
try {
|
||||
const response = await fetch(
|
||||
`https://ucdpapi.pcr.uu.se/api/gedevents/${version}?pagesize=${UCDP_PAGE_SIZE}&page=${page}`,
|
||||
{ headers: { Accept: 'application/json' }, signal: controller.signal }
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`UCDP GED API error (${version}, page ${page}): ${response.status}`);
|
||||
}
|
||||
return response.json();
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
async function discoverGedVersion() {
|
||||
const candidates = buildVersionCandidates();
|
||||
for (const version of candidates) {
|
||||
try {
|
||||
const page0 = await fetchGedPage(version, 0);
|
||||
if (Array.isArray(page0?.Result)) {
|
||||
return { version, page0 };
|
||||
}
|
||||
} catch {
|
||||
// Try the next version candidate.
|
||||
}
|
||||
}
|
||||
throw new Error('Unable to fetch UCDP GED metadata from known API versions');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed', data: [] }, {
|
||||
status: 405, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed', data: [] }, {
|
||||
status: 403, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited', data: [] }, {
|
||||
status: 429,
|
||||
headers: { ...corsHeaders, 'Retry-After': '60' },
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/ucdp-events', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/ucdp-events', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const { version, page0 } = await discoverGedVersion();
|
||||
const totalPages = Math.max(1, Number(page0?.TotalPages) || 1);
|
||||
const newestPage = totalPages - 1;
|
||||
|
||||
let allEvents = [];
|
||||
let latestDatasetMs = NaN;
|
||||
|
||||
for (let offset = 0; offset < MAX_PAGES && (newestPage - offset) >= 0; offset++) {
|
||||
const page = newestPage - offset;
|
||||
const rawData = page === 0 ? page0 : await fetchGedPage(version, page);
|
||||
const events = Array.isArray(rawData?.Result) ? rawData.Result : [];
|
||||
allEvents = allEvents.concat(events);
|
||||
|
||||
const pageMaxMs = getMaxDateMs(events);
|
||||
if (!Number.isFinite(latestDatasetMs) && Number.isFinite(pageMaxMs)) {
|
||||
latestDatasetMs = pageMaxMs;
|
||||
}
|
||||
|
||||
// Pages are ordered oldest->newest; once we are fully outside trailing window, stop.
|
||||
if (Number.isFinite(latestDatasetMs) && Number.isFinite(pageMaxMs)) {
|
||||
const cutoffMs = latestDatasetMs - TRAILING_WINDOW_MS;
|
||||
if (pageMaxMs < cutoffMs) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const sanitized = allEvents
|
||||
.filter((event) => {
|
||||
if (!Number.isFinite(latestDatasetMs)) return true;
|
||||
const eventMs = parseDateMs(event?.date_start);
|
||||
if (!Number.isFinite(eventMs)) return false;
|
||||
return eventMs >= (latestDatasetMs - TRAILING_WINDOW_MS);
|
||||
})
|
||||
.map(e => ({
|
||||
id: String(e.id || ''),
|
||||
date_start: e.date_start || '',
|
||||
date_end: e.date_end || '',
|
||||
latitude: Number(e.latitude) || 0,
|
||||
longitude: Number(e.longitude) || 0,
|
||||
country: e.country || '',
|
||||
side_a: (e.side_a || '').substring(0, 200),
|
||||
side_b: (e.side_b || '').substring(0, 200),
|
||||
deaths_best: Number(e.best) || 0,
|
||||
deaths_low: Number(e.low) || 0,
|
||||
deaths_high: Number(e.high) || 0,
|
||||
type_of_violence: VIOLENCE_TYPE_MAP[e.type_of_violence] || 'state-based',
|
||||
source_original: (e.source_original || '').substring(0, 300),
|
||||
}))
|
||||
.sort((a, b) => {
|
||||
const bMs = parseDateMs(b.date_start);
|
||||
const aMs = parseDateMs(a.date_start);
|
||||
return (Number.isFinite(bMs) ? bMs : 0) - (Number.isFinite(aMs) ? aMs : 0);
|
||||
});
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: sanitized.length,
|
||||
data: sanitized,
|
||||
version,
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/ucdp-events', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/ucdp-events', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/ucdp-events', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, data: [] }, {
|
||||
status: 500, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
-150
@@ -1,150 +0,0 @@
|
||||
// UCDP (Uppsala Conflict Data Program) proxy
|
||||
// Returns conflict classification per country with intensity levels
|
||||
// No auth required - public API
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const CACHE_KEY = 'ucdp:country-conflicts:v2';
|
||||
const CACHE_TTL_SECONDS = 24 * 60 * 60; // 24 hours (annual data)
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
const RESPONSE_CACHE_CONTROL = 'public, max-age=3600';
|
||||
|
||||
// In-memory fallback when Redis is unavailable.
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(
|
||||
data &&
|
||||
typeof data === 'object' &&
|
||||
Array.isArray(data.conflicts)
|
||||
);
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/ucdp', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'REDIS-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/ucdp', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'MEMORY-HIT',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
// Fetch all pages of conflicts
|
||||
let allConflicts = [];
|
||||
let page = 0;
|
||||
let totalPages = 1;
|
||||
|
||||
while (page < totalPages) {
|
||||
const response = await fetch(`https://ucdpapi.pcr.uu.se/api/ucdpprioconflict/24.1?pagesize=100&page=${page}`, {
|
||||
headers: { 'Accept': 'application/json' },
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`UCDP API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const rawData = await response.json();
|
||||
totalPages = rawData.TotalPages || 1;
|
||||
const conflicts = rawData.Result || [];
|
||||
allConflicts = allConflicts.concat(conflicts);
|
||||
page++;
|
||||
}
|
||||
|
||||
// Fields are snake_case: conflict_id, location, side_a, side_b, year, intensity_level, type_of_conflict
|
||||
const countryConflicts = {};
|
||||
for (const c of allConflicts) {
|
||||
const name = c.location || '';
|
||||
const year = parseInt(c.year, 10) || 0;
|
||||
const intensity = parseInt(c.intensity_level, 10) || 0;
|
||||
|
||||
const entry = {
|
||||
conflictId: parseInt(c.conflict_id, 10) || 0,
|
||||
conflictName: c.side_b || '',
|
||||
location: name,
|
||||
year,
|
||||
intensityLevel: intensity,
|
||||
typeOfConflict: parseInt(c.type_of_conflict, 10) || 0,
|
||||
startDate: c.start_date,
|
||||
startDate2: c.start_date2,
|
||||
sideA: c.side_a,
|
||||
sideB: c.side_b,
|
||||
region: c.region,
|
||||
};
|
||||
|
||||
// Keep most recent / highest intensity per location
|
||||
if (!countryConflicts[name] || year > countryConflicts[name].year ||
|
||||
(year === countryConflicts[name].year && intensity > countryConflicts[name].intensityLevel)) {
|
||||
countryConflicts[name] = entry;
|
||||
}
|
||||
}
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
count: Object.keys(countryConflicts).length,
|
||||
conflicts: Object.values(countryConflicts),
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/ucdp', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': RESPONSE_CACHE_CONTROL,
|
||||
'X-Cache': 'MISS',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/ucdp', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: {
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120',
|
||||
'X-Cache': 'STALE',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/ucdp', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, conflicts: [] }, {
|
||||
status: 500,
|
||||
headers: { ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,270 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const CACHE_KEY = 'unhcr:population:v2';
|
||||
const CACHE_TTL_SECONDS = 24 * 60 * 60;
|
||||
const CACHE_TTL_MS = CACHE_TTL_SECONDS * 1000;
|
||||
|
||||
let fallbackCache = { data: null, timestamp: 0 };
|
||||
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: 20,
|
||||
windowMs: 60 * 1000,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
function isValidResult(data) {
|
||||
return Boolean(data && typeof data === 'object' && Array.isArray(data.countries));
|
||||
}
|
||||
|
||||
const COUNTRY_CENTROIDS = {
|
||||
AFG: [33.9, 67.7], SYR: [35.0, 38.0], UKR: [48.4, 31.2], SDN: [15.5, 32.5],
|
||||
SSD: [6.9, 31.3], SOM: [5.2, 46.2], COD: [-4.0, 21.8], MMR: [19.8, 96.7],
|
||||
YEM: [15.6, 48.5], ETH: [9.1, 40.5], VEN: [6.4, -66.6], IRQ: [33.2, 43.7],
|
||||
COL: [4.6, -74.1], NGA: [9.1, 7.5], PSE: [31.9, 35.2], TUR: [39.9, 32.9],
|
||||
DEU: [51.2, 10.4], PAK: [30.4, 69.3], UGA: [1.4, 32.3], BGD: [23.7, 90.4],
|
||||
KEN: [0.0, 38.0], TCD: [15.5, 19.0], JOR: [31.0, 36.0], LBN: [33.9, 35.5],
|
||||
EGY: [26.8, 30.8], IRN: [32.4, 53.7], TZA: [-6.4, 34.9], RWA: [-1.9, 29.9],
|
||||
CMR: [7.4, 12.4], MLI: [17.6, -4.0], BFA: [12.3, -1.6], NER: [17.6, 8.1],
|
||||
CAF: [6.6, 20.9], MOZ: [-18.7, 35.5], USA: [37.1, -95.7], FRA: [46.2, 2.2],
|
||||
GBR: [55.4, -3.4], IND: [20.6, 79.0], CHN: [35.9, 104.2], RUS: [61.5, 105.3],
|
||||
};
|
||||
|
||||
async function fetchUnhcrYearItems(year) {
|
||||
const limit = 10000;
|
||||
const maxPageGuard = 25;
|
||||
const items = [];
|
||||
|
||||
for (let page = 1; page <= maxPageGuard; page++) {
|
||||
const response = await fetch(
|
||||
`https://api.unhcr.org/population/v1/population/?year=${year}&limit=${limit}&page=${page}`,
|
||||
{ headers: { Accept: 'application/json' } }
|
||||
);
|
||||
|
||||
if (!response.ok) return null;
|
||||
|
||||
const data = await response.json();
|
||||
const pageItems = Array.isArray(data.items) ? data.items : [];
|
||||
if (pageItems.length === 0) break;
|
||||
items.push(...pageItems);
|
||||
|
||||
const maxPages = Number(data.maxPages);
|
||||
if (Number.isFinite(maxPages) && maxPages > 0) {
|
||||
if (page >= maxPages) break;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (pageItems.length < limit) break;
|
||||
}
|
||||
|
||||
return items;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited' }, {
|
||||
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
|
||||
});
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
const cached = await getCachedJson(CACHE_KEY);
|
||||
if (isValidResult(cached)) {
|
||||
recordCacheTelemetry('/api/unhcr-population', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'REDIS-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidResult(fallbackCache.data) && now - fallbackCache.timestamp < CACHE_TTL_MS) {
|
||||
recordCacheTelemetry('/api/unhcr-population', 'MEMORY-HIT');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MEMORY-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const currentYear = new Date().getFullYear();
|
||||
let rawItems = [];
|
||||
let dataYearUsed = null;
|
||||
|
||||
for (let year = currentYear; year >= currentYear - 2; year--) {
|
||||
const yearItems = await fetchUnhcrYearItems(year);
|
||||
if (!yearItems) {
|
||||
continue;
|
||||
}
|
||||
rawItems = yearItems;
|
||||
if (rawItems.length > 0) {
|
||||
dataYearUsed = year;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
const byOrigin = {};
|
||||
const byAsylum = {};
|
||||
const flowMap = {};
|
||||
let totalRefugees = 0, totalAsylumSeekers = 0, totalIdps = 0, totalStateless = 0;
|
||||
|
||||
for (const item of rawItems) {
|
||||
const originCode = item.coo_iso || '';
|
||||
const asylumCode = item.coa_iso || '';
|
||||
const refugees = Number(item.refugees) || 0;
|
||||
const asylumSeekers = Number(item.asylum_seekers) || 0;
|
||||
const idps = Number(item.idps) || 0;
|
||||
const stateless = Number(item.stateless) || 0;
|
||||
|
||||
totalRefugees += refugees;
|
||||
totalAsylumSeekers += asylumSeekers;
|
||||
totalIdps += idps;
|
||||
totalStateless += stateless;
|
||||
|
||||
if (originCode) {
|
||||
if (!byOrigin[originCode]) byOrigin[originCode] = { refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, name: item.coo_name || originCode };
|
||||
byOrigin[originCode].refugees += refugees;
|
||||
byOrigin[originCode].asylumSeekers += asylumSeekers;
|
||||
byOrigin[originCode].idps += idps;
|
||||
byOrigin[originCode].stateless += stateless;
|
||||
}
|
||||
|
||||
if (asylumCode) {
|
||||
if (!byAsylum[asylumCode]) byAsylum[asylumCode] = { refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, name: item.coa_name || asylumCode };
|
||||
byAsylum[asylumCode].refugees += refugees;
|
||||
byAsylum[asylumCode].asylumSeekers += asylumSeekers;
|
||||
}
|
||||
|
||||
if (originCode && asylumCode && refugees > 0) {
|
||||
const flowKey = `${originCode}->${asylumCode}`;
|
||||
if (!flowMap[flowKey]) {
|
||||
flowMap[flowKey] = {
|
||||
originCode, originName: item.coo_name || originCode,
|
||||
asylumCode, asylumName: item.coa_name || asylumCode,
|
||||
refugees: 0,
|
||||
};
|
||||
}
|
||||
flowMap[flowKey].refugees += refugees;
|
||||
}
|
||||
}
|
||||
|
||||
const countries = {};
|
||||
for (const [code, data] of Object.entries(byOrigin)) {
|
||||
const centroid = COUNTRY_CENTROIDS[code];
|
||||
countries[code] = {
|
||||
code, name: data.name,
|
||||
refugees: data.refugees, asylumSeekers: data.asylumSeekers,
|
||||
idps: data.idps, stateless: data.stateless,
|
||||
totalDisplaced: data.refugees + data.asylumSeekers + data.idps + data.stateless,
|
||||
hostRefugees: 0,
|
||||
hostAsylumSeekers: 0,
|
||||
hostTotal: 0,
|
||||
lat: centroid?.[0], lon: centroid?.[1],
|
||||
};
|
||||
}
|
||||
for (const [code, data] of Object.entries(byAsylum)) {
|
||||
const hostRefugees = data.refugees;
|
||||
const hostAsylumSeekers = data.asylumSeekers;
|
||||
const hostTotal = hostRefugees + hostAsylumSeekers;
|
||||
if (!countries[code]) {
|
||||
const centroid = COUNTRY_CENTROIDS[code];
|
||||
countries[code] = {
|
||||
code, name: data.name,
|
||||
refugees: 0, asylumSeekers: 0, idps: 0, stateless: 0, totalDisplaced: 0,
|
||||
hostRefugees,
|
||||
hostAsylumSeekers,
|
||||
hostTotal,
|
||||
lat: centroid?.[0], lon: centroid?.[1],
|
||||
};
|
||||
} else {
|
||||
countries[code].hostRefugees = hostRefugees;
|
||||
countries[code].hostAsylumSeekers = hostAsylumSeekers;
|
||||
countries[code].hostTotal = hostTotal;
|
||||
}
|
||||
}
|
||||
|
||||
const topFlows = Object.values(flowMap)
|
||||
.sort((a, b) => b.refugees - a.refugees)
|
||||
.slice(0, 50)
|
||||
.map(f => {
|
||||
const oC = COUNTRY_CENTROIDS[f.originCode];
|
||||
const aC = COUNTRY_CENTROIDS[f.asylumCode];
|
||||
return {
|
||||
...f,
|
||||
originLat: oC?.[0], originLon: oC?.[1],
|
||||
asylumLat: aC?.[0], asylumLon: aC?.[1],
|
||||
};
|
||||
});
|
||||
|
||||
const result = {
|
||||
success: true,
|
||||
year: dataYearUsed ?? currentYear,
|
||||
globalTotals: {
|
||||
refugees: totalRefugees,
|
||||
asylumSeekers: totalAsylumSeekers,
|
||||
idps: totalIdps,
|
||||
stateless: totalStateless,
|
||||
total: totalRefugees + totalAsylumSeekers + totalIdps + totalStateless,
|
||||
},
|
||||
countries: Object.values(countries).sort((a, b) => {
|
||||
const aSize = Math.max(a.totalDisplaced || 0, a.hostTotal || 0);
|
||||
const bSize = Math.max(b.totalDisplaced || 0, b.hostTotal || 0);
|
||||
return bSize - aSize;
|
||||
}),
|
||||
topFlows,
|
||||
cached_at: new Date().toISOString(),
|
||||
};
|
||||
|
||||
fallbackCache = { data: result, timestamp: now };
|
||||
void setCachedJson(CACHE_KEY, result, CACHE_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/unhcr-population', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600', 'X-Cache': 'MISS' },
|
||||
});
|
||||
} catch (error) {
|
||||
if (isValidResult(fallbackCache.data)) {
|
||||
recordCacheTelemetry('/api/unhcr-population', 'STALE');
|
||||
return Response.json(fallbackCache.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=600, s-maxage=600, stale-while-revalidate=120', 'X-Cache': 'STALE' },
|
||||
});
|
||||
}
|
||||
|
||||
recordCacheTelemetry('/api/unhcr-population', 'ERROR');
|
||||
return Response.json({ error: `Fetch failed: ${toErrorMessage(error)}`, countries: [], topFlows: [] }, {
|
||||
status: 500, headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,4 @@
|
||||
// Non-sebuf: returns XML/HTML, stays as standalone Vercel function
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const RELEASES_URL = 'https://api.github.com/repos/koala73/worldmonitor/releases/latest';
|
||||
|
||||
@@ -1,304 +0,0 @@
|
||||
// Wingbits API proxy - keeps API key server-side
|
||||
// Note: Edge runtime is stateless - caching happens client-side and via HTTP Cache-Control
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const url = new URL(req.url);
|
||||
const path = url.pathname.replace('/api/wingbits', '');
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, POST, OPTIONS');
|
||||
|
||||
// Handle CORS preflight
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
return new Response(null, {
|
||||
status: 204,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Get API key from server-side env
|
||||
const apiKey = process.env.WINGBITS_API_KEY;
|
||||
|
||||
if (!apiKey) {
|
||||
return Response.json({
|
||||
error: 'Wingbits not configured',
|
||||
configured: false
|
||||
}, {
|
||||
status: 200,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Route: GET /details/:icao24 - Aircraft details
|
||||
const detailsMatch = path.match(/^\/details\/([a-fA-F0-9]+)$/);
|
||||
if (detailsMatch) {
|
||||
const icao24 = detailsMatch[1].toLowerCase();
|
||||
|
||||
try {
|
||||
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return Response.json({
|
||||
error: `Wingbits API error: ${response.status}`,
|
||||
icao24,
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
return Response.json(data, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', // 24h - aircraft details rarely change
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${error.message}`,
|
||||
icao24,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Route: POST /details/batch - Batch lookup multiple aircraft (parallel)
|
||||
if (path === '/details/batch' && req.method === 'POST') {
|
||||
try {
|
||||
const body = await req.json();
|
||||
const icao24List = body.icao24s || [];
|
||||
|
||||
if (!Array.isArray(icao24List) || icao24List.length === 0) {
|
||||
return Response.json({ error: 'icao24s array required' }, {
|
||||
status: 400,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Limit batch size to avoid overwhelming the API
|
||||
const limitedList = icao24List.slice(0, 20).map(id => id.toLowerCase());
|
||||
const results = {};
|
||||
|
||||
// Fetch all in parallel
|
||||
const fetchPromises = limitedList.map(async (icao24) => {
|
||||
try {
|
||||
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
const data = await response.json();
|
||||
return { icao24, data };
|
||||
}
|
||||
} catch {
|
||||
// Skip failed lookups
|
||||
}
|
||||
return null;
|
||||
});
|
||||
|
||||
const fetchResults = await Promise.all(fetchPromises);
|
||||
|
||||
for (const result of fetchResults) {
|
||||
if (result) {
|
||||
results[result.icao24] = result.data;
|
||||
}
|
||||
}
|
||||
|
||||
return Response.json({
|
||||
results,
|
||||
fetched: Object.keys(results).length,
|
||||
requested: limitedList.length,
|
||||
}, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({
|
||||
error: `Batch lookup failed: ${error.message}`,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Route: GET /flights - Get live flight positions in a geographic area
|
||||
// Query params: la (lat), lo (lon), w (width), h (height), unit (km|nm)
|
||||
if (path === '/flights' && req.method === 'GET') {
|
||||
try {
|
||||
const params = new URLSearchParams(url.search);
|
||||
const la = params.get('la') || params.get('lat');
|
||||
const lo = params.get('lo') || params.get('lon');
|
||||
const w = params.get('w') || params.get('width') || '500';
|
||||
const h = params.get('h') || params.get('height') || '500';
|
||||
const unit = params.get('unit') || 'nm';
|
||||
|
||||
if (!la || !lo) {
|
||||
return Response.json({ error: 'lat (la) and lon (lo) required' }, {
|
||||
status: 400,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const wingbitsUrl = `https://customer-api.wingbits.com/v1/flights?by=box&la=${la}&lo=${lo}&w=${w}&h=${h}&unit=${unit}`;
|
||||
console.log('[Wingbits] Fetching flights:', wingbitsUrl);
|
||||
|
||||
const response = await fetch(wingbitsUrl, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
console.error('[Wingbits] API error:', response.status, errorText);
|
||||
return Response.json({
|
||||
error: `Wingbits API error: ${response.status}`,
|
||||
details: errorText,
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
console.log('[Wingbits] Got', Array.isArray(data) ? data.length : 0, 'flights');
|
||||
|
||||
return Response.json(data, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15', // 30 seconds - live data
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[Wingbits] Flights fetch error:', error);
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${error.message}`,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Route: POST /flights/batch - Get flights for multiple areas (for theater posture)
|
||||
if (path === '/flights/batch' && req.method === 'POST') {
|
||||
try {
|
||||
const body = await req.json();
|
||||
const areas = body.areas || [];
|
||||
|
||||
if (!Array.isArray(areas) || areas.length === 0) {
|
||||
return Response.json({ error: 'areas array required' }, {
|
||||
status: 400,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Wingbits batch endpoint format
|
||||
const wingbitsAreas = areas.map(area => ({
|
||||
alias: area.id || area.alias,
|
||||
by: 'box',
|
||||
la: (area.north + area.south) / 2,
|
||||
lo: (area.east + area.west) / 2,
|
||||
w: Math.abs(area.east - area.west) * 60, // degrees to nautical miles (approx)
|
||||
h: Math.abs(area.north - area.south) * 60,
|
||||
unit: 'nm',
|
||||
}));
|
||||
|
||||
const response = await fetch('https://customer-api.wingbits.com/v1/flights', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify(wingbitsAreas),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const errorText = await response.text();
|
||||
return Response.json({
|
||||
error: `Wingbits API error: ${response.status}`,
|
||||
details: errorText,
|
||||
}, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
console.log('[Wingbits] Batch got', data.length, 'area results');
|
||||
|
||||
return Response.json(data, {
|
||||
headers: {
|
||||
...corsHeaders,
|
||||
'Cache-Control': 'public, max-age=30, s-maxage=30, stale-while-revalidate=15',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[Wingbits] Batch flights error:', error);
|
||||
return Response.json({
|
||||
error: `Fetch failed: ${error.message}`,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Route: GET /health - Check Wingbits status
|
||||
if (path === '/health' || path === '') {
|
||||
try {
|
||||
const response = await fetch('https://customer-api.wingbits.com/health', {
|
||||
headers: { 'x-api-key': apiKey },
|
||||
});
|
||||
const data = await response.json();
|
||||
return Response.json({
|
||||
...data,
|
||||
configured: true,
|
||||
}, {
|
||||
headers: corsHeaders,
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({
|
||||
error: error.message,
|
||||
configured: true,
|
||||
}, {
|
||||
status: 500,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return Response.json({ error: 'Not found' }, {
|
||||
status: 404,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
@@ -1,57 +0,0 @@
|
||||
// Wingbits single aircraft details
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req, { params }) {
|
||||
const icao24 = params.icao24?.toLowerCase();
|
||||
const apiKey = process.env.WINGBITS_API_KEY;
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (!apiKey) {
|
||||
return Response.json({ error: 'Wingbits not configured', configured: false }, {
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
if (!icao24 || !/^[a-f0-9]+$/i.test(icao24)) {
|
||||
return Response.json({ error: 'Invalid icao24' }, { status: 400, headers: corsHeaders });
|
||||
}
|
||||
|
||||
try {
|
||||
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return Response.json({ error: `Wingbits API error: ${response.status}`, icao24 }, {
|
||||
status: response.status,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
return Response.json(data, {
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600' },
|
||||
});
|
||||
} catch (error) {
|
||||
return Response.json({ error: error.message, icao24 }, { status: 500, headers: corsHeaders });
|
||||
}
|
||||
}
|
||||
@@ -1,74 +0,0 @@
|
||||
// Wingbits batch aircraft details
|
||||
import { getCorsHeaders, isDisallowedOrigin } from '../../_cors.js';
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
export default async function handler(req) {
|
||||
const apiKey = process.env.WINGBITS_API_KEY;
|
||||
|
||||
const corsHeaders = getCorsHeaders(req, 'POST, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(null, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'POST') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (!apiKey) {
|
||||
return Response.json({ error: 'Wingbits not configured', configured: false }, {
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const body = await req.json();
|
||||
const icao24List = body.icao24s || [];
|
||||
|
||||
if (!Array.isArray(icao24List) || icao24List.length === 0) {
|
||||
return Response.json({ error: 'icao24s array required' }, { status: 400, headers: corsHeaders });
|
||||
}
|
||||
|
||||
// Limit batch size
|
||||
const limitedList = icao24List.slice(0, 20).map(id => id.toLowerCase());
|
||||
const results = {};
|
||||
|
||||
// Fetch all in parallel
|
||||
const fetchPromises = limitedList.map(async (icao24) => {
|
||||
try {
|
||||
const response = await fetch(`https://customer-api.wingbits.com/v1/flights/details/${icao24}`, {
|
||||
headers: {
|
||||
'x-api-key': apiKey,
|
||||
'Accept': 'application/json',
|
||||
},
|
||||
});
|
||||
if (response.ok) {
|
||||
return { icao24, data: await response.json() };
|
||||
}
|
||||
} catch {
|
||||
// Skip failed lookups
|
||||
}
|
||||
return null;
|
||||
});
|
||||
|
||||
const fetchResults = await Promise.all(fetchPromises);
|
||||
for (const result of fetchResults) {
|
||||
if (result) results[result.icao24] = result.data;
|
||||
}
|
||||
|
||||
return Response.json({
|
||||
results,
|
||||
fetched: Object.keys(results).length,
|
||||
requested: limitedList.length,
|
||||
}, { headers: { 'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', ...corsHeaders } });
|
||||
} catch (error) {
|
||||
return Response.json({ error: error.message }, { status: 500, headers: corsHeaders });
|
||||
}
|
||||
}
|
||||
@@ -1,145 +0,0 @@
|
||||
// World Bank API proxy (Web API handler for Edge + sidecar compatibility)
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
export const config = {
|
||||
runtime: 'edge',
|
||||
};
|
||||
|
||||
const TECH_INDICATORS = {
|
||||
'IT.NET.USER.ZS': 'Internet Users (% of population)',
|
||||
'IT.CEL.SETS.P2': 'Mobile Subscriptions (per 100 people)',
|
||||
'IT.NET.BBND.P2': 'Fixed Broadband Subscriptions (per 100 people)',
|
||||
'IT.NET.SECR.P6': 'Secure Internet Servers (per million people)',
|
||||
'GB.XPD.RSDV.GD.ZS': 'R&D Expenditure (% of GDP)',
|
||||
'IP.PAT.RESD': 'Patent Applications (residents)',
|
||||
'IP.PAT.NRES': 'Patent Applications (non-residents)',
|
||||
'IP.TMK.TOTL': 'Trademark Applications',
|
||||
'TX.VAL.TECH.MF.ZS': 'High-Tech Exports (% of manufactured exports)',
|
||||
'BX.GSR.CCIS.ZS': 'ICT Service Exports (% of service exports)',
|
||||
'TM.VAL.ICTG.ZS.UN': 'ICT Goods Imports (% of total goods imports)',
|
||||
'SE.TER.ENRR': 'Tertiary Education Enrollment (%)',
|
||||
'SE.XPD.TOTL.GD.ZS': 'Education Expenditure (% of GDP)',
|
||||
'NY.GDP.MKTP.KD.ZG': 'GDP Growth (annual %)',
|
||||
'NY.GDP.PCAP.CD': 'GDP per Capita (current US$)',
|
||||
'NE.EXP.GNFS.ZS': 'Exports of Goods & Services (% of GDP)',
|
||||
};
|
||||
|
||||
const TECH_COUNTRIES = [
|
||||
'USA', 'CHN', 'JPN', 'DEU', 'KOR', 'GBR', 'IND', 'ISR', 'SGP', 'TWN',
|
||||
'FRA', 'CAN', 'SWE', 'NLD', 'CHE', 'FIN', 'IRL', 'AUS', 'BRA', 'IDN',
|
||||
'ARE', 'SAU', 'QAT', 'BHR', 'EGY', 'TUR',
|
||||
'MYS', 'THA', 'VNM', 'PHL',
|
||||
'ESP', 'ITA', 'POL', 'CZE', 'DNK', 'NOR', 'AUT', 'BEL', 'PRT', 'EST',
|
||||
'MEX', 'ARG', 'CHL', 'COL',
|
||||
'ZAF', 'NGA', 'KEN',
|
||||
];
|
||||
|
||||
export default async function handler(request) {
|
||||
const CORS = getCorsHeaders(request);
|
||||
if (isDisallowedOrigin(request)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: CORS });
|
||||
}
|
||||
|
||||
function json(data, status = 200, extra = {}) {
|
||||
return new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { 'Content-Type': 'application/json', ...CORS, ...extra },
|
||||
});
|
||||
}
|
||||
|
||||
if (request.method === 'OPTIONS') {
|
||||
return new Response(null, { status: 204, headers: CORS });
|
||||
}
|
||||
|
||||
const url = new URL(request.url);
|
||||
const indicator = url.searchParams.get('indicator');
|
||||
const country = url.searchParams.get('country');
|
||||
const countries = url.searchParams.get('countries');
|
||||
const years = url.searchParams.get('years') || '5';
|
||||
const action = url.searchParams.get('action');
|
||||
|
||||
if (action === 'indicators') {
|
||||
return json({ indicators: TECH_INDICATORS, defaultCountries: TECH_COUNTRIES }, 200, { 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600' });
|
||||
}
|
||||
|
||||
if (!indicator) {
|
||||
return json({ error: 'Missing indicator parameter', availableIndicators: Object.keys(TECH_INDICATORS) }, 400);
|
||||
}
|
||||
|
||||
try {
|
||||
let countryList = country || countries || TECH_COUNTRIES.join(';');
|
||||
if (countries) {
|
||||
countryList = countries.split(',').join(';');
|
||||
}
|
||||
|
||||
const currentYear = new Date().getFullYear();
|
||||
const startYear = currentYear - parseInt(years);
|
||||
|
||||
const wbUrl = `https://api.worldbank.org/v2/country/${countryList}/indicator/${indicator}?format=json&date=${startYear}:${currentYear}&per_page=1000`;
|
||||
|
||||
const response = await fetch(wbUrl, {
|
||||
headers: {
|
||||
'Accept': 'application/json',
|
||||
'User-Agent': 'Mozilla/5.0 (compatible; WorldMonitor/1.0; +https://worldmonitor.app)',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`World Bank API error: ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (!data || !Array.isArray(data) || data.length < 2 || !data[1]) {
|
||||
return json({
|
||||
indicator,
|
||||
indicatorName: TECH_INDICATORS[indicator] || indicator,
|
||||
metadata: { page: 1, pages: 1, total: 0 },
|
||||
byCountry: {},
|
||||
latestByCountry: {},
|
||||
timeSeries: [],
|
||||
}, 200, { 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' });
|
||||
}
|
||||
|
||||
const [metadata, records] = data;
|
||||
|
||||
const transformed = {
|
||||
indicator,
|
||||
indicatorName: TECH_INDICATORS[indicator] || (records[0]?.indicator?.value || indicator),
|
||||
metadata: { page: metadata.page, pages: metadata.pages, total: metadata.total },
|
||||
byCountry: {},
|
||||
latestByCountry: {},
|
||||
timeSeries: [],
|
||||
};
|
||||
|
||||
for (const record of records || []) {
|
||||
const countryCode = record.countryiso3code || record.country?.id;
|
||||
const countryName = record.country?.value;
|
||||
const year = record.date;
|
||||
const value = record.value;
|
||||
|
||||
if (!countryCode || value === null) continue;
|
||||
|
||||
if (!transformed.byCountry[countryCode]) {
|
||||
transformed.byCountry[countryCode] = { code: countryCode, name: countryName, values: [] };
|
||||
}
|
||||
transformed.byCountry[countryCode].values.push({ year, value });
|
||||
|
||||
if (!transformed.latestByCountry[countryCode] || year > transformed.latestByCountry[countryCode].year) {
|
||||
transformed.latestByCountry[countryCode] = { code: countryCode, name: countryName, year, value };
|
||||
}
|
||||
|
||||
transformed.timeSeries.push({ countryCode, countryName, year, value });
|
||||
}
|
||||
|
||||
for (const c of Object.values(transformed.byCountry)) {
|
||||
c.values.sort((a, b) => a.year - b.year);
|
||||
}
|
||||
|
||||
transformed.timeSeries.sort((a, b) => b.year - a.year || a.countryCode.localeCompare(b.countryCode));
|
||||
|
||||
return json(transformed, 200, { 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' });
|
||||
} catch (error) {
|
||||
return json({ error: error.message, indicator }, 500);
|
||||
}
|
||||
}
|
||||
@@ -1,171 +0,0 @@
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
import { getCachedJson, setCachedJson } from './_upstash-cache.js';
|
||||
import { recordCacheTelemetry } from './_cache-telemetry.js';
|
||||
import { createIpRateLimiter } from './_ip-rate-limit.js';
|
||||
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
const COUNTRIES_CACHE_KEY = 'worldpop:countries:v1';
|
||||
const COUNTRIES_TTL_SECONDS = 7 * 24 * 60 * 60;
|
||||
const COUNTRIES_TTL_MS = COUNTRIES_TTL_SECONDS * 1000;
|
||||
const EXPOSURE_TTL_SECONDS = 24 * 60 * 60;
|
||||
|
||||
let countriesFallback = { data: null, timestamp: 0 };
|
||||
|
||||
const rateLimiter = createIpRateLimiter({
|
||||
limit: 30,
|
||||
windowMs: 60 * 1000,
|
||||
maxEntries: 5000,
|
||||
});
|
||||
|
||||
function getClientIp(req) {
|
||||
return req.headers.get('x-forwarded-for')?.split(',')[0] ||
|
||||
req.headers.get('x-real-ip') ||
|
||||
'unknown';
|
||||
}
|
||||
|
||||
function toErrorMessage(error) {
|
||||
if (error instanceof Error) return error.message;
|
||||
return String(error || 'unknown error');
|
||||
}
|
||||
|
||||
const PRIORITY_COUNTRIES = {
|
||||
UKR: { name: 'Ukraine', pop: 37000000, area: 603550 },
|
||||
RUS: { name: 'Russia', pop: 144100000, area: 17098242 },
|
||||
ISR: { name: 'Israel', pop: 9800000, area: 22072 },
|
||||
PSE: { name: 'Palestine', pop: 5400000, area: 6020 },
|
||||
SYR: { name: 'Syria', pop: 22100000, area: 185180 },
|
||||
IRN: { name: 'Iran', pop: 88600000, area: 1648195 },
|
||||
TWN: { name: 'Taiwan', pop: 23600000, area: 36193 },
|
||||
ETH: { name: 'Ethiopia', pop: 126500000, area: 1104300 },
|
||||
SDN: { name: 'Sudan', pop: 48100000, area: 1861484 },
|
||||
SSD: { name: 'South Sudan', pop: 11400000, area: 619745 },
|
||||
SOM: { name: 'Somalia', pop: 18100000, area: 637657 },
|
||||
YEM: { name: 'Yemen', pop: 34400000, area: 527968 },
|
||||
AFG: { name: 'Afghanistan', pop: 42200000, area: 652230 },
|
||||
PAK: { name: 'Pakistan', pop: 240500000, area: 881913 },
|
||||
IND: { name: 'India', pop: 1428600000, area: 3287263 },
|
||||
MMR: { name: 'Myanmar', pop: 54200000, area: 676578 },
|
||||
COD: { name: 'DR Congo', pop: 102300000, area: 2344858 },
|
||||
NGA: { name: 'Nigeria', pop: 223800000, area: 923768 },
|
||||
MLI: { name: 'Mali', pop: 22600000, area: 1240192 },
|
||||
BFA: { name: 'Burkina Faso', pop: 22700000, area: 274200 },
|
||||
};
|
||||
|
||||
function isValidCountries(data) {
|
||||
return Boolean(data && typeof data === 'object' && Array.isArray(data.countries));
|
||||
}
|
||||
|
||||
async function handleCountries(corsHeaders, now) {
|
||||
const cached = await getCachedJson(COUNTRIES_CACHE_KEY);
|
||||
if (isValidCountries(cached)) {
|
||||
recordCacheTelemetry('/api/worldpop-exposure?countries', 'REDIS-HIT');
|
||||
return Response.json(cached, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'REDIS-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
if (isValidCountries(countriesFallback.data) && now - countriesFallback.timestamp < COUNTRIES_TTL_MS) {
|
||||
recordCacheTelemetry('/api/worldpop-exposure?countries', 'MEMORY-HIT');
|
||||
return Response.json(countriesFallback.data, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'MEMORY-HIT' },
|
||||
});
|
||||
}
|
||||
|
||||
const countries = Object.entries(PRIORITY_COUNTRIES).map(([code, info]) => ({
|
||||
code,
|
||||
name: info.name,
|
||||
population: info.pop,
|
||||
densityPerKm2: Math.round(info.pop / info.area),
|
||||
}));
|
||||
|
||||
const result = { success: true, countries, cached_at: new Date().toISOString() };
|
||||
countriesFallback = { data: result, timestamp: now };
|
||||
void setCachedJson(COUNTRIES_CACHE_KEY, result, COUNTRIES_TTL_SECONDS);
|
||||
recordCacheTelemetry('/api/worldpop-exposure?countries', 'MISS');
|
||||
|
||||
return Response.json(result, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=86400, s-maxage=86400, stale-while-revalidate=3600', 'X-Cache': 'MISS' },
|
||||
});
|
||||
}
|
||||
|
||||
function handleExposure(corsHeaders, lat, lon, radius) {
|
||||
let bestMatch = null;
|
||||
let bestDist = Infinity;
|
||||
|
||||
const CENTROIDS = {
|
||||
UKR: [48.4, 31.2], RUS: [61.5, 105.3], ISR: [31.0, 34.8], PSE: [31.9, 35.2],
|
||||
SYR: [35.0, 38.0], IRN: [32.4, 53.7], TWN: [23.7, 121.0], ETH: [9.1, 40.5],
|
||||
SDN: [15.5, 32.5], SSD: [6.9, 31.3], SOM: [5.2, 46.2], YEM: [15.6, 48.5],
|
||||
AFG: [33.9, 67.7], PAK: [30.4, 69.3], IND: [20.6, 79.0], MMR: [19.8, 96.7],
|
||||
COD: [-4.0, 21.8], NGA: [9.1, 7.5], MLI: [17.6, -4.0], BFA: [12.3, -1.6],
|
||||
};
|
||||
|
||||
for (const [code, [cLat, cLon]] of Object.entries(CENTROIDS)) {
|
||||
const dist = Math.sqrt(Math.pow(lat - cLat, 2) + Math.pow(lon - cLon, 2));
|
||||
if (dist < bestDist) {
|
||||
bestDist = dist;
|
||||
bestMatch = code;
|
||||
}
|
||||
}
|
||||
|
||||
const info = PRIORITY_COUNTRIES[bestMatch] || { pop: 50000000, area: 500000 };
|
||||
const density = info.pop / info.area;
|
||||
const areaKm2 = Math.PI * radius * radius;
|
||||
const exposed = Math.round(density * areaKm2);
|
||||
|
||||
return Response.json({
|
||||
success: true,
|
||||
exposedPopulation: exposed,
|
||||
exposureRadiusKm: radius,
|
||||
nearestCountry: bestMatch,
|
||||
densityPerKm2: Math.round(density),
|
||||
}, {
|
||||
status: 200,
|
||||
headers: { ...corsHeaders, 'Cache-Control': 'public, max-age=3600, s-maxage=3600, stale-while-revalidate=600' },
|
||||
});
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const corsHeaders = getCorsHeaders(req, 'GET, OPTIONS');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
if (isDisallowedOrigin(req)) return new Response(null, { status: 403, headers: corsHeaders });
|
||||
return new Response(null, { status: 204, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (req.method !== 'GET') {
|
||||
return Response.json({ error: 'Method not allowed' }, { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return Response.json({ error: 'Origin not allowed' }, { status: 403, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const ip = getClientIp(req);
|
||||
if (!rateLimiter.check(ip)) {
|
||||
return Response.json({ error: 'Rate limited' }, {
|
||||
status: 429, headers: { ...corsHeaders, 'Retry-After': '60' },
|
||||
});
|
||||
}
|
||||
|
||||
const url = new URL(req.url);
|
||||
const mode = url.searchParams.get('mode') || 'countries';
|
||||
|
||||
if (mode === 'exposure') {
|
||||
const lat = Number(url.searchParams.get('lat'));
|
||||
const lon = Number(url.searchParams.get('lon'));
|
||||
const radius = Number(url.searchParams.get('radius')) || 50;
|
||||
|
||||
if (isNaN(lat) || isNaN(lon)) {
|
||||
return Response.json({ error: 'lat and lon required' }, { status: 400, headers: corsHeaders });
|
||||
}
|
||||
|
||||
return handleExposure(corsHeaders, lat, lon, radius);
|
||||
}
|
||||
|
||||
return handleCountries(corsHeaders, Date.now());
|
||||
}
|
||||
Vendored
-54
@@ -1,54 +0,0 @@
|
||||
export const config = { runtime: 'edge' };
|
||||
|
||||
import { getCorsHeaders, isDisallowedOrigin } from './_cors.js';
|
||||
|
||||
const SYMBOL_PATTERN = /^[A-Za-z0-9.^=\-]+$/;
|
||||
const MAX_SYMBOL_LENGTH = 20;
|
||||
|
||||
function validateSymbol(symbol) {
|
||||
if (!symbol) return null;
|
||||
const trimmed = symbol.trim().toUpperCase();
|
||||
if (trimmed.length > MAX_SYMBOL_LENGTH) return null;
|
||||
if (!SYMBOL_PATTERN.test(trimmed)) return null;
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
export default async function handler(req) {
|
||||
const cors = getCorsHeaders(req);
|
||||
if (isDisallowedOrigin(req)) {
|
||||
return new Response(JSON.stringify({ error: 'Origin not allowed' }), { status: 403, headers: cors });
|
||||
}
|
||||
const url = new URL(req.url);
|
||||
const symbol = validateSymbol(url.searchParams.get('symbol'));
|
||||
|
||||
if (!symbol) {
|
||||
return new Response(JSON.stringify({ error: 'Invalid or missing symbol parameter' }), {
|
||||
status: 400,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
|
||||
try {
|
||||
const yahooUrl = `https://query1.finance.yahoo.com/v8/finance/chart/${encodeURIComponent(symbol)}`;
|
||||
const response = await fetch(yahooUrl, {
|
||||
headers: {
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
|
||||
},
|
||||
});
|
||||
|
||||
const data = await response.text();
|
||||
return new Response(data, {
|
||||
status: response.status,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...cors,
|
||||
'Cache-Control': 'public, max-age=60, s-maxage=60, stale-while-revalidate=30',
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
return new Response(JSON.stringify({ error: 'Failed to fetch data' }), {
|
||||
status: 500,
|
||||
headers: { 'Content-Type': 'application/json', ...cors },
|
||||
});
|
||||
}
|
||||
}
|
||||
+49
-13
@@ -19,17 +19,31 @@ const ALLOWED_ORIGINS = [
|
||||
/^tauri:\/\/localhost$/,
|
||||
];
|
||||
|
||||
function sanitizeOrigin(raw) {
|
||||
if (!raw) return 'https://worldmonitor.app';
|
||||
const ALLOWED_PARENT_ORIGINS = [
|
||||
...ALLOWED_ORIGINS,
|
||||
/^https?:\/\/tauri\.localhost$/,
|
||||
/^https?:\/\/[a-z0-9-]+\.tauri\.localhost$/,
|
||||
];
|
||||
|
||||
function sanitizeAllowedOrigin(raw, fallback, allowList = ALLOWED_ORIGINS) {
|
||||
if (!raw) return fallback;
|
||||
try {
|
||||
const parsed = new URL(raw);
|
||||
if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:' && parsed.protocol !== 'tauri:') {
|
||||
return 'https://worldmonitor.app';
|
||||
if (!['https:', 'http:', 'tauri:'].includes(parsed.protocol)) {
|
||||
return fallback;
|
||||
}
|
||||
const origin = parsed.origin !== 'null' ? parsed.origin : raw;
|
||||
if (ALLOWED_ORIGINS.some(p => p.test(origin))) return origin;
|
||||
if (allowList.some(p => p.test(origin))) return origin;
|
||||
} catch { /* invalid URL */ }
|
||||
return 'https://worldmonitor.app';
|
||||
return fallback;
|
||||
}
|
||||
|
||||
function sanitizeOrigin(raw) {
|
||||
return sanitizeAllowedOrigin(raw, 'https://worldmonitor.app', ALLOWED_ORIGINS);
|
||||
}
|
||||
|
||||
function sanitizeParentOrigin(raw, fallback) {
|
||||
return sanitizeAllowedOrigin(raw, fallback, ALLOWED_PARENT_ORIGINS);
|
||||
}
|
||||
|
||||
export default async function handler(request) {
|
||||
@@ -45,10 +59,12 @@ export default async function handler(request) {
|
||||
|
||||
const autoplay = parseFlag(url.searchParams.get('autoplay'), '1');
|
||||
const mute = parseFlag(url.searchParams.get('mute'), '1');
|
||||
const vq = ['small', 'medium', 'large', 'hd720', 'hd1080'].includes(url.searchParams.get('vq') || '') ? url.searchParams.get('vq') : '';
|
||||
|
||||
const origin = sanitizeOrigin(url.searchParams.get('origin'));
|
||||
const parentOrigin = sanitizeParentOrigin(url.searchParams.get('parentOrigin'), origin);
|
||||
|
||||
const embedSrc = new URL(`https://www.youtube-nocookie.com/embed/${videoId}`);
|
||||
const embedSrc = new URL(`https://www.youtube.com/embed/${videoId}`);
|
||||
embedSrc.searchParams.set('autoplay', autoplay);
|
||||
embedSrc.searchParams.set('mute', mute);
|
||||
embedSrc.searchParams.set('playsinline', '1');
|
||||
@@ -79,21 +95,39 @@ export default async function handler(request) {
|
||||
var tag=document.createElement('script');
|
||||
tag.src='https://www.youtube.com/iframe_api';
|
||||
document.head.appendChild(tag);
|
||||
var player,overlay=document.getElementById('play-overlay'),started=false;
|
||||
var player,overlay=document.getElementById('play-overlay'),started=false,muteSyncIntervalId,parentOrigin=${JSON.stringify(parentOrigin)},allowedOrigin=${JSON.stringify(parentOrigin)};
|
||||
function hideOverlay(){overlay.classList.add('hidden')}
|
||||
function readMuted(){
|
||||
if(!player)return null;
|
||||
if(typeof player.isMuted==='function')return player.isMuted();
|
||||
if(typeof player.getVolume==='function')return player.getVolume()===0;
|
||||
return null;
|
||||
}
|
||||
function stopMuteSync(){if(muteSyncIntervalId){clearInterval(muteSyncIntervalId);muteSyncIntervalId=null}}
|
||||
function startMuteSync(){
|
||||
if(muteSyncIntervalId)return;
|
||||
var lastMuted=readMuted();
|
||||
if(lastMuted!==null)window.parent.postMessage({type:'yt-mute-state',muted:lastMuted},parentOrigin);
|
||||
muteSyncIntervalId=setInterval(function(){
|
||||
var m=readMuted();
|
||||
if(m!==null&&m!==lastMuted){lastMuted=m;window.parent.postMessage({type:'yt-mute-state',muted:m},parentOrigin)}
|
||||
},500);
|
||||
}
|
||||
function onYouTubeIframeAPIReady(){
|
||||
player=new YT.Player('player',{
|
||||
videoId:'${videoId}',
|
||||
host:'https://www.youtube-nocookie.com',
|
||||
host:'https://www.youtube.com',
|
||||
playerVars:{autoplay:${autoplay},mute:${mute},playsinline:1,rel:0,controls:1,modestbranding:1,enablejsapi:1,origin:${JSON.stringify(origin)},widget_referrer:${JSON.stringify(origin)}},
|
||||
events:{
|
||||
onReady:function(){
|
||||
window.parent.postMessage({type:'yt-ready'},'*');
|
||||
window.parent.postMessage({type:'yt-ready'},parentOrigin);
|
||||
${vq ? `if(player.setPlaybackQuality)player.setPlaybackQuality('${vq}');` : ''}
|
||||
if(${autoplay}===1){player.playVideo()}
|
||||
startMuteSync();
|
||||
},
|
||||
onError:function(e){window.parent.postMessage({type:'yt-error',code:e.data},'*')},
|
||||
onError:function(e){stopMuteSync();window.parent.postMessage({type:'yt-error',code:e.data},parentOrigin)},
|
||||
onStateChange:function(e){
|
||||
window.parent.postMessage({type:'yt-state',state:e.data},'*');
|
||||
window.parent.postMessage({type:'yt-state',state:e.data},parentOrigin);
|
||||
if(e.data===1||e.data===3){hideOverlay();started=true}
|
||||
}
|
||||
}
|
||||
@@ -104,6 +138,7 @@ export default async function handler(request) {
|
||||
});
|
||||
setTimeout(function(){if(!started)overlay.classList.remove('hidden')},3000);
|
||||
window.addEventListener('message',function(e){
|
||||
if(allowedOrigin!=='*'&&e.origin!==allowedOrigin)return;
|
||||
if(!player||!player.getPlayerState)return;
|
||||
var m=e.data;if(!m||!m.type)return;
|
||||
switch(m.type){
|
||||
@@ -112,6 +147,7 @@ export default async function handler(request) {
|
||||
case'mute':player.mute();break;
|
||||
case'unmute':player.unMute();break;
|
||||
case'loadVideo':if(m.videoId)player.loadVideoById(m.videoId);break;
|
||||
case'setQuality':if(m.quality&&player.setPlaybackQuality)player.setPlaybackQuality(m.quality);break;
|
||||
}
|
||||
});
|
||||
</script>
|
||||
@@ -122,7 +158,7 @@ export default async function handler(request) {
|
||||
status: 200,
|
||||
headers: {
|
||||
'content-type': 'text/html; charset=utf-8',
|
||||
'cache-control': 'public, s-maxage=60, stale-while-revalidate=300',
|
||||
'cache-control': 'public, s-maxage=900, stale-while-revalidate=300',
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
@@ -21,7 +21,7 @@ test('returns embeddable html for valid video id', async () => {
|
||||
|
||||
const html = await response.text();
|
||||
assert.equal(html.includes("videoId:'iEpJwprxDdk'"), true);
|
||||
assert.equal(html.includes("host:'https://www.youtube-nocookie.com'"), true);
|
||||
assert.equal(html.includes("host:'https://www.youtube.com'"), true);
|
||||
assert.equal(html.includes('autoplay:0'), true);
|
||||
assert.equal(html.includes('mute:1'), true);
|
||||
assert.equal(html.includes('origin:"https://worldmonitor.app"'), true);
|
||||
@@ -33,3 +33,18 @@ test('accepts custom origin parameter', async () => {
|
||||
const html = await response.text();
|
||||
assert.equal(html.includes('origin:"http://127.0.0.1:46123"'), true);
|
||||
});
|
||||
|
||||
test('uses dedicated parentOrigin for iframe postMessage target', async () => {
|
||||
const response = await handler(makeRequest('?videoId=iEpJwprxDdk&origin=https://worldmonitor.app&parentOrigin=https://tauri.localhost'));
|
||||
const html = await response.text();
|
||||
assert.match(html, /playerVars:\{[^}]*origin:"https:\/\/worldmonitor\.app"/);
|
||||
assert.match(html, /parentOrigin="https:\/\/tauri\.localhost"/);
|
||||
assert.match(html, /if\(allowedOrigin!==['"]\*['"]&&e\.origin!==allowedOrigin\)return/);
|
||||
});
|
||||
|
||||
test('does not accept wildcard parentOrigin query parameter', async () => {
|
||||
const response = await handler(makeRequest('?videoId=iEpJwprxDdk&origin=https://worldmonitor.app&parentOrigin=*'));
|
||||
const html = await response.text();
|
||||
assert.equal(html.includes('parentOrigin="*"'), false);
|
||||
assert.match(html, /parentOrigin="https:\/\/worldmonitor\.app"/);
|
||||
});
|
||||
|
||||
+47
-14
@@ -15,6 +15,28 @@ export default async function handler(request) {
|
||||
}
|
||||
const url = new URL(request.url);
|
||||
const channel = url.searchParams.get('channel');
|
||||
const videoIdParam = url.searchParams.get('videoId');
|
||||
|
||||
// Video ID lookup: resolve author name via oembed
|
||||
if (videoIdParam && /^[A-Za-z0-9_-]{11}$/.test(videoIdParam)) {
|
||||
try {
|
||||
const oembedRes = await fetch(
|
||||
`https://www.youtube.com/oembed?url=https://www.youtube.com/watch?v=${videoIdParam}&format=json`,
|
||||
{ headers: { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36' } },
|
||||
);
|
||||
if (oembedRes.ok) {
|
||||
const data = await oembedRes.json();
|
||||
return new Response(JSON.stringify({ channelName: data.author_name || null, title: data.title || null, videoId: videoIdParam }), {
|
||||
status: 200,
|
||||
headers: { ...cors, 'Content-Type': 'application/json', 'Cache-Control': 'public, max-age=3600, s-maxage=3600' },
|
||||
});
|
||||
}
|
||||
} catch {}
|
||||
return new Response(JSON.stringify({ channelName: null, title: null, videoId: videoIdParam }), {
|
||||
status: 200,
|
||||
headers: { ...cors, 'Content-Type': 'application/json' },
|
||||
});
|
||||
}
|
||||
|
||||
if (!channel) {
|
||||
return new Response(JSON.stringify({ error: 'Missing channel parameter' }), {
|
||||
@@ -36,7 +58,7 @@ export default async function handler(request) {
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return new Response(JSON.stringify({ videoId: null }), {
|
||||
return new Response(JSON.stringify({ videoId: null, channelExists: false }), {
|
||||
status: 200,
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
});
|
||||
@@ -44,22 +66,33 @@ export default async function handler(request) {
|
||||
|
||||
const html = await response.text();
|
||||
|
||||
// Extract video ID from the page
|
||||
const videoIdMatch = html.match(/"videoId":"([a-zA-Z0-9_-]{11})"/);
|
||||
const isLiveMatch = html.match(/"isLive":\s*true/);
|
||||
// Channel exists if the page contains canonical channel metadata
|
||||
const channelExists = html.includes('"channelId"') || html.includes('og:url');
|
||||
|
||||
if (videoIdMatch && isLiveMatch) {
|
||||
return new Response(JSON.stringify({ videoId: videoIdMatch[1], isLive: true }), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
'Cache-Control': 'public, max-age=300, s-maxage=300, stale-while-revalidate=60', // Cache for 5 minutes
|
||||
},
|
||||
});
|
||||
// Extract channel name from page metadata (prefer channel name over video title)
|
||||
let channelName = null;
|
||||
const ownerMatch = html.match(/"ownerChannelName"\s*:\s*"([^"]+)"/);
|
||||
if (ownerMatch) {
|
||||
channelName = ownerMatch[1];
|
||||
} else {
|
||||
const authorMatch = html.match(/"author"\s*:\s*"([^"]+)"/);
|
||||
if (authorMatch) channelName = authorMatch[1];
|
||||
}
|
||||
|
||||
// Return null if no live stream found
|
||||
return new Response(JSON.stringify({ videoId: null, isLive: false }), {
|
||||
// Scope both fields to the same videoDetails block so we don't
|
||||
// combine a videoId from one object with isLive from another.
|
||||
let videoId = null;
|
||||
const detailsIdx = html.indexOf('"videoDetails"');
|
||||
if (detailsIdx !== -1) {
|
||||
const block = html.substring(detailsIdx, detailsIdx + 5000);
|
||||
const vidMatch = block.match(/"videoId":"([a-zA-Z0-9_-]{11})"/);
|
||||
const liveMatch = block.match(/"isLive"\s*:\s*true/);
|
||||
if (vidMatch && liveMatch) {
|
||||
videoId = vidMatch[1];
|
||||
}
|
||||
}
|
||||
|
||||
return new Response(JSON.stringify({ videoId, isLive: videoId !== null, channelExists, channelName }), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
import { mutation } from "./_generated/server";
|
||||
import { v } from "convex/values";
|
||||
|
||||
export const register = mutation({
|
||||
args: {
|
||||
email: v.string(),
|
||||
source: v.optional(v.string()),
|
||||
appVersion: v.optional(v.string()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const normalizedEmail = args.email.trim().toLowerCase();
|
||||
|
||||
const existing = await ctx.db
|
||||
.query("registrations")
|
||||
.withIndex("by_normalized_email", (q) => q.eq("normalizedEmail", normalizedEmail))
|
||||
.first();
|
||||
|
||||
if (existing) {
|
||||
return { status: "already_registered" as const };
|
||||
}
|
||||
|
||||
await ctx.db.insert("registrations", {
|
||||
email: args.email.trim(),
|
||||
normalizedEmail,
|
||||
registeredAt: Date.now(),
|
||||
source: args.source ?? "unknown",
|
||||
appVersion: args.appVersion ?? "unknown",
|
||||
});
|
||||
|
||||
return { status: "registered" as const };
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,12 @@
|
||||
import { defineSchema, defineTable } from "convex/server";
|
||||
import { v } from "convex/values";
|
||||
|
||||
export default defineSchema({
|
||||
registrations: defineTable({
|
||||
email: v.string(),
|
||||
normalizedEmail: v.string(),
|
||||
registeredAt: v.number(),
|
||||
source: v.optional(v.string()),
|
||||
appVersion: v.optional(v.string()),
|
||||
}).index("by_normalized_email", ["normalizedEmail"]),
|
||||
});
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ESNext",
|
||||
"lib": ["ES2021"],
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "Bundler",
|
||||
"strict": true,
|
||||
"skipLibCheck": true,
|
||||
"allowJs": true,
|
||||
"outDir": "./_generated"
|
||||
},
|
||||
"include": ["./**/*.ts"],
|
||||
"exclude": ["./_generated"]
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
{
|
||||
"version": 1,
|
||||
"updatedAt": "2026-02-23T12:19:41Z",
|
||||
"note": "Product-managed curated list. Not user-configurable.",
|
||||
"channels": {
|
||||
"full": [
|
||||
{
|
||||
"handle": "VahidOnline",
|
||||
"label": "Vahid Online",
|
||||
"topic": "politics",
|
||||
"tier": 1,
|
||||
"enabled": true,
|
||||
"region": "iran",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "BNONews",
|
||||
"label": "BNO News",
|
||||
"topic": "breaking",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 25
|
||||
},
|
||||
{
|
||||
"handle": "LiveUAMap",
|
||||
"label": "LiveUAMap",
|
||||
"topic": "breaking",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 25
|
||||
},
|
||||
{
|
||||
"handle": "ClashReport",
|
||||
"label": "Clash Report",
|
||||
"topic": "conflict",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 30
|
||||
},
|
||||
{
|
||||
"handle": "OSINTdefender",
|
||||
"label": "OSINTdefender",
|
||||
"topic": "conflict",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 25
|
||||
},
|
||||
{
|
||||
"handle": "AuroraIntel",
|
||||
"label": "Aurora Intel",
|
||||
"topic": "conflict",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "GeopoliticalCenter",
|
||||
"label": "GeopoliticalCenter",
|
||||
"topic": "geopolitics",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "Osintlatestnews",
|
||||
"label": "OSIntOps News",
|
||||
"topic": "osint",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "air_alert_ua",
|
||||
"label": "Повітряна Тривога",
|
||||
"topic": "alerts",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "ukraine",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "kpszsu",
|
||||
"label": "Air Force of the Armed Forces of Ukraine",
|
||||
"topic": "alerts",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "ukraine",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "war_monitor",
|
||||
"label": "monitor",
|
||||
"topic": "alerts",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "ukraine",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "DeepStateUA",
|
||||
"label": "DeepState",
|
||||
"topic": "conflict",
|
||||
"tier": 2,
|
||||
"enabled": true,
|
||||
"region": "ukraine",
|
||||
"maxMessages": 20
|
||||
},
|
||||
{
|
||||
"handle": "bellingcat",
|
||||
"label": "Bellingcat",
|
||||
"topic": "osint",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "global",
|
||||
"maxMessages": 10
|
||||
},
|
||||
{
|
||||
"handle": "nexta_live",
|
||||
"label": "NEXTA Live",
|
||||
"topic": "breaking",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "europe",
|
||||
"maxMessages": 15
|
||||
},
|
||||
{
|
||||
"handle": "nexta_tv",
|
||||
"label": "NEXTA",
|
||||
"topic": "politics",
|
||||
"tier": 3,
|
||||
"enabled": true,
|
||||
"region": "europe",
|
||||
"maxMessages": 15
|
||||
}
|
||||
],
|
||||
"tech": [],
|
||||
"finance": []
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
# Nginx API proxy compression baseline for WorldMonitor.
|
||||
# Requires ngx_brotli (or Nginx Plus Brotli module) to be installed.
|
||||
|
||||
# Prefer Brotli for HTTPS clients and keep gzip as fallback.
|
||||
brotli on;
|
||||
brotli_comp_level 5;
|
||||
brotli_min_length 1024;
|
||||
brotli_types application/json application/javascript text/css text/plain application/xml text/xml;
|
||||
|
||||
gzip on;
|
||||
gzip_comp_level 5;
|
||||
gzip_min_length 1024;
|
||||
gzip_vary on;
|
||||
gzip_proxied any;
|
||||
gzip_types application/json application/javascript text/css text/plain application/xml text/xml;
|
||||
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name api.worldmonitor.local;
|
||||
|
||||
location /api/ {
|
||||
proxy_pass http://127.0.0.1:8787;
|
||||
proxy_http_version 1.1;
|
||||
|
||||
# Preserve upstream compression behavior and pass through client preferences.
|
||||
proxy_set_header Accept-Encoding $http_accept_encoding;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# If upstream sends pre-compressed content, do not decompress.
|
||||
gunzip off;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,461 @@
|
||||
# Adding API Endpoints
|
||||
|
||||
All JSON API endpoints in WorldMonitor **must** use sebuf. Do not create standalone `api/*.js` files — the legacy pattern is deprecated and being removed.
|
||||
|
||||
This guide walks through adding a new RPC to an existing service and adding an entirely new service.
|
||||
|
||||
> **Important:** After modifying any `.proto` file, you **must** run `make generate` before building or pushing. The generated TypeScript files in `src/generated/` are checked into the repo and must stay in sync with the proto definitions. CI does not run generation yet — this is your responsibility until we add it to the pipeline (see [#200](https://github.com/koala73/worldmonitor/issues/200)).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You need **Go 1.21+** and **Node.js 18+** installed. Everything else is installed automatically:
|
||||
|
||||
```bash
|
||||
make install # one-time: installs buf, sebuf plugins, npm deps, proto deps
|
||||
```
|
||||
|
||||
This installs:
|
||||
|
||||
- **buf** — proto linting, dependency management, and code generation orchestrator
|
||||
- **protoc-gen-ts-client** — generates TypeScript client classes (from [sebuf](https://github.com/SebastienMelki/sebuf))
|
||||
- **protoc-gen-ts-server** — generates TypeScript server handler interfaces (from sebuf)
|
||||
- **protoc-gen-openapiv3** — generates OpenAPI v3 specs (from sebuf)
|
||||
- **npm dependencies** — all Node.js packages
|
||||
|
||||
Run code generation from the repo root:
|
||||
|
||||
```bash
|
||||
make generate # regenerate all TypeScript + OpenAPI from protos
|
||||
```
|
||||
|
||||
This produces three outputs per service:
|
||||
|
||||
- `src/generated/client/{domain}/v1/service_client.ts` — typed fetch client for the frontend
|
||||
- `src/generated/server/{domain}/v1/service_server.ts` — handler interface + route factory for the backend
|
||||
- `docs/api/{Domain}Service.openapi.yaml` + `.json` — OpenAPI v3 documentation
|
||||
|
||||
## Adding an RPC to an existing service
|
||||
|
||||
Example: adding `GetEarthquakeDetails` to `SeismologyService`.
|
||||
|
||||
### 1. Define the request/response messages
|
||||
|
||||
Create `proto/worldmonitor/seismology/v1/get_earthquake_details.proto`:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package worldmonitor.seismology.v1;
|
||||
|
||||
import "buf/validate/validate.proto";
|
||||
import "worldmonitor/seismology/v1/earthquake.proto";
|
||||
|
||||
// GetEarthquakeDetailsRequest specifies which earthquake to retrieve.
|
||||
message GetEarthquakeDetailsRequest {
|
||||
// USGS event identifier (e.g., "us7000abcd").
|
||||
string earthquake_id = 1 [
|
||||
(buf.validate.field).required = true,
|
||||
(buf.validate.field).string.min_len = 1,
|
||||
(buf.validate.field).string.max_len = 100
|
||||
];
|
||||
}
|
||||
|
||||
// GetEarthquakeDetailsResponse contains the full earthquake record.
|
||||
message GetEarthquakeDetailsResponse {
|
||||
// The earthquake matching the requested ID.
|
||||
Earthquake earthquake = 1;
|
||||
}
|
||||
```
|
||||
|
||||
### 2. Add the RPC to the service definition
|
||||
|
||||
Edit `proto/worldmonitor/seismology/v1/service.proto`:
|
||||
|
||||
```protobuf
|
||||
import "worldmonitor/seismology/v1/get_earthquake_details.proto";
|
||||
|
||||
service SeismologyService {
|
||||
// ... existing RPCs ...
|
||||
|
||||
// GetEarthquakeDetails retrieves a single earthquake by its USGS event ID.
|
||||
rpc GetEarthquakeDetails(GetEarthquakeDetailsRequest) returns (GetEarthquakeDetailsResponse) {
|
||||
option (sebuf.http.config) = {path: "/get-earthquake-details"};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Lint and generate
|
||||
|
||||
```bash
|
||||
make check # lint + generate in one step
|
||||
```
|
||||
|
||||
At this point, `npx tsc --noEmit` will **fail** because the handler doesn't implement the new method yet. This is by design — the compiler enforces the contract.
|
||||
|
||||
### 4. Implement the handler
|
||||
|
||||
Create `server/worldmonitor/seismology/v1/get-earthquake-details.ts`:
|
||||
|
||||
```typescript
|
||||
import type {
|
||||
SeismologyServiceHandler,
|
||||
ServerContext,
|
||||
GetEarthquakeDetailsRequest,
|
||||
GetEarthquakeDetailsResponse,
|
||||
} from '../../../../src/generated/server/worldmonitor/seismology/v1/service_server';
|
||||
|
||||
export const getEarthquakeDetails: SeismologyServiceHandler['getEarthquakeDetails'] = async (
|
||||
_ctx: ServerContext,
|
||||
req: GetEarthquakeDetailsRequest,
|
||||
): Promise<GetEarthquakeDetailsResponse> => {
|
||||
const response = await fetch(
|
||||
`https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/${req.earthquakeId}.geojson`,
|
||||
);
|
||||
if (!response.ok) {
|
||||
throw new Error(`USGS API error: ${response.status}`);
|
||||
}
|
||||
const f: any = await response.json();
|
||||
return {
|
||||
earthquake: {
|
||||
id: f.id,
|
||||
place: f.properties.place || '',
|
||||
magnitude: f.properties.mag ?? 0,
|
||||
depthKm: f.geometry.coordinates[2] ?? 0,
|
||||
location: {
|
||||
latitude: f.geometry.coordinates[1],
|
||||
longitude: f.geometry.coordinates[0],
|
||||
},
|
||||
occurredAt: f.properties.time,
|
||||
sourceUrl: f.properties.url || '',
|
||||
},
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
### 5. Wire it into the handler re-export
|
||||
|
||||
Edit `server/worldmonitor/seismology/v1/handler.ts`:
|
||||
|
||||
```typescript
|
||||
import type { SeismologyServiceHandler } from '../../../../src/generated/server/worldmonitor/seismology/v1/service_server';
|
||||
|
||||
import { listEarthquakes } from './list-earthquakes';
|
||||
import { getEarthquakeDetails } from './get-earthquake-details';
|
||||
|
||||
export const seismologyHandler: SeismologyServiceHandler = {
|
||||
listEarthquakes,
|
||||
getEarthquakeDetails,
|
||||
};
|
||||
```
|
||||
|
||||
### 6. Verify
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit # should pass with zero errors
|
||||
```
|
||||
|
||||
The route is already live. `createSeismologyServiceRoutes()` picks up the new RPC automatically — no changes needed to `api/[[...path]].ts` or `vite.config.ts`.
|
||||
|
||||
### 7. Check the generated docs
|
||||
|
||||
Open `docs/api/SeismologyService.openapi.yaml` — the new endpoint should appear with all validation constraints from your proto annotations.
|
||||
|
||||
## Adding a new service
|
||||
|
||||
Example: adding a `SanctionsService`.
|
||||
|
||||
### 1. Create the proto directory
|
||||
|
||||
```
|
||||
proto/worldmonitor/sanctions/v1/
|
||||
```
|
||||
|
||||
### 2. Define entity messages
|
||||
|
||||
Create `proto/worldmonitor/sanctions/v1/sanctions_entry.proto`:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package worldmonitor.sanctions.v1;
|
||||
|
||||
import "buf/validate/validate.proto";
|
||||
import "sebuf/http/annotations.proto";
|
||||
|
||||
// SanctionsEntry represents a single entity on a sanctions list.
|
||||
message SanctionsEntry {
|
||||
// Unique identifier.
|
||||
string id = 1 [
|
||||
(buf.validate.field).required = true,
|
||||
(buf.validate.field).string.min_len = 1
|
||||
];
|
||||
// Name of the sanctioned entity or individual.
|
||||
string name = 2;
|
||||
// Issuing authority (e.g., "OFAC", "EU", "UN").
|
||||
string authority = 3;
|
||||
// ISO 3166-1 alpha-2 country code of the target.
|
||||
string country_code = 4;
|
||||
// Date the sanction was imposed, as Unix epoch milliseconds.
|
||||
int64 imposed_at = 5 [(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER];
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Define request/response messages
|
||||
|
||||
Create `proto/worldmonitor/sanctions/v1/list_sanctions.proto`:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package worldmonitor.sanctions.v1;
|
||||
|
||||
import "buf/validate/validate.proto";
|
||||
import "worldmonitor/core/v1/pagination.proto";
|
||||
import "worldmonitor/sanctions/v1/sanctions_entry.proto";
|
||||
|
||||
// ListSanctionsRequest specifies filters for sanctions data.
|
||||
message ListSanctionsRequest {
|
||||
// Filter by issuing authority (e.g., "OFAC"). Empty returns all.
|
||||
string authority = 1;
|
||||
// Filter by country code.
|
||||
string country_code = 2 [(buf.validate.field).string.max_len = 2];
|
||||
// Pagination parameters.
|
||||
worldmonitor.core.v1.PaginationRequest pagination = 3;
|
||||
}
|
||||
|
||||
// ListSanctionsResponse contains the matching sanctions entries.
|
||||
message ListSanctionsResponse {
|
||||
// The list of sanctions entries.
|
||||
repeated SanctionsEntry entries = 1;
|
||||
// Pagination metadata.
|
||||
worldmonitor.core.v1.PaginationResponse pagination = 2;
|
||||
}
|
||||
```
|
||||
|
||||
### 4. Define the service
|
||||
|
||||
Create `proto/worldmonitor/sanctions/v1/service.proto`:
|
||||
|
||||
```protobuf
|
||||
syntax = "proto3";
|
||||
package worldmonitor.sanctions.v1;
|
||||
|
||||
import "sebuf/http/annotations.proto";
|
||||
import "worldmonitor/sanctions/v1/list_sanctions.proto";
|
||||
|
||||
// SanctionsService provides APIs for international sanctions monitoring.
|
||||
service SanctionsService {
|
||||
option (sebuf.http.service_config) = {base_path: "/api/sanctions/v1"};
|
||||
|
||||
// ListSanctions retrieves sanctions entries matching the given filters.
|
||||
rpc ListSanctions(ListSanctionsRequest) returns (ListSanctionsResponse) {
|
||||
option (sebuf.http.config) = {path: "/list-sanctions"};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5. Generate
|
||||
|
||||
```bash
|
||||
make check # lint + generate in one step
|
||||
```
|
||||
|
||||
### 6. Implement the handler
|
||||
|
||||
Create the handler directory and files:
|
||||
|
||||
```
|
||||
server/worldmonitor/sanctions/v1/
|
||||
├── handler.ts # thin re-export
|
||||
└── list-sanctions.ts # RPC implementation
|
||||
```
|
||||
|
||||
`server/worldmonitor/sanctions/v1/list-sanctions.ts`:
|
||||
```typescript
|
||||
import type {
|
||||
SanctionsServiceHandler,
|
||||
ServerContext,
|
||||
ListSanctionsRequest,
|
||||
ListSanctionsResponse,
|
||||
} from '../../../../src/generated/server/worldmonitor/sanctions/v1/service_server';
|
||||
|
||||
export const listSanctions: SanctionsServiceHandler['listSanctions'] = async (
|
||||
_ctx: ServerContext,
|
||||
req: ListSanctionsRequest,
|
||||
): Promise<ListSanctionsResponse> => {
|
||||
// Your implementation here — fetch from upstream API, transform to proto shape
|
||||
return { entries: [], pagination: undefined };
|
||||
};
|
||||
```
|
||||
|
||||
`server/worldmonitor/sanctions/v1/handler.ts`:
|
||||
```typescript
|
||||
import type { SanctionsServiceHandler } from '../../../../src/generated/server/worldmonitor/sanctions/v1/service_server';
|
||||
|
||||
import { listSanctions } from './list-sanctions';
|
||||
|
||||
export const sanctionsHandler: SanctionsServiceHandler = {
|
||||
listSanctions,
|
||||
};
|
||||
```
|
||||
|
||||
### 7. Register the service in the gateway
|
||||
|
||||
Edit `api/[[...path]].js` — add the import and mount the routes:
|
||||
|
||||
```typescript
|
||||
import { createSanctionsServiceRoutes } from '../src/generated/server/worldmonitor/sanctions/v1/service_server';
|
||||
import { sanctionsHandler } from './server/worldmonitor/sanctions/v1/handler';
|
||||
|
||||
const allRoutes = [
|
||||
// ... existing routes ...
|
||||
...createSanctionsServiceRoutes(sanctionsHandler, serverOptions),
|
||||
];
|
||||
```
|
||||
|
||||
### 8. Register in the Vite dev server
|
||||
|
||||
Edit `vite.config.ts` — add the lazy import and route mount inside the `sebufApiPlugin()` function. Follow the existing pattern (search for any other service to see the exact locations).
|
||||
|
||||
### 9. Create the frontend service wrapper
|
||||
|
||||
Create `src/services/sanctions.ts`:
|
||||
|
||||
```typescript
|
||||
import {
|
||||
SanctionsServiceClient,
|
||||
type SanctionsEntry,
|
||||
type ListSanctionsResponse,
|
||||
} from '@/generated/client/worldmonitor/sanctions/v1/service_client';
|
||||
import { createCircuitBreaker } from '@/utils';
|
||||
|
||||
export type { SanctionsEntry };
|
||||
|
||||
const client = new SanctionsServiceClient('', { fetch: fetch.bind(globalThis) });
|
||||
const breaker = createCircuitBreaker<ListSanctionsResponse>({ name: 'Sanctions' });
|
||||
|
||||
const emptyFallback: ListSanctionsResponse = { entries: [] };
|
||||
|
||||
export async function fetchSanctions(authority?: string): Promise<SanctionsEntry[]> {
|
||||
const response = await breaker.execute(async () => {
|
||||
return client.listSanctions({ authority: authority ?? '', countryCode: '', pagination: undefined });
|
||||
}, emptyFallback);
|
||||
return response.entries;
|
||||
}
|
||||
```
|
||||
|
||||
### 10. Verify
|
||||
|
||||
```bash
|
||||
npx tsc --noEmit # zero errors
|
||||
```
|
||||
|
||||
## Proto conventions
|
||||
|
||||
These conventions are enforced across the codebase. Follow them for consistency.
|
||||
|
||||
### File naming
|
||||
|
||||
- One file per message type: `earthquake.proto`, `sanctions_entry.proto`
|
||||
- One file per RPC pair: `list_earthquakes.proto`, `get_earthquake_details.proto`
|
||||
- Service definition: `service.proto`
|
||||
- Use `snake_case` for file names and field names
|
||||
|
||||
### Time fields
|
||||
|
||||
Always use `int64` with Unix epoch milliseconds. Never use `google.protobuf.Timestamp`.
|
||||
|
||||
Always add the `INT64_ENCODING_NUMBER` annotation so TypeScript gets `number` instead of `string`:
|
||||
|
||||
```protobuf
|
||||
int64 occurred_at = 6 [(sebuf.http.int64_encoding) = INT64_ENCODING_NUMBER];
|
||||
```
|
||||
|
||||
### Validation annotations
|
||||
|
||||
Import `buf/validate/validate.proto` and annotate fields at the proto level. These constraints flow through to the generated OpenAPI spec automatically.
|
||||
|
||||
Common patterns:
|
||||
|
||||
```protobuf
|
||||
// Required string with length bounds
|
||||
string id = 1 [
|
||||
(buf.validate.field).required = true,
|
||||
(buf.validate.field).string.min_len = 1,
|
||||
(buf.validate.field).string.max_len = 100
|
||||
];
|
||||
|
||||
// Numeric range (e.g., score 0-100)
|
||||
double risk_score = 2 [
|
||||
(buf.validate.field).double.gte = 0,
|
||||
(buf.validate.field).double.lte = 100
|
||||
];
|
||||
|
||||
// Non-negative value
|
||||
double min_magnitude = 3 [(buf.validate.field).double.gte = 0];
|
||||
|
||||
// Coordinate bounds (prefer using core.v1.GeoCoordinates instead)
|
||||
double latitude = 1 [
|
||||
(buf.validate.field).double.gte = -90,
|
||||
(buf.validate.field).double.lte = 90
|
||||
];
|
||||
```
|
||||
|
||||
### Shared core types
|
||||
|
||||
Reuse these instead of redefining:
|
||||
|
||||
| Type | Import | Use for |
|
||||
|------|--------|---------|
|
||||
| `GeoCoordinates` | `worldmonitor/core/v1/geo.proto` | Any lat/lon location (has built-in -90/90 and -180/180 bounds) |
|
||||
| `BoundingBox` | `worldmonitor/core/v1/geo.proto` | Spatial filtering |
|
||||
| `TimeRange` | `worldmonitor/core/v1/time.proto` | Time-based filtering (has `INT64_ENCODING_NUMBER`) |
|
||||
| `PaginationRequest` | `worldmonitor/core/v1/pagination.proto` | Request pagination (has page_size 1-100 constraint) |
|
||||
| `PaginationResponse` | `worldmonitor/core/v1/pagination.proto` | Response pagination metadata |
|
||||
|
||||
### Comments
|
||||
|
||||
buf lint enforces comments on all messages, fields, services, RPCs, and enum values. Every proto element must have a `//` comment. This is not optional — `buf lint` will fail without them.
|
||||
|
||||
### Route paths
|
||||
|
||||
- Service base path: `/api/{domain}/v1`
|
||||
- RPC path: `/{verb}-{noun}` in kebab-case (e.g., `/list-earthquakes`, `/get-vessel-snapshot`)
|
||||
|
||||
### Handler typing
|
||||
|
||||
Always type the handler function against the generated interface using indexed access:
|
||||
|
||||
```typescript
|
||||
export const listSanctions: SanctionsServiceHandler['listSanctions'] = async (
|
||||
_ctx: ServerContext,
|
||||
req: ListSanctionsRequest,
|
||||
): Promise<ListSanctionsResponse> => {
|
||||
// ...
|
||||
};
|
||||
```
|
||||
|
||||
This ensures the compiler catches any mismatch between your implementation and the proto contract.
|
||||
|
||||
### Client construction
|
||||
|
||||
Always pass `{ fetch: fetch.bind(globalThis) }` when creating clients:
|
||||
|
||||
```typescript
|
||||
const client = new SanctionsServiceClient('', { fetch: fetch.bind(globalThis) });
|
||||
```
|
||||
|
||||
The empty string base URL works because both Vite dev server and Vercel serve the API on the same origin. The `fetch.bind(globalThis)` is required for Tauri compatibility.
|
||||
|
||||
## Generated documentation
|
||||
|
||||
Every time you run `make generate`, OpenAPI v3 specs are generated for each service:
|
||||
|
||||
- `docs/api/{Domain}Service.openapi.yaml` — human-readable YAML
|
||||
- `docs/api/{Domain}Service.openapi.json` — machine-readable JSON
|
||||
|
||||
These specs include:
|
||||
|
||||
- All endpoints with request/response schemas
|
||||
- Validation constraints from `buf.validate` annotations (min/max, required fields, ranges)
|
||||
- Field descriptions from proto comments
|
||||
- Error response schemas (400 validation errors, 500 server errors)
|
||||
|
||||
You do not need to write or maintain OpenAPI specs by hand. They are generated artifacts. If you need to change the API documentation, change the proto and regenerate.
|
||||
@@ -0,0 +1,140 @@
|
||||
# API Key Gating & Registration — Deployment Guide
|
||||
|
||||
## Overview
|
||||
|
||||
Desktop cloud fallback is gated on a `WORLDMONITOR_API_KEY`. Without a valid key, the desktop app operates local-only (sidecar). A registration form collects emails via Convex DB for future key distribution.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Desktop App Cloud (Vercel)
|
||||
┌──────────────────┐ ┌──────────────────────┐
|
||||
│ fetch('/api/...')│ │ api/[domain]/v1/[rpc]│
|
||||
│ │ │ │ │ │
|
||||
│ ┌──────▼───────┐ │ │ ┌──────▼───────┐ │
|
||||
│ │ sidecar try │ │ │ │ validateApiKey│ │
|
||||
│ │ (local-first)│ │ │ │ (origin-aware)│ │
|
||||
│ └──────┬───────┘ │ │ └──────┬───────┘ │
|
||||
│ fail │ │ │ 401 if invalid │
|
||||
│ ┌──────▼───────┐ │ fallback │ │
|
||||
│ │ WM key check │─┼──────────────►│ ┌──────────────┐ │
|
||||
│ │ (gate) │ │ +header │ │ route handler │ │
|
||||
│ └──────────────┘ │ │ └──────────────┘ │
|
||||
└──────────────────┘ └──────────────────────┘
|
||||
```
|
||||
|
||||
## Required Environment Variables
|
||||
|
||||
### Vercel
|
||||
|
||||
| Variable | Description | Example |
|
||||
|----------|-------------|---------|
|
||||
| `WORLDMONITOR_VALID_KEYS` | Comma-separated list of valid API keys | `wm_abc123def456,wm_xyz789` |
|
||||
| `CONVEX_URL` | Convex deployment URL (from `npx convex deploy`) | `https://xyz-123.convex.cloud` |
|
||||
|
||||
### Generating API keys
|
||||
|
||||
Keys must be at least 16 characters (validated client-side). Recommended format:
|
||||
|
||||
```bash
|
||||
# Generate a key
|
||||
openssl rand -hex 24 | sed 's/^/wm_/'
|
||||
# Example output: wm_a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6
|
||||
```
|
||||
|
||||
Add to `WORLDMONITOR_VALID_KEYS` in Vercel dashboard (comma-separated, no spaces).
|
||||
|
||||
## Convex Setup
|
||||
|
||||
### First-time deployment
|
||||
|
||||
```bash
|
||||
# 1. Install (already in package.json)
|
||||
npm install
|
||||
|
||||
# 2. Login to Convex
|
||||
npx convex login
|
||||
|
||||
# 3. Initialize project (creates .env.local with CONVEX_URL)
|
||||
npx convex init
|
||||
|
||||
# 4. Deploy schema and functions
|
||||
npx convex deploy
|
||||
|
||||
# 5. Copy the deployment URL to Vercel env vars
|
||||
# The URL is printed by `npx convex deploy` and saved in .env.local
|
||||
```
|
||||
|
||||
### Verify Convex deployment
|
||||
|
||||
```bash
|
||||
# Typecheck Convex functions
|
||||
npx convex dev --typecheck
|
||||
|
||||
# Open Convex dashboard to see registrations
|
||||
npx convex dashboard
|
||||
```
|
||||
|
||||
### Schema
|
||||
|
||||
The `registrations` table stores:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `email` | string | Original email (for display) |
|
||||
| `normalizedEmail` | string | Lowercased email (for dedup) |
|
||||
| `registeredAt` | number | Unix timestamp |
|
||||
| `source` | string? | Where the registration came from |
|
||||
| `appVersion` | string? | Desktop app version |
|
||||
|
||||
Indexed by `normalizedEmail` for duplicate detection.
|
||||
|
||||
## Security Model
|
||||
|
||||
### Client-side (desktop app)
|
||||
|
||||
- `installRuntimeFetchPatch()` checks `WORLDMONITOR_API_KEY` before allowing cloud fallback
|
||||
- Key must be present AND valid (min 16 chars)
|
||||
- `secretsReady` promise ensures secrets are loaded before first fetch (2s timeout)
|
||||
- Fail-closed: any error in key check blocks cloud fallback
|
||||
|
||||
### Server-side (Vercel edge)
|
||||
|
||||
- `api/_api-key.js` validates `X-WorldMonitor-Key` header on sebuf routes
|
||||
- **Origin-aware**: desktop origins (`tauri.localhost`, `tauri://`, `asset://`) require a key
|
||||
- Web origins (`worldmonitor.app`) pass through without a key
|
||||
- Non-desktop origin with key header: key is still validated
|
||||
- Invalid key returns `401 { error: "Invalid API key" }`
|
||||
|
||||
### CORS
|
||||
|
||||
`X-WorldMonitor-Key` is allowed in both `server/cors.ts` and `api/_cors.js`.
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After deployment:
|
||||
|
||||
- [ ] Set `WORLDMONITOR_VALID_KEYS` in Vercel
|
||||
- [ ] Set `CONVEX_URL` in Vercel
|
||||
- [ ] Run `npx convex deploy` to push schema
|
||||
- [ ] Desktop without key: cloud fallback blocked (console shows `cloud fallback blocked`)
|
||||
- [ ] Desktop with invalid key: sebuf requests get `401`
|
||||
- [ ] Desktop with valid key: cloud fallback works as before
|
||||
- [ ] Web access: no key required, works normally
|
||||
- [ ] Registration form: submit email, check Convex dashboard
|
||||
- [ ] Duplicate email: shows "already registered"
|
||||
- [ ] Existing settings tabs (LLMs, API Keys, Debug) unchanged
|
||||
|
||||
## Files Reference
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `src/services/runtime.ts` | Client-side key gate + header attachment |
|
||||
| `src/services/runtime-config.ts` | `WORLDMONITOR_API_KEY` type, validation, `secretsReady` |
|
||||
| `api/_api-key.js` | Server-side key validation (origin-aware) |
|
||||
| `api/[domain]/v1/[rpc].ts` | Sebuf gateway — calls `validateApiKey` |
|
||||
| `api/register-interest.js` | Registration endpoint → Convex |
|
||||
| `server/cors.ts` / `api/_cors.js` | CORS headers with `X-WorldMonitor-Key` |
|
||||
| `src/components/WorldMonitorTab.ts` | Settings UI for key + registration |
|
||||
| `convex/schema.ts` | Convex DB schema |
|
||||
| `convex/registerInterest.ts` | Convex mutation |
|
||||
@@ -0,0 +1,184 @@
|
||||
# World Monitor — Community Promotion Guide
|
||||
|
||||
Thank you for helping spread the word about World Monitor! This guide provides talking points, must-see features, and visual suggestions to help you create compelling content for your audience.
|
||||
|
||||
---
|
||||
|
||||
## What is World Monitor?
|
||||
|
||||
**One-line pitch**: A free, open-source, real-time global intelligence dashboard — like Bloomberg Terminal meets OSINT, for everyone.
|
||||
|
||||
**Longer description**: World Monitor aggregates 150+ news feeds, military tracking, financial markets, conflict data, protest monitoring, satellite imagery, and AI-powered analysis into a single unified dashboard with an interactive globe. Available as a web app, desktop app (macOS/Windows/Linux), and installable PWA.
|
||||
|
||||
---
|
||||
|
||||
## Key URLs
|
||||
|
||||
| Link | Description |
|
||||
|------|-------------|
|
||||
| [worldmonitor.app](https://worldmonitor.app) | Main dashboard — geopolitics, military, conflicts |
|
||||
| [tech.worldmonitor.app](https://tech.worldmonitor.app) | Tech variant — startups, AI/ML, cybersecurity |
|
||||
| [finance.worldmonitor.app](https://finance.worldmonitor.app) | Finance variant — markets, exchanges, central banks |
|
||||
| [GitHub](https://github.com/koala73/worldmonitor) | Source code (AGPL-3.0) |
|
||||
|
||||
---
|
||||
|
||||
## Must-See Features (Top 10)
|
||||
|
||||
### 1. Interactive Globe with 35+ Data Layers
|
||||
|
||||
The centerpiece. A WebGL-accelerated globe (deck.gl) with toggleable layers for conflicts, military bases, nuclear facilities, undersea cables, pipelines, satellite fires, protests, cyber threats, and more. Zoom in and the detail layers progressively reveal.
|
||||
|
||||
**Show**: Toggle different layers on/off. Zoom into a conflict region. Show the layer panel.
|
||||
|
||||
### 2. AI-Powered World Brief
|
||||
|
||||
One-click AI summary of the top global developments. Three-tier LLM provider chain: local Ollama/LM Studio (fully private, offline), Groq (fast cloud), or OpenRouter (fallback). Redis caching for instant responses on repeat queries.
|
||||
|
||||
**Show**: The summary card at the top of the news panel.
|
||||
|
||||
### 3. Country Intelligence Dossiers
|
||||
|
||||
Click any country on the map for a full-page intelligence brief: instability score ring, AI-generated analysis, top headlines, prediction markets, 7-day event timeline, active signal chips, infrastructure exposure, and stock market data.
|
||||
|
||||
**Show**: Click a country (e.g., Japan, Ukraine, or Iran) → full dossier page.
|
||||
|
||||
### 4. 14 Languages Support
|
||||
|
||||
Full UI in 14 languages including Japanese. Regional news feeds auto-adapt — Japanese users see NHK World, Nikkei Asia, and Japan-relevant sources. Language bundles are lazy-loaded for fast performance.
|
||||
|
||||
**Show**: Switch language to Japanese in the settings. Note how feeds change.
|
||||
|
||||
### 5. Live Military Tracking
|
||||
|
||||
Real-time ADS-B military flight tracking and AIS naval vessel monitoring. Strategic Posture panel shows theater-level risk assessment across 9 global regions (Baltic, Black Sea, South China Sea, Eastern Mediterranean, etc.).
|
||||
|
||||
**Show**: Enable the Military layer. Show the Strategic Posture panel.
|
||||
|
||||
### 6. Three Variant Dashboards
|
||||
|
||||
One codebase, three specialized views — switch between World (geopolitics), Tech (startups/AI), and Finance (markets/exchanges) with one click in the header bar.
|
||||
|
||||
**Show**: Click the variant switcher (🌍 WORLD | 💻 TECH | 📈 FINANCE).
|
||||
|
||||
### 7. Market & Crypto Intelligence
|
||||
|
||||
7-signal macro radar with composite BUY/CASH verdict, BTC spot ETF flow tracker, stablecoin peg monitor, Fear & Greed Index, and Bitcoin technical indicators. Sparkline charts and donut gauges for visual trends.
|
||||
|
||||
**Show**: Scroll to the crypto/market panels. Point out the sparklines.
|
||||
|
||||
### 8. Live Video & Webcam Feeds
|
||||
|
||||
8 live news streams (Bloomberg, Al Jazeera, Sky News, etc.) + 19 live webcams from geopolitical hotspots across 4 regions. Idle-aware — auto-pauses after 5 minutes of inactivity.
|
||||
|
||||
**Show**: Open the video panel or webcam panel.
|
||||
|
||||
### 9. Desktop Application (Free)
|
||||
|
||||
Native app for macOS, Windows, and Linux via Tauri. API keys stored in OS keychain (not plaintext). Local Node.js sidecar runs all 60+ API handlers offline-capable. Run local LLMs for fully private, offline AI summaries.
|
||||
|
||||
**Show**: The download buttons on the site, or the desktop app running natively.
|
||||
|
||||
### 10. Story Sharing & Social Export
|
||||
|
||||
Generate intelligence briefs for any country and share to Twitter/X, LinkedIn, WhatsApp, Telegram, Reddit. Includes canvas-rendered PNG images with QR codes linking back to the live dashboard.
|
||||
|
||||
**Show**: Generate a story for a country → share dialog with platform options.
|
||||
|
||||
### 11. Local LLM Support (Ollama / LM Studio)
|
||||
|
||||
Run AI summarization entirely on your own hardware — no API keys, no cloud, no data leaving your machine. The desktop app auto-discovers models from Ollama or LM Studio, with a three-tier fallback chain: local → Groq → OpenRouter. Settings are split into dedicated LLMs and API Keys tabs for easy configuration.
|
||||
|
||||
**Show**: Open Settings → LLMs tab → Ollama model dropdown auto-populated → generate a summary with the local model.
|
||||
|
||||
---
|
||||
|
||||
## Visual Content Suggestions
|
||||
|
||||
### Screenshots Worth Taking
|
||||
|
||||
1. **Full dashboard overview** — globe in center, panels on sides, news feed visible
|
||||
2. **Country dossier page** — click Japan or a hotspot country, show the full brief
|
||||
3. **Layer toggle demo** — before/after with conflicts + military bases enabled
|
||||
4. **Finance variant** — stock exchanges, financial centers, market panels
|
||||
5. **Japanese UI** — show the language switcher and Japanese interface
|
||||
6. **Webcam grid** — 4 live feeds from different regions
|
||||
7. **Strategic Posture** — theater risk levels panel
|
||||
8. **Settings LLMs tab** — Ollama model dropdown with local models discovered
|
||||
|
||||
### Video/GIF Ideas
|
||||
|
||||
1. **30-second tour**: Open site → rotate globe → toggle layers → click country → show brief
|
||||
2. **Language switch**: English → Japanese, show how feeds adapt
|
||||
3. **Layer stacking**: Start empty → add conflicts → military → cyber → fires → wow
|
||||
4. **Variant switching**: World → Tech → Finance in quick succession
|
||||
|
||||
---
|
||||
|
||||
## Talking Points for Posts
|
||||
|
||||
### For General Audience
|
||||
|
||||
- "An open-source Bloomberg Terminal for everyone — free, no login required"
|
||||
- "150+ news sources, military tracking, AI analysis — all in one dashboard"
|
||||
- "Run AI summaries locally with Ollama — your data never leaves your machine"
|
||||
- "Available in Japanese with NHK and Nikkei feeds built in"
|
||||
- "Native desktop app for macOS/Windows/Linux, completely free"
|
||||
|
||||
### For Tech Audience
|
||||
|
||||
- "Built with TypeScript, Vite, deck.gl, MapLibre GL, Tauri"
|
||||
- "35+ WebGL data layers running at 60fps"
|
||||
- "ONNX Runtime Web for browser-based ML inference (sentiment, NER, summarization)"
|
||||
- "Local LLM support — plug in Ollama or LM Studio, zero cloud dependency"
|
||||
- "Open source under AGPL-3.0 — contribute on GitHub"
|
||||
|
||||
### For Finance/OSINT Audience
|
||||
|
||||
- "7-signal crypto macro radar with BUY/CASH composite verdict"
|
||||
- "92 global stock exchanges mapped with market caps and trading hours"
|
||||
- "Country Instability Index tracking 22 nations in real-time"
|
||||
- "Prediction market integration for geopolitical forecasting"
|
||||
- "Air-gapped AI analysis — run Ollama locally for sensitive intelligence work"
|
||||
|
||||
### For Japanese Audience Specifically
|
||||
|
||||
- 日本語完全対応 — UI、ニュースフィード、AI要約すべて日本語で利用可能
|
||||
- NHK World、日経アジアなど日本向けニュースソース内蔵
|
||||
- 無料・オープンソース — アカウント登録不要
|
||||
- macOS/Windows/Linux対応のデスクトップアプリあり
|
||||
|
||||
---
|
||||
|
||||
## Recent Major Features (Changelog Highlights)
|
||||
|
||||
| Version | Feature |
|
||||
|---------|---------|
|
||||
| v2.5.1 | Batch FRED fetching, parallel UCDP, partial cache TTL, bot middleware |
|
||||
| v2.5.0 | Ollama/LM Studio local LLM support, settings split into LLMs + API Keys tabs, keychain vault consolidation |
|
||||
| v2.4.1 | Ultra-wide layout (panels wrap around map on 2000px+ screens) |
|
||||
| v2.4.0 | Live webcams from 19 geopolitical hotspots, 4 regions |
|
||||
| v2.3.9 | Full i18n: 14 languages including Japanese, Arabic (RTL), Chinese |
|
||||
| v2.3.8 | Finance variant with 92 exchanges, Gulf FDI investments |
|
||||
| v2.3.7 | Light/dark theme system, UCDP/UNHCR/Climate panels |
|
||||
| v2.3.6 | Desktop app with Tauri, OS keychain, auto-updates |
|
||||
| v2.3.0 | Country Intelligence dossiers, story sharing |
|
||||
|
||||
---
|
||||
|
||||
## Branding Notes
|
||||
|
||||
- **Name**: "World Monitor" (two words, capitalized)
|
||||
- **Tagline**: "Real-time global intelligence dashboard"
|
||||
- **License**: AGPL-3.0 (free and open source)
|
||||
- **Creator**: Credit "World Monitor by Elie Habib" or link to the GitHub repo
|
||||
- **Variants**: You can mention all three (World/Tech/Finance) or focus on the main one
|
||||
- **No login required**: Anyone can use the web app immediately — no signup, no paywall
|
||||
|
||||
---
|
||||
|
||||
## Thank You
|
||||
|
||||
We genuinely appreciate community members helping grow World Monitor's reach. Feel free to interpret these guidelines creatively — there's no strict template. The most compelling content comes from showing what YOU find most interesting or useful about the tool.
|
||||
|
||||
If you have questions or want specific screenshots/assets, open a Discussion on the GitHub repo or reach out directly.
|
||||
@@ -4,7 +4,7 @@ World Monitor desktop now uses a runtime configuration schema with per-feature t
|
||||
|
||||
## Secret keys
|
||||
|
||||
The desktop vault schema supports the following 17 keys used by services and relays:
|
||||
The desktop vault schema (Rust `SUPPORTED_SECRET_KEYS`) supports the following 22 keys:
|
||||
|
||||
- `GROQ_API_KEY`
|
||||
- `OPENROUTER_API_KEY`
|
||||
@@ -18,11 +18,18 @@ The desktop vault schema supports the following 17 keys used by services and rel
|
||||
- `ABUSEIPDB_API_KEY`
|
||||
- `NASA_FIRMS_API_KEY`
|
||||
- `WINGBITS_API_KEY`
|
||||
- `WS_RELAY_URL`
|
||||
- `VITE_WS_RELAY_URL`
|
||||
- `VITE_OPENSKY_RELAY_URL`
|
||||
- `OPENSKY_CLIENT_ID`
|
||||
- `OPENSKY_CLIENT_SECRET`
|
||||
- `AISSTREAM_API_KEY`
|
||||
- `VITE_WS_RELAY_URL`
|
||||
- `OLLAMA_API_URL`
|
||||
- `OLLAMA_MODEL`
|
||||
- `WORLDMONITOR_API_KEY` — gates cloud fallback access (min 16 chars)
|
||||
- `WTO_API_KEY`
|
||||
|
||||
Note: `UC_DP_KEY` exists in the TypeScript `RuntimeSecretKey` union but is not in the desktop Rust keychain or sidecar.
|
||||
|
||||
## Feature schema
|
||||
|
||||
@@ -51,3 +58,4 @@ If required secrets are missing/disabled:
|
||||
- NASA FIRMS: satellite fire detection returns empty state.
|
||||
- Wingbits: flight enrichment disabled, heuristic-only flight classification remains.
|
||||
- AIS / OpenSky relay: live tracking features are disabled cleanly.
|
||||
- WorldMonitor API key: cloud fallback is blocked; desktop operates local-only.
|
||||
|
||||
+31
-17
@@ -7,7 +7,7 @@ AI-powered real-time global intelligence dashboard aggregating news, markets, ge
|
||||

|
||||

|
||||

|
||||

|
||||

|
||||
|
||||

|
||||
|
||||
@@ -3298,13 +3298,15 @@ const header = `World Monitor v${__APP_VERSION__}`;
|
||||
|
||||
## Installation
|
||||
|
||||
**Requirements:** Go 1.21+ and Node.js 18+.
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/koala73/worldmonitor.git
|
||||
cd worldmonitor
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
# Install everything (buf, sebuf plugins, npm deps, proto deps)
|
||||
make install
|
||||
|
||||
# Start development server
|
||||
npm run dev
|
||||
@@ -3313,6 +3315,14 @@ npm run dev
|
||||
npm run build
|
||||
```
|
||||
|
||||
If you modify any `.proto` files, regenerate before building or pushing:
|
||||
|
||||
```bash
|
||||
make generate # regenerate TypeScript clients, servers, and OpenAPI docs
|
||||
```
|
||||
|
||||
See [ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md) for the full proto workflow.
|
||||
|
||||
## API Dependencies
|
||||
|
||||
The dashboard fetches data from various public APIs and data sources:
|
||||
@@ -3642,7 +3652,7 @@ The system degrades gracefully—blocked sources are skipped while others contin
|
||||
|
||||
## Roadmap
|
||||
|
||||
See [ROADMAP.md](ROADMAP.md) for detailed planning. Recent intelligence enhancements:
|
||||
See [ROADMAP.md](../.planning/ROADMAP.md) for detailed planning. Recent intelligence enhancements:
|
||||
|
||||
### Completed
|
||||
|
||||
@@ -3698,7 +3708,7 @@ See [ROADMAP.md](ROADMAP.md) for detailed planning. Recent intelligence enhancem
|
||||
- **Additional Data Sources** - World Bank, IMF, OFAC sanctions, UNHCR refugee data, FAO food security
|
||||
- **Think Tank Feeds** - RUSI, Chatham House, ECFR, CFR, Wilson Center, CNAS, Arms Control Association
|
||||
|
||||
The full [ROADMAP.md](ROADMAP.md) documents implementation details, API endpoints, and 30+ free data sources for future integration.
|
||||
The full [ROADMAP.md](../.planning/ROADMAP.md) documents implementation details, API endpoints, and 30+ free data sources for future integration.
|
||||
|
||||
---
|
||||
|
||||
@@ -3994,20 +4004,24 @@ PRs that don't follow the code style or introduce security issues will be asked
|
||||
|
||||
### Development Tips
|
||||
|
||||
**Adding or Modifying API Endpoints**
|
||||
|
||||
All JSON API endpoints **must** use sebuf. Do not create standalone `api/*.js` files — the legacy pattern is deprecated.
|
||||
|
||||
See **[docs/ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md)** for the complete guide covering:
|
||||
|
||||
- Adding an RPC to an existing service
|
||||
- Adding an entirely new service
|
||||
- Proto conventions (validation, time fields, shared types)
|
||||
- Generated OpenAPI documentation
|
||||
|
||||
**Adding a New Data Layer**
|
||||
|
||||
1. Create service in `src/services/` for data fetching
|
||||
2. Add layer toggle in `src/components/Map.ts`
|
||||
3. Add rendering logic for map markers/overlays
|
||||
4. Add to help panel documentation
|
||||
5. Update README with layer description
|
||||
|
||||
**Adding a New API Proxy**
|
||||
|
||||
1. Create handler in `api/` directory
|
||||
2. Implement input validation (see existing proxies)
|
||||
3. Add appropriate cache headers
|
||||
4. Document any required environment variables
|
||||
1. Define the proto contract and generate code (see [ADDING_ENDPOINTS.md](ADDING_ENDPOINTS.md))
|
||||
2. Implement the handler in `server/worldmonitor/{domain}/v1/`
|
||||
3. Create the frontend service wrapper in `src/services/`
|
||||
4. Add layer toggle in `src/components/Map.ts`
|
||||
5. Add rendering logic for map markers/overlays
|
||||
|
||||
**Debugging**
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,937 @@
|
||||
# Architecture
|
||||
|
||||
World Monitor is an AI-powered real-time global intelligence dashboard built as a TypeScript single-page application. It aggregates 30+ external data sources — covering geopolitics, military activity, financial markets, cyber threats, climate events, and more — into a unified operational picture rendered through an interactive 3D globe and a grid of specialised panels.
|
||||
|
||||
This document covers the full system architecture: deployment topology, variant configuration, data pipelines, signal intelligence, map rendering, caching, desktop packaging, machine-learning inference, and error handling.
|
||||
|
||||
---
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [High-Level System Diagram](#1-high-level-system-diagram)
|
||||
2. [Variant Architecture](#2-variant-architecture)
|
||||
3. [Data Flow: RSS Ingestion to Display](#3-data-flow-rss-ingestion-to-display)
|
||||
4. [Signal Intelligence Pipeline](#4-signal-intelligence-pipeline)
|
||||
5. [Map Rendering Pipeline](#5-map-rendering-pipeline)
|
||||
6. [Caching Architecture](#6-caching-architecture)
|
||||
7. [Desktop Architecture](#7-desktop-architecture)
|
||||
8. [ML Pipeline](#8-ml-pipeline)
|
||||
9. [Error Handling Hierarchy](#9-error-handling-hierarchy)
|
||||
|
||||
---
|
||||
|
||||
## 1. High-Level System Diagram
|
||||
|
||||
The system follows a classic edge-compute pattern: a static SPA served from a CDN communicates with serverless API endpoints that proxy, normalise, and cache upstream data.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Browser
|
||||
SPA["TypeScript SPA<br/>(Vite 6, class-based)"]
|
||||
SW["Service Worker<br/>(Workbox)"]
|
||||
IDB["IndexedDB<br/>(snapshots & baselines)"]
|
||||
MLW["ML Web Worker<br/>(ONNX / Transformers.js)"]
|
||||
SPA --> SW
|
||||
SPA --> IDB
|
||||
SPA --> MLW
|
||||
end
|
||||
|
||||
subgraph Vercel["Vercel Edge Functions"]
|
||||
API["60+ API Endpoints<br/>(api/ directory, plain JS)"]
|
||||
end
|
||||
|
||||
subgraph External["External APIs (30+)"]
|
||||
RSS["RSS Feeds"]
|
||||
ACLED["ACLED"]
|
||||
UCDP["UCDP"]
|
||||
GDELT["GDELT"]
|
||||
OpenSky["OpenSky"]
|
||||
Finnhub["Finnhub"]
|
||||
Yahoo["Yahoo Finance"]
|
||||
FRED["FRED"]
|
||||
CoinGecko["CoinGecko"]
|
||||
Polymarket["Polymarket"]
|
||||
FIRMS["NASA FIRMS"]
|
||||
GROQ["Groq / OpenRouter"]
|
||||
Others["+ 20 more"]
|
||||
end
|
||||
|
||||
subgraph Cache["Upstash Redis"]
|
||||
Redis["Server-side<br/>API Response Cache"]
|
||||
end
|
||||
|
||||
subgraph Desktop["Tauri Desktop Shell"]
|
||||
Tauri["Tauri 2 (Rust)"]
|
||||
Sidecar["Node.js Sidecar<br/>127.0.0.1:46123"]
|
||||
Tauri --> Sidecar
|
||||
end
|
||||
|
||||
SPA <-->|"fetch()"| API
|
||||
SPA <-->|"Tauri IPC"| Tauri
|
||||
SPA <-->|"fetch()"| Sidecar
|
||||
API <--> Redis
|
||||
API <--> RSS
|
||||
API <--> ACLED
|
||||
API <--> UCDP
|
||||
API <--> GDELT
|
||||
API <--> OpenSky
|
||||
API <--> Finnhub
|
||||
API <--> Yahoo
|
||||
API <--> FRED
|
||||
API <--> CoinGecko
|
||||
API <--> Polymarket
|
||||
API <--> FIRMS
|
||||
API <--> GROQ
|
||||
API <--> Others
|
||||
```
|
||||
|
||||
### Component Summary
|
||||
|
||||
| Layer | Technology | Role |
|
||||
|---|---|---|
|
||||
| **SPA** | TypeScript, Vite 6, no framework | UI rendering via class-based components extending a `Panel` base class. 44 panels in the full variant. |
|
||||
| **Vercel Edge Functions** | Plain JS (60+ files in api/) | Proxy, normalise, and cache upstream API calls. Each file exports a default Vercel handler. |
|
||||
| **External APIs** | 30+ heterogeneous sources | RSS feeds, conflict databases (ACLED, UCDP), geospatial (GDELT, NASA FIRMS, OpenSky), markets (Finnhub, Yahoo Finance, CoinGecko), LLMs (Groq, OpenRouter), and more. |
|
||||
| **Upstash Redis** | Redis REST API | Server-side response cache with TTL-based expiry. Falls back to in-memory Map in sidecar mode. |
|
||||
| **Service Worker** | Workbox | Offline support, runtime caching strategies, background sync. |
|
||||
| **IndexedDB** | `worldmonitor_db` | Client-side storage for playback snapshots and temporal baseline data. |
|
||||
| **Tauri Shell** | Tauri 2 (Rust) + Node.js sidecar | Desktop packaging. Sidecar runs a local API server; Rust layer provides OS keychain, window management, and IPC. |
|
||||
| **ML Worker** | Web Worker + ONNX Runtime / Transformers.js | In-browser inference for embeddings, sentiment, summarisation, and NER. |
|
||||
|
||||
---
|
||||
|
||||
## 2. Variant Architecture
|
||||
|
||||
World Monitor ships as three product variants from a single codebase. Each variant surfaces a different subset of panels, map layers, and data sources.
|
||||
|
||||
| Variant | Domain | Focus |
|
||||
|---|---|---|
|
||||
| `full` | worldmonitor.app | Geopolitics, military, OSINT, conflicts, markets |
|
||||
| `tech` | tech.worldmonitor.app | AI/ML, startups, cybersecurity, developer tools |
|
||||
| `finance` | finance.worldmonitor.app | Markets, trading, central banks, macro indicators |
|
||||
|
||||
### Variant Resolution
|
||||
|
||||
The active variant is resolved at startup in src/config/variant.ts via a strict priority chain:
|
||||
|
||||
```
|
||||
localStorage('worldmonitor-variant') → import.meta.env.VITE_VARIANT → default 'full'
|
||||
```
|
||||
|
||||
The exported constant `SITE_VARIANT` is computed once as an IIFE:
|
||||
|
||||
```typescript
|
||||
export const SITE_VARIANT: string = (() => {
|
||||
if (typeof window !== 'undefined') {
|
||||
const stored = localStorage.getItem('worldmonitor-variant');
|
||||
if (stored === 'tech' || stored === 'full' || stored === 'finance') return stored;
|
||||
}
|
||||
return import.meta.env.VITE_VARIANT || 'full';
|
||||
})();
|
||||
```
|
||||
|
||||
The `localStorage` override enables runtime variant switching on the settings page without a rebuild. The `VITE_VARIANT` env var is set at deploy time (one Vercel project per subdomain).
|
||||
|
||||
### Configuration Tree-Shaking
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph ConfigTree["src/config/variants/"]
|
||||
Base["base.ts<br/>VariantConfig interface<br/>API_URLS, REFRESH_INTERVALS<br/>STORAGE_KEYS, MONITOR_COLORS"]
|
||||
Full["full.ts<br/>VARIANT_CONFIG"]
|
||||
Tech["tech.ts<br/>VARIANT_CONFIG"]
|
||||
Finance["finance.ts<br/>VARIANT_CONFIG"]
|
||||
Base --> Full
|
||||
Base --> Tech
|
||||
Base --> Finance
|
||||
end
|
||||
|
||||
subgraph Panels["src/config/panels.ts"]
|
||||
FP["FULL_PANELS (44)"]
|
||||
FM["FULL_MAP_LAYERS (35+)"]
|
||||
FMM["FULL_MOBILE_MAP_LAYERS"]
|
||||
TP["TECH_PANELS"]
|
||||
TM["TECH_MAP_LAYERS"]
|
||||
FiP["FINANCE_PANELS"]
|
||||
FiM["FINANCE_MAP_LAYERS"]
|
||||
end
|
||||
|
||||
Variant["SITE_VARIANT"] --> Switch{"Ternary switch"}
|
||||
Switch -->|"full"| FP
|
||||
Switch -->|"tech"| TP
|
||||
Switch -->|"finance"| FiP
|
||||
|
||||
Switch --> DefaultPanels["DEFAULT_PANELS"]
|
||||
Switch --> DefaultLayers["DEFAULT_MAP_LAYERS"]
|
||||
Switch --> MobileLayers["MOBILE_DEFAULT_MAP_LAYERS"]
|
||||
```
|
||||
|
||||
The `VariantConfig` interface in src/config/variants/base.ts defines the shape:
|
||||
|
||||
```typescript
|
||||
interface VariantConfig {
|
||||
name: string;
|
||||
description: string;
|
||||
panels: Record<string, PanelConfig>;
|
||||
mapLayers: MapLayers;
|
||||
mobileMapLayers: MapLayers;
|
||||
}
|
||||
```
|
||||
|
||||
Each variant file (full.ts, tech.ts, finance.ts) exports a `VARIANT_CONFIG` conforming to this interface. The shared base re-exports common constants: `API_URLS`, `REFRESH_INTERVALS`, `STORAGE_KEYS`, `MONITOR_COLORS`, `SECTORS`, `COMMODITIES`, `MARKET_SYMBOLS`, `UNDERSEA_CABLES`, and `AI_DATA_CENTERS`.
|
||||
|
||||
At build time, Vite's tree-shaking eliminates the unused variant configs. If `VITE_VARIANT=tech`, the full and finance panel definitions are dead-code-eliminated from the production bundle.
|
||||
|
||||
At runtime, src/config/panels.ts selects the active config via ternary expressions:
|
||||
|
||||
```typescript
|
||||
export const DEFAULT_PANELS = SITE_VARIANT === 'tech'
|
||||
? TECH_PANELS
|
||||
: SITE_VARIANT === 'finance'
|
||||
? FINANCE_PANELS
|
||||
: FULL_PANELS;
|
||||
```
|
||||
|
||||
The same pattern applies to `DEFAULT_MAP_LAYERS` and `MOBILE_DEFAULT_MAP_LAYERS`.
|
||||
|
||||
### Panel and Layer Counts
|
||||
|
||||
| Variant | Panels | Desktop Map Layers | Mobile Map Layers |
|
||||
|---|---|---|---|
|
||||
| `full` | 44 | 35+ | Reduced subset |
|
||||
| `tech` | ~20 | Tech-focused layers (cloud regions, startup hubs, accelerators) | Minimal |
|
||||
| `finance` | ~18 | Finance-focused layers (stock exchanges, financial centres, central banks) | Minimal |
|
||||
|
||||
The `MapLayers` interface contains 35+ boolean toggle keys including: `conflicts`, `bases`, `cables`, `pipelines`, `hotspots`, `ais`, `nuclear`, `irradiators`, `sanctions`, `weather`, `economic`, `waterways`, `outages`, `cyberThreats`, `datacenters`, `protests`, `flights`, `military`, `natural`, `spaceports`, `minerals`, `fires`, `ucdpEvents`, `displacement`, `climate`, `startupHubs`, `cloudRegions`, `accelerators`, `techHQs`, `techEvents`, `stockExchanges`, `financialCenters`, `centralBanks`, `commodityHubs`, and `gulfInvestments`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Data Flow: RSS Ingestion to Display
|
||||
|
||||
The core intelligence pipeline transforms raw RSS feeds into clustered, classified, and scored events displayed across panels. This pipeline runs entirely in the browser.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant RSS as RSS Sources
|
||||
participant Proxy as /api/rss-proxy
|
||||
participant Cache as Upstash Redis
|
||||
participant SPA as Browser SPA
|
||||
participant Cluster as clustering.ts
|
||||
participant ML as ML Worker
|
||||
participant Threat as threat-classifier.ts
|
||||
participant Entity as entity-extraction.ts
|
||||
participant Panel as Panel Components
|
||||
|
||||
SPA->>Proxy: fetch(feedUrl)
|
||||
Proxy->>Cache: getCachedJson(key)
|
||||
alt Cache hit
|
||||
Cache-->>Proxy: cached response
|
||||
else Cache miss
|
||||
Proxy->>RSS: GET feed XML/JSON
|
||||
RSS-->>Proxy: raw feed data
|
||||
Proxy->>Cache: setCachedJson(key, data, ttl)
|
||||
end
|
||||
Proxy-->>SPA: NewsItem[]
|
||||
|
||||
SPA->>Cluster: clusterNews(items)
|
||||
Note over Cluster: Jaccard similarity<br/>on title token sets
|
||||
|
||||
alt ML Worker available
|
||||
SPA->>Cluster: clusterNewsHybrid(items)
|
||||
Cluster->>ML: embed(clusterTexts)
|
||||
ML-->>Cluster: embeddings[][]
|
||||
Cluster->>Cluster: mergeSemanticallySimilarClusters()
|
||||
end
|
||||
|
||||
Cluster-->>SPA: ClusteredEvent[]
|
||||
SPA->>Threat: classifyCluster(event)
|
||||
Threat-->>SPA: ThreatClassification
|
||||
SPA->>Entity: extractEntitiesFromCluster(event)
|
||||
Entity-->>SPA: NewsEntityContext
|
||||
SPA->>Panel: render(scoredEvents)
|
||||
```
|
||||
|
||||
### Pipeline Stages
|
||||
|
||||
**Stage 1 — RSS Fetch** (src/services/rss.ts)
|
||||
|
||||
The `fetchFeed()` function calls the `/api/rss-proxy` endpoint, which fetches and parses upstream RSS/Atom feeds on the server side. Responses are cached in Upstash Redis (or the sidecar in-memory cache). On the client, a per-feed in-memory cache (`feedCache` Map) prevents redundant network requests within the refresh interval, and a persistent cache layer (via src/services/persistent-cache.ts) provides resilience across page reloads and desktop restarts.
|
||||
|
||||
The `fetchAllFeeds()` function orchestrates concurrent fetching across all enabled feeds with configurable `onBatch` callbacks for progressive rendering.
|
||||
|
||||
**Stage 2 — Clustering** (src/services/clustering.ts)
|
||||
|
||||
Two clustering strategies are available:
|
||||
|
||||
- `clusterNews(items)` — fast Jaccard similarity over title token sets via `clusterNewsCore()`. Groups headlines with high textual overlap into `ClusteredEvent[]`. This is the default path when ML is unavailable.
|
||||
- `clusterNewsHybrid(items)` — first runs Jaccard clustering, then refines results using semantic embeddings from the ML Worker. `mergeSemanticallySimilarClusters()` reduces fragmentation by joining clusters whose embedding centroids exceed the `semanticClusterThreshold` (default 0.75). Requires at least `minClustersForML` (5) initial clusters to activate.
|
||||
|
||||
**Stage 3 — Classification** (src/services/threat-classifier.ts)
|
||||
|
||||
Each clustered event receives a `ThreatClassification` with a `ThreatLevel` (`critical | high | medium | low | info`). The classifier uses keyword pattern matching and source-tier weighting. Threat levels map to CSS variables (`--threat-critical`, `--threat-high`, etc.) for consistent colour coding across panels.
|
||||
|
||||
**Stage 4 — Entity Extraction** (src/services/entity-extraction.ts + src/services/entity-index.ts)
|
||||
|
||||
The `extractEntitiesFromTitle()` function matches text against a pre-built entity index. The `extractEntitiesFromCluster()` function aggregates entities across all items in a cluster to produce a `NewsEntityContext` containing primary and related entities.
|
||||
|
||||
The entity index (src/services/entity-index.ts) is a multi-index structure with five `Map` lookups:
|
||||
|
||||
| Index | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `byId` | `Map<string, EntityEntry>` | Canonical lookup by entity ID |
|
||||
| `byAlias` | `Map<string, string>` | Alias-to-ID resolution (case-insensitive) |
|
||||
| `byKeyword` | `Map<string, Set<string>>` | Keyword-to-entity-IDs for text matching |
|
||||
| `bySector` | `Map<string, Set<string>>` | Sector-based grouping |
|
||||
| `byType` | `Map<string, Set<string>>` | Entity type grouping (person, org, country, etc.) |
|
||||
|
||||
**Stage 5 — Display**
|
||||
|
||||
Classified and entity-enriched events are distributed to panels. The `Panel` base class provides a consistent rendering contract. Each panel subclass (LiveNewsPanel, IntelligencePanel, etc.) decides how to filter, sort, and present events relevant to its domain.
|
||||
|
||||
---
|
||||
|
||||
## 4. Signal Intelligence Pipeline
|
||||
|
||||
The signal aggregator fuses heterogeneous geospatial data sources into a unified intelligence picture with country-level clustering and regional convergence detection.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Sources["Data Sources"]
|
||||
IO["Internet Outages"]
|
||||
MF["Military Flights<br/>(OpenSky)"]
|
||||
MV["Military Vessels<br/>(AIS)"]
|
||||
PR["Protests<br/>(ACLED)"]
|
||||
AD["AIS Disruptions"]
|
||||
SF["Satellite Fires<br/>(NASA FIRMS)"]
|
||||
TA["Temporal Anomalies<br/>(Baseline Deviations)"]
|
||||
end
|
||||
|
||||
subgraph Aggregator["SignalAggregator (src/services/signal-aggregator.ts)"]
|
||||
Extract["Signal Extraction<br/>normalise to GeoSignal"]
|
||||
Geo["Geo-Spatial Correlation<br/>country code lookup"]
|
||||
Country["Country Clustering<br/>CountrySignalCluster"]
|
||||
Regional["Regional Convergence<br/>REGION_DEFINITIONS (6 regions)"]
|
||||
Score["Convergence Scoring<br/>multi-signal co-occurrence"]
|
||||
Summary["SignalSummary<br/>AI context generation"]
|
||||
end
|
||||
|
||||
IO --> Extract
|
||||
MF --> Extract
|
||||
MV --> Extract
|
||||
PR --> Extract
|
||||
AD --> Extract
|
||||
SF --> Extract
|
||||
TA --> Extract
|
||||
|
||||
Extract --> Geo
|
||||
Geo --> Country
|
||||
Country --> Regional
|
||||
Regional --> Score
|
||||
Score --> Summary
|
||||
|
||||
Summary --> Insights["AI Insights Panel"]
|
||||
Summary --> MapVis["Map Visualisation"]
|
||||
Summary --> SignalModal["Signal Modal"]
|
||||
```
|
||||
|
||||
### Type Hierarchy
|
||||
|
||||
The pipeline defined in src/services/signal-aggregator.ts operates on a layered type system:
|
||||
|
||||
```
|
||||
SignalType (enum-like union)
|
||||
├── internet_outage
|
||||
├── military_flight
|
||||
├── military_vessel
|
||||
├── protest
|
||||
├── ais_disruption
|
||||
├── satellite_fire
|
||||
└── temporal_anomaly
|
||||
|
||||
GeoSignal (individual signal)
|
||||
├── type: SignalType
|
||||
├── country: string (ISO 3166-1 alpha-2)
|
||||
├── countryName: string
|
||||
├── lat / lon: number
|
||||
├── severity: 'low' | 'medium' | 'high'
|
||||
├── title: string
|
||||
└── timestamp: Date
|
||||
|
||||
CountrySignalCluster (per-country aggregation)
|
||||
├── country / countryName
|
||||
├── signals: GeoSignal[]
|
||||
├── signalTypes: Set<SignalType>
|
||||
├── totalCount / highSeverityCount
|
||||
└── convergenceScore: number
|
||||
|
||||
RegionalConvergence (cross-country pattern)
|
||||
├── region: string
|
||||
├── countries: string[]
|
||||
├── signalTypes: SignalType[]
|
||||
├── totalSignals: number
|
||||
└── description: string
|
||||
|
||||
SignalSummary (final output)
|
||||
├── timestamp: Date
|
||||
├── totalSignals: number
|
||||
├── byType: Record<SignalType, number>
|
||||
├── convergenceZones: RegionalConvergence[]
|
||||
├── topCountries: CountrySignalCluster[]
|
||||
└── aiContext: string
|
||||
```
|
||||
|
||||
### Region Definitions
|
||||
|
||||
The `REGION_DEFINITIONS` constant maps six monitored regions to their constituent country codes:
|
||||
|
||||
| Region | Name | Countries |
|
||||
|---|---|---|
|
||||
| `middle_east` | Middle East | IR, IL, SA, AE, IQ, SY, YE, JO, LB, KW, QA, OM, BH |
|
||||
| `east_asia` | East Asia | CN, TW, JP, KR, KP, HK, MN |
|
||||
| `south_asia` | South Asia | IN, PK, BD, AF, NP, LK, MM |
|
||||
| `europe_east` | Eastern Europe | UA, RU, BY, PL, RO, MD, HU, CZ, SK, BG |
|
||||
| `africa_north` | North Africa | EG, LY, DZ, TN, MA, SD, SS |
|
||||
| `africa_sahel` | Sahel Region | ML, NE, BF, TD, NG, CM, CF |
|
||||
|
||||
### Convergence Scoring
|
||||
|
||||
The `convergenceScore` on each `CountrySignalCluster` quantifies multi-signal co-occurrence. A high score indicates that multiple independent signal types are present in the same country within the 24-hour analysis window (`WINDOW_MS`). This score drives the AI Insights panel prioritisation and the signal modal display.
|
||||
|
||||
The `SignalAggregator` class maintains a rolling window of signals and a `WeakMap`-based source tracking for temporal anomaly provenance. Individual `ingest*()` methods (e.g., `ingestInternetOutages()`, `ingestMilitaryFlights()`) clear stale signals by type before inserting fresh data, ensuring the aggregation always reflects the latest state.
|
||||
|
||||
---
|
||||
|
||||
## 5. Map Rendering Pipeline
|
||||
|
||||
The map system combines a 2D vector tile base map (MapLibre GL JS) with a 3D WebGL overlay (deck.gl) for globe rendering, supporting 35+ toggleable data layers.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph MapStack["Map Rendering Stack"]
|
||||
Container["MapContainer.ts<br/>Layout & resize management"]
|
||||
BaseMap["Map.ts<br/>MapLibre GL JS<br/>Vector tiles, region controls"]
|
||||
DeckGL["DeckGLMap.ts<br/>deck.gl WebGL overlay<br/>3D globe & data layers"]
|
||||
Popup["MapPopup.ts<br/>Feature interaction"]
|
||||
end
|
||||
|
||||
subgraph LayerConfig["Layer Configuration"]
|
||||
Defaults["FULL_MAP_LAYERS<br/>(35+ boolean toggles)"]
|
||||
UserPref["localStorage overrides<br/>(worldmonitor-layers)"]
|
||||
URLState["URL state overrides"]
|
||||
Variant["Variant-specific defaults"]
|
||||
end
|
||||
|
||||
subgraph DataLayers["Data Layers (toggleable)"]
|
||||
Geo["Geopolitical:<br/>conflicts, bases, nuclear,<br/>sanctions, waterways"]
|
||||
Military["Military:<br/>flights, military, ais"]
|
||||
Infra["Infrastructure:<br/>cables, pipelines,<br/>datacenters, spaceports"]
|
||||
Environmental["Environmental:<br/>weather, fires, climate,<br/>natural, minerals"]
|
||||
Threat["Threat:<br/>outages, cyberThreats,<br/>protests, hotspots"]
|
||||
Data["Data Sources:<br/>ucdpEvents, displacement"]
|
||||
TechLayers["Tech:<br/>startupHubs, cloudRegions,<br/>accelerators, techHQs"]
|
||||
FinanceLayers["Finance:<br/>stockExchanges,<br/>financialCenters,<br/>centralBanks"]
|
||||
end
|
||||
|
||||
Defaults --> Merge["Layer Merge Logic"]
|
||||
UserPref --> Merge
|
||||
URLState --> Merge
|
||||
Variant --> Merge
|
||||
Merge --> ActiveLayers["Active MapLayers"]
|
||||
|
||||
ActiveLayers --> DeckGL
|
||||
Container --> BaseMap
|
||||
Container --> DeckGL
|
||||
BaseMap --> Popup
|
||||
DeckGL --> Popup
|
||||
|
||||
Geo --> DeckGL
|
||||
Military --> DeckGL
|
||||
Infra --> DeckGL
|
||||
Environmental --> DeckGL
|
||||
Threat --> DeckGL
|
||||
Data --> DeckGL
|
||||
TechLayers --> DeckGL
|
||||
FinanceLayers --> DeckGL
|
||||
```
|
||||
|
||||
### Layer Toggle Resolution
|
||||
|
||||
Map layers follow a three-tier override system:
|
||||
|
||||
1. **Variant defaults** — `FULL_MAP_LAYERS`, `TECH_MAP_LAYERS`, or `FINANCE_MAP_LAYERS` define the base layer state for each variant. The full variant enables `conflicts`, `bases`, `hotspots`, `nuclear`, `sanctions`, `weather`, `economic`, `waterways`, `outages`, and `military` by default.
|
||||
|
||||
2. **User localStorage** — Stored under the key `worldmonitor-layers`. Users toggle layers in the map controls UI, and their preferences persist across sessions.
|
||||
|
||||
3. **URL state** — Query parameters can override individual layers for shareable links and embeds.
|
||||
|
||||
The merge logic applies overrides in this order, meaning URL state has the highest priority.
|
||||
|
||||
### Mobile Adaptation
|
||||
|
||||
Mobile devices receive a reduced layer set via `MOBILE_DEFAULT_MAP_LAYERS` (variant-specific). This disables heavier layers (bases, nuclear, cables, pipelines, spaceports, minerals) that would degrade performance on constrained devices while retaining the most operationally relevant overlays (conflicts, hotspots, sanctions, weather).
|
||||
|
||||
### Rendering Pipeline
|
||||
|
||||
The rendering stack works in two layers:
|
||||
|
||||
- **MapLibre GL JS** (src/components/Map.ts) provides the base map with vector tiles, region-specific map controls, and the 2D rendering context. It handles camera management, style loading, and base interaction events.
|
||||
|
||||
- **deck.gl** (src/components/DeckGLMap.ts) overlays a WebGL context for 3D globe rendering and data-driven layers. Each toggleable layer maps to a deck.gl layer instance (ScatterplotLayer, IconLayer, ArcLayer, etc.) that is conditionally created based on the active `MapLayers` state.
|
||||
|
||||
The **MapPopup** component (src/components/MapPopup.ts) provides a unified popup system for feature interaction across both rendering layers, displaying contextual information when users click or hover over map features.
|
||||
|
||||
---
|
||||
|
||||
## 6. Caching Architecture
|
||||
|
||||
World Monitor employs a five-tier caching strategy to minimise API costs, reduce latency, and enable offline operation.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Tier1["Tier 1: Upstash Redis (Server)"]
|
||||
Redis["api/_upstash-cache.js<br/>getCachedJson() / setCachedJson()<br/>TTL-based expiry"]
|
||||
end
|
||||
|
||||
subgraph Tier1b["Tier 1b: Sidecar In-Memory Cache"]
|
||||
MemCache["In-memory Map<br/>+ disk persistence (api-cache.json)<br/>Max 5000 entries"]
|
||||
end
|
||||
|
||||
subgraph Tier2["Tier 2: Vercel CDN"]
|
||||
CDN["s-maxage headers<br/>stale-while-revalidate<br/>Edge caching"]
|
||||
end
|
||||
|
||||
subgraph Tier3["Tier 3: Service Worker"]
|
||||
Workbox["Workbox Runtime Caching<br/>Offline support<br/>Cache-first / network-first strategies"]
|
||||
end
|
||||
|
||||
subgraph Tier4["Tier 4: IndexedDB (Client)"]
|
||||
IDB["worldmonitor_db"]
|
||||
Baselines["baselines store<br/>(keyPath: 'key')"]
|
||||
Snapshots["snapshots store<br/>(keyPath: 'timestamp'<br/>index: 'by_time')"]
|
||||
IDB --> Baselines
|
||||
IDB --> Snapshots
|
||||
end
|
||||
|
||||
subgraph Tier5["Tier 5: Persistent Cache"]
|
||||
PC["persistent-cache.ts<br/>CacheEnvelope<T>"]
|
||||
TauriInvoke["Tauri invoke<br/>(OS filesystem)"]
|
||||
LSFallback["localStorage fallback<br/>prefix: worldmonitor-persistent-cache:"]
|
||||
PC --> TauriInvoke
|
||||
PC --> LSFallback
|
||||
end
|
||||
|
||||
Browser["Browser SPA"] --> Workbox
|
||||
Workbox --> CDN
|
||||
CDN --> Redis
|
||||
Redis --> ExternalAPI["External APIs"]
|
||||
|
||||
Browser --> IDB
|
||||
Browser --> PC
|
||||
|
||||
Sidecar["Desktop Sidecar"] --> MemCache
|
||||
MemCache --> ExternalAPI
|
||||
```
|
||||
|
||||
### Tier 1: Upstash Redis (Server-Side)
|
||||
|
||||
The api/_upstash-cache.js module wraps all API fetch operations with Redis GET/SET. Every API endpoint calls `getCachedJson(key)` before hitting upstream. On cache miss, the upstream response is stored with `setCachedJson(key, value, ttlSeconds)`. The module lazily initialises the Redis client from `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` environment variables.
|
||||
|
||||
A `hashString()` utility produces compact cache keys from request parameters using a DJB2 hash.
|
||||
|
||||
### Tier 1b: Sidecar In-Memory Cache
|
||||
|
||||
When running in desktop/sidecar mode (`LOCAL_API_MODE=sidecar`), Redis is bypassed entirely. An in-memory `Map` stores cache entries with expiry timestamps. Entries persist to disk as `api-cache.json` via debounced writes (2-second delay). A periodic cleanup interval (60 seconds) evicts expired entries. The maximum persisted entry count is capped at `MAX_PERSIST_ENTRIES` (default 5000).
|
||||
|
||||
The disk persistence uses atomic writes: data is written to a `.tmp` file first, then renamed to the final path. A `persistInFlight` flag with `persistQueued` prevents concurrent writes.
|
||||
|
||||
### Tier 2: Vercel CDN
|
||||
|
||||
API responses include `Cache-Control` headers with `s-maxage` and `stale-while-revalidate` directives. This enables Vercel's CDN edge nodes to serve cached responses without invoking the serverless function, reducing cold starts and upstream API calls.
|
||||
|
||||
### Tier 3: Service Worker (Workbox)
|
||||
|
||||
The Service Worker (configured via Workbox) provides runtime caching with strategy selection per route:
|
||||
|
||||
- **Cache-first** for static assets and infrequently changing data
|
||||
- **Network-first** for real-time feeds and market data
|
||||
- **Stale-while-revalidate** for semi-static resources
|
||||
|
||||
The offline fallback page (public/offline.html) is served when the network is unavailable and no cached response exists.
|
||||
|
||||
### Tier 4: IndexedDB
|
||||
|
||||
The `worldmonitor_db` IndexedDB database contains two object stores:
|
||||
|
||||
| Store | keyPath | Index | Purpose |
|
||||
|---|---|---|---|
|
||||
| `baselines` | `key` | — | Stores baseline values for temporal deviation tracking. The signal aggregator compares current values against baselines to detect anomalies. |
|
||||
| `snapshots` | `timestamp` | `by_time` | Stores periodic system state snapshots for the playback control feature, enabling users to replay historical states. |
|
||||
|
||||
### Tier 5: Persistent Cache
|
||||
|
||||
The src/services/persistent-cache.ts module provides a cross-platform persistent storage abstraction. Data is wrapped in a `CacheEnvelope<T>`:
|
||||
|
||||
```typescript
|
||||
type CacheEnvelope<T> = {
|
||||
key: string;
|
||||
updatedAt: number;
|
||||
data: T;
|
||||
};
|
||||
```
|
||||
|
||||
On desktop, `getPersistentCache()` and `setPersistentCache()` attempt Tauri IPC invocations (`read_cache_entry` / `write_cache_entry`) first, which store data on the OS filesystem via the Rust backend. If the Tauri call fails (or in web mode), the module falls back to `localStorage` with the prefix `worldmonitor-persistent-cache:`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Desktop Architecture
|
||||
|
||||
The desktop application uses Tauri 2 (Rust) as a native shell around the web SPA, with a Node.js sidecar process providing a local API server.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph TauriApp["Tauri 2 Desktop Application"]
|
||||
subgraph Rust["Rust Backend (src-tauri/)"]
|
||||
TauriCore["tauri.conf.json<br/>(+ variant overrides)"]
|
||||
BuildRS["build.rs"]
|
||||
Cargo["Cargo.toml"]
|
||||
Commands["IPC Commands<br/>(read_cache_entry,<br/>write_cache_entry, etc.)"]
|
||||
Keychain["OS Keychain<br/>(18 RuntimeSecretKeys)"]
|
||||
end
|
||||
|
||||
subgraph SidecarProc["Node.js Sidecar"]
|
||||
LocalAPI["Local API Server<br/>http://127.0.0.1:46123"]
|
||||
MemCache["In-memory Map<br/>+ api-cache.json"]
|
||||
LocalAPI --> MemCache
|
||||
end
|
||||
|
||||
subgraph WebView["WebView (SPA)"]
|
||||
Runtime["runtime.ts<br/>detectDesktopRuntime()"]
|
||||
Bridge["tauri-bridge.ts<br/>Typed IPC wrapper"]
|
||||
Config["runtime-config.ts<br/>Feature toggles & secrets"]
|
||||
PCache["persistent-cache.ts"]
|
||||
end
|
||||
end
|
||||
|
||||
Runtime -->|"isDesktopRuntime()"| Bridge
|
||||
Bridge -->|"invokeTauri()"| Commands
|
||||
Config -->|"readSecret()"| Keychain
|
||||
PCache -->|"read/write_cache_entry"| Commands
|
||||
WebView -->|"fetch() via patch"| LocalAPI
|
||||
```
|
||||
|
||||
### Runtime Detection
|
||||
|
||||
The src/services/runtime.ts module detects the desktop environment through multiple signals:
|
||||
|
||||
```typescript
|
||||
function detectDesktopRuntime(probe: RuntimeProbe): boolean {
|
||||
// Checks: window.__TAURI__, user agent, location host (127.0.0.1)
|
||||
}
|
||||
```
|
||||
|
||||
When desktop mode is detected, `getApiBaseUrl()` returns `http://127.0.0.1:46123` instead of relative paths, routing all API calls through the local sidecar. A global `fetch()` monkey-patch (applied once via `__wmFetchPatched` guard) rewrites API URLs to point at the sidecar.
|
||||
|
||||
### Tauri Configuration
|
||||
|
||||
The src-tauri/ directory contains:
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| tauri.conf.json | Base Tauri configuration (window size, CSP, bundle settings) |
|
||||
| tauri.tech.conf.json | Tech variant overrides (app name, window title, icons) |
|
||||
| tauri.finance.conf.json | Finance variant overrides |
|
||||
| build.rs | Rust build script for Tauri codegen |
|
||||
| Cargo.toml | Rust dependencies |
|
||||
| sidecar/ | Node.js sidecar source (local API server) |
|
||||
| capabilities/ | Tauri capability definitions (permissions) |
|
||||
| icons/ | Application icons for each platform |
|
||||
|
||||
### Tauri Bridge
|
||||
|
||||
The src/services/tauri-bridge.ts module provides a typed TypeScript wrapper around Tauri's IPC invoke mechanism. It exposes functions like `invokeTauri<T>(command, args)` that handle serialisation and error mapping.
|
||||
|
||||
### Runtime Configuration
|
||||
|
||||
The src/services/runtime-config.ts module manages two concerns:
|
||||
|
||||
**1. Runtime Secrets** — 18 `RuntimeSecretKey` values representing API keys and credentials:
|
||||
|
||||
`GROQ_API_KEY`, `OPENROUTER_API_KEY`, `FRED_API_KEY`, `EIA_API_KEY`, `CLOUDFLARE_API_TOKEN`, `ACLED_ACCESS_TOKEN`, `URLHAUS_AUTH_KEY`, `OTX_API_KEY`, `ABUSEIPDB_API_KEY`, `WINGBITS_API_KEY`, `WS_RELAY_URL`, `VITE_OPENSKY_RELAY_URL`, `OPENSKY_CLIENT_ID`, `OPENSKY_CLIENT_SECRET`, `AISSTREAM_API_KEY`, `FINNHUB_API_KEY`, `NASA_FIRMS_API_KEY`, `UC_DP_KEY`.
|
||||
|
||||
On desktop, secrets are read from the OS keychain via Tauri IPC. In web mode, they fall back to environment variables. A `validateSecret()` function provides format validation with user-facing hints.
|
||||
|
||||
**2. Feature Toggles** — 14 `RuntimeFeatureId` values stored in localStorage under the key `worldmonitor-runtime-feature-toggles`:
|
||||
|
||||
`aiGroq`, `aiOpenRouter`, `economicFred`, `energyEia`, `internetOutages`, `acledConflicts`, `abuseChThreatIntel`, `alienvaultOtxThreatIntel`, `abuseIpdbThreatIntel`, `wingbitsEnrichment`, `aisRelay`, `openskyRelay`, `finnhubMarkets`, `nasaFirms`.
|
||||
|
||||
Each `RuntimeFeatureDefinition` declares its required secrets (and optionally desktop-specific overrides via `desktopRequiredSecrets`), along with a `fallback` description explaining behaviour when the feature is unavailable. The `isFeatureAvailable()` function checks both the toggle state and secret availability.
|
||||
|
||||
The settings page listens for `storage` events on the toggles key, enabling cross-tab synchronisation.
|
||||
|
||||
---
|
||||
|
||||
## 8. ML Pipeline
|
||||
|
||||
World Monitor runs machine-learning inference directly in the browser using ONNX Runtime Web via Transformers.js, with API-based fallbacks for constrained devices.
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Capabilities["Capability Detection"]
|
||||
Detect["ml-capabilities.ts<br/>detectMLCapabilities()"]
|
||||
WebGPU["WebGPU check"]
|
||||
WebGL["WebGL check"]
|
||||
SIMD["SIMD check"]
|
||||
Threads["SharedArrayBuffer check"]
|
||||
Memory["Device memory estimation"]
|
||||
Detect --> WebGPU
|
||||
Detect --> WebGL
|
||||
Detect --> SIMD
|
||||
Detect --> Threads
|
||||
Detect --> Memory
|
||||
end
|
||||
|
||||
subgraph Config["Model Configuration (ml-config.ts)"]
|
||||
Models["MODEL_CONFIGS"]
|
||||
Embed["embeddings<br/>all-MiniLM-L6-v2<br/>23 MB"]
|
||||
Sentiment["sentiment<br/>DistilBERT-SST2<br/>65 MB"]
|
||||
Summarize["summarization<br/>Flan-T5-base<br/>250 MB"]
|
||||
SumSmall["summarization-beta<br/>Flan-T5-small<br/>60 MB"]
|
||||
NER["ner<br/>BERT-NER<br/>65 MB"]
|
||||
Models --> Embed
|
||||
Models --> Sentiment
|
||||
Models --> Summarize
|
||||
Models --> SumSmall
|
||||
Models --> NER
|
||||
end
|
||||
|
||||
subgraph WorkerPipeline["ML Worker Pipeline"]
|
||||
Manager["MLWorkerManager<br/>(ml-worker.ts)"]
|
||||
Worker["ml.worker.ts<br/>(Web Worker)"]
|
||||
ONNX["ONNX Runtime Web<br/>(@xenova/transformers)"]
|
||||
Manager -->|"postMessage"| Worker
|
||||
Worker --> ONNX
|
||||
end
|
||||
|
||||
subgraph Fallback["Fallback Chain"]
|
||||
Groq["Groq API<br/>(cloud LLM)"]
|
||||
OpenRouter["OpenRouter API<br/>(cloud LLM)"]
|
||||
BrowserML["Browser Transformers.js<br/>(offline capable)"]
|
||||
Groq -->|"unavailable"| OpenRouter
|
||||
OpenRouter -->|"unavailable"| BrowserML
|
||||
end
|
||||
|
||||
subgraph Results["Worker Message Types"]
|
||||
EmbedR["embed-result"]
|
||||
SumR["summarize-result"]
|
||||
SentR["sentiment-result"]
|
||||
EntR["entities-result"]
|
||||
ClusterR["cluster-semantic-result"]
|
||||
end
|
||||
|
||||
Detect -->|"isSupported"| Manager
|
||||
Config --> Worker
|
||||
Manager --> Results
|
||||
```
|
||||
|
||||
### Capability Detection
|
||||
|
||||
The src/services/ml-capabilities.ts module probes the browser environment before loading any models:
|
||||
|
||||
```typescript
|
||||
interface MLCapabilities {
|
||||
isSupported: boolean;
|
||||
isDesktop: boolean;
|
||||
hasWebGL: boolean;
|
||||
hasWebGPU: boolean;
|
||||
hasSIMD: boolean;
|
||||
hasThreads: boolean;
|
||||
estimatedMemoryMB: number;
|
||||
recommendedExecutionProvider: 'webgpu' | 'webgl' | 'wasm';
|
||||
recommendedThreads: number;
|
||||
}
|
||||
```
|
||||
|
||||
ML is only enabled on desktop-class devices (`!isMobileDevice()`) with at least WebGL support and an estimated 100+ MB of available memory. The `recommendedExecutionProvider` selects the optimal ONNX backend: WebGPU (fastest, if available), WebGL, or WASM fallback.
|
||||
|
||||
### Model Configuration
|
||||
|
||||
The src/config/ml-config.ts module defines five model configurations:
|
||||
|
||||
| Model ID | HuggingFace Model | Size | Task | Required |
|
||||
|---|---|---|---|---|
|
||||
| `embeddings` | Xenova/all-MiniLM-L6-v2 | 23 MB | feature-extraction | Yes |
|
||||
| `sentiment` | Xenova/distilbert-base-uncased-finetuned-sst-2-english | 65 MB | text-classification | No |
|
||||
| `summarization` | Xenova/flan-t5-base | 250 MB | text2text-generation | No |
|
||||
| `summarization-beta` | Xenova/flan-t5-small | 60 MB | text2text-generation | No |
|
||||
| `ner` | Xenova/bert-base-NER | 65 MB | token-classification | No |
|
||||
|
||||
Only the embeddings model is marked as `required` — it powers semantic clustering. Other models are loaded on-demand based on feature flags (`ML_FEATURE_FLAGS`) and available memory budget (`ML_THRESHOLDS.memoryBudgetMB`, default 200 MB).
|
||||
|
||||
### ML Thresholds
|
||||
|
||||
```typescript
|
||||
const ML_THRESHOLDS = {
|
||||
semanticClusterThreshold: 0.75, // cosine similarity for merging clusters
|
||||
minClustersForML: 5, // minimum clusters before ML refinement
|
||||
maxTextsPerBatch: 20, // batch size for embedding requests
|
||||
modelLoadTimeoutMs: 600_000, // 10 min model download/compile timeout
|
||||
inferenceTimeoutMs: 120_000, // 2 min per inference call
|
||||
memoryBudgetMB: 200, // max memory for all loaded models
|
||||
};
|
||||
```
|
||||
|
||||
### Worker Architecture
|
||||
|
||||
The `MLWorkerManager` class (src/services/ml-worker.ts) manages the lifecycle of a dedicated Web Worker (src/workers/ml.worker.ts). Communication uses a request-response pattern over `postMessage`:
|
||||
|
||||
1. **Initialisation** — `init()` calls `detectMLCapabilities()`, creates the worker if supported, and waits for a `worker-ready` message (10-second timeout).
|
||||
|
||||
2. **Request dispatch** — Each method (`embed()`, `summarize()`, `sentiment()`, `entities()`, `clusterSemantic()`) generates a unique request ID, posts a message to the worker, and returns a `Promise` that resolves when the worker posts back a matching result message.
|
||||
|
||||
3. **Timeout handling** — Each pending request has an independent timeout. If the worker fails to respond within `inferenceTimeoutMs`, the promise rejects and the request is cleaned up.
|
||||
|
||||
4. **Model lifecycle** — Models are loaded lazily on first use. The worker emits `model-progress` events during download, enabling progress UI. `model-loaded` and `model-unloaded` events track the loaded model set.
|
||||
|
||||
### Worker Result Message Types
|
||||
|
||||
| Message Type | Payload | Used By |
|
||||
|---|---|---|
|
||||
| `embed-result` | `embeddings: number[][]` | Semantic clustering |
|
||||
| `summarize-result` | `summaries: string[]` | AI Insights panel |
|
||||
| `sentiment-result` | `results: SentimentResult[]` | Threat classification augmentation |
|
||||
| `entities-result` | `entities: NEREntity[][]` | Entity extraction (ML-backed) |
|
||||
| `cluster-semantic-result` | `clusters: number[][]` | Cluster merging |
|
||||
|
||||
### Fallback Chain
|
||||
|
||||
When browser-based ML is not available (mobile devices, constrained hardware, or feature disabled), the system falls back to cloud-based LLM APIs:
|
||||
|
||||
1. **Groq API** — Primary cloud fallback. Used for summarisation and classification via /api/groq-summarize.
|
||||
2. **OpenRouter API** — Secondary cloud fallback via /api/openrouter-summarize.
|
||||
3. **Browser Transformers.js** — Tertiary fallback for offline operation. Even without API access, the embeddings model enables basic semantic clustering.
|
||||
|
||||
The fallback is not automatic at the ML worker level; each consumer service chooses its preferred provider and handles degradation independently.
|
||||
|
||||
---
|
||||
|
||||
## 9. Error Handling Hierarchy
|
||||
|
||||
World Monitor uses a circuit-breaker pattern to manage transient failures across its many data sources, preventing cascade failures and providing graceful degradation.
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> Closed: Initial state
|
||||
|
||||
Closed --> Closed: fetch() success → recordSuccess()
|
||||
Closed --> HalfOpen: fetch() failure<br/>(failures < MAX_FAILURES)
|
||||
HalfOpen --> Open: fetch() failure<br/>(failures >= MAX_FAILURES)
|
||||
Open --> Recovery: COOLDOWN_MS elapsed
|
||||
Recovery --> Closed: retry success → reset
|
||||
Recovery --> Open: retry failure → extend cooldown
|
||||
|
||||
state Closed {
|
||||
[*] --> Live
|
||||
Live: mode = 'live'
|
||||
Live: Serve fresh data
|
||||
}
|
||||
|
||||
state HalfOpen {
|
||||
[*] --> Degraded
|
||||
Degraded: failures > 0
|
||||
Degraded: Still attempting fetches
|
||||
}
|
||||
|
||||
state Open {
|
||||
[*] --> CircuitOpen
|
||||
CircuitOpen: mode = 'cached' or 'unavailable'
|
||||
CircuitOpen: Serve cached data if available
|
||||
CircuitOpen: Skip fetch until cooldown expires
|
||||
}
|
||||
|
||||
state Recovery {
|
||||
[*] --> Retry
|
||||
Retry: Single probe request
|
||||
Retry: On success → reset to Closed
|
||||
}
|
||||
```
|
||||
|
||||
### Circuit Breaker Implementation
|
||||
|
||||
The `CircuitBreaker<T>` class in src/utils/circuit-breaker.ts implements per-feed failure tracking with automatic cooldowns:
|
||||
|
||||
```typescript
|
||||
interface CircuitState {
|
||||
failures: number;
|
||||
cooldownUntil: number;
|
||||
lastError?: string;
|
||||
}
|
||||
|
||||
type BreakerDataMode = 'live' | 'cached' | 'unavailable';
|
||||
```
|
||||
|
||||
**Constants:**
|
||||
|
||||
| Constant | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `DEFAULT_MAX_FAILURES` | 2 | Consecutive failures before opening the circuit |
|
||||
| `DEFAULT_COOLDOWN_MS` | 5 min (300,000 ms) | How long to wait before retrying |
|
||||
| `DEFAULT_CACHE_TTL_MS` | 10 min (600,000 ms) | How long cached data remains valid |
|
||||
|
||||
### Lifecycle
|
||||
|
||||
1. **Closed (Live)** — Normal operation. Each successful `fetch()` calls `recordSuccess()`, resetting the failure counter.
|
||||
|
||||
2. **Failure Tracking** — On fetch failure, the failure counter increments. The `lastError` is recorded for diagnostics.
|
||||
|
||||
3. **Open (Circuit Tripped)** — When `failures >= maxFailures`, the circuit opens. `cooldownUntil` is set to `Date.now() + cooldownMs`. While open:
|
||||
- `isOnCooldown()` returns `true`
|
||||
- No fetch attempts are made
|
||||
- `getCached()` serves the last successful response if within `cacheTtlMs`
|
||||
- If no cached data exists, the data mode is `'unavailable'`
|
||||
|
||||
4. **Recovery (Cooldown Expired)** — After the cooldown period, `isOnCooldown()` returns `false` and resets the state. The next fetch attempt acts as a probe:
|
||||
- On success → circuit fully resets to closed
|
||||
- On failure → circuit re-opens with a fresh cooldown
|
||||
|
||||
### Data State Reporting
|
||||
|
||||
Each breaker tracks a `BreakerDataState` for UI display:
|
||||
|
||||
```typescript
|
||||
interface BreakerDataState {
|
||||
mode: BreakerDataMode; // 'live' | 'cached' | 'unavailable'
|
||||
timestamp: number | null;
|
||||
offline: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
Panels use this state to display freshness indicators — e.g., showing a "cached" badge with the last successful timestamp, or an "unavailable" state with the `lastError` message.
|
||||
|
||||
### Desktop Offline Mode
|
||||
|
||||
The `isDesktopOfflineMode()` helper detects when the Tauri desktop app loses network connectivity (`navigator.onLine === false`). In this mode, all circuit breakers immediately fall back to cached data without attempting network requests, preserving the user experience during temporary disconnections.
|
||||
|
||||
### Global Breaker Registry
|
||||
|
||||
A module-level `Map<string, CircuitBreaker<unknown>>` maintains all active breakers. Utility functions provide system-wide observability:
|
||||
|
||||
| Function | Purpose |
|
||||
|---|---|
|
||||
| `createCircuitBreaker<T>(options)` | Create and register a new breaker |
|
||||
| `getCircuitBreakerStatus()` | Returns status of all breakers (for diagnostics) |
|
||||
| `isCircuitBreakerOnCooldown(name)` | Check if a specific breaker is in cooldown |
|
||||
| `getCircuitBreakerCooldownInfo(name)` | Get cooldown state and remaining seconds |
|
||||
| `removeCircuitBreaker(name)` | Deregister a breaker |
|
||||
|
||||
### Degradation Hierarchy
|
||||
|
||||
The overall error handling follows a predictable degradation path:
|
||||
|
||||
```
|
||||
Live data (fresh fetch)
|
||||
└── on failure → Stale cache (within cacheTtlMs)
|
||||
└── expired cache → 'unavailable' state in UI
|
||||
└── desktop offline → immediate cache fallback
|
||||
```
|
||||
|
||||
Each panel independently manages its breaker, so a failure in one data source (e.g., OpenSky API downtime) does not affect other panels. The AI Insights panel aggregates breaker states to provide a system-wide health summary.
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user