hanzo/dataroom
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: add skills

Browse Source
This commit is contained in:
Marc Seitz
2026-02-20 00:10:28 +11:00
parent 07f56a1b51
commit 9e0cc88a4a
26 changed files with 1360 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/frontend-design/LICENSE.txt
+177
View File
@@ -0,0 +1,177 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
.agents/skills/frontend-design/SKILL.md
+42
View File
@@ -0,0 +1,42 @@
---
name: frontend-design
description: Create distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, artifacts, posters, or applications (examples include websites, landing pages, dashboards, React components, HTML/CSS layouts, or when styling/beautifying any web UI). Generates creative, polished code and UI design that avoids generic AI aesthetics.
license: Complete terms in LICENSE.txt
---
This skill guides creation of distinctive, production-grade frontend interfaces that avoid generic "AI slop" aesthetics. Implement real working code with exceptional attention to aesthetic details and creative choices.
The user provides frontend requirements: a component, page, application, or interface to build. They may include context about the purpose, audience, or technical constraints.
## Design Thinking
Before coding, understand the context and commit to a BOLD aesthetic direction:
- **Purpose**: What problem does this interface solve? Who uses it?
- **Tone**: Pick an extreme: brutally minimal, maximalist chaos, retro-futuristic, organic/natural, luxury/refined, playful/toy-like, editorial/magazine, brutalist/raw, art deco/geometric, soft/pastel, industrial/utilitarian, etc. There are so many flavors to choose from. Use these for inspiration but design one that is true to the aesthetic direction.
- **Constraints**: Technical requirements (framework, performance, accessibility).
- **Differentiation**: What makes this UNFORGETTABLE? What's the one thing someone will remember?
**CRITICAL**: Choose a clear conceptual direction and execute it with precision. Bold maximalism and refined minimalism both work - the key is intentionality, not intensity.
Then implement working code (HTML/CSS/JS, React, Vue, etc.) that is:
- Production-grade and functional
- Visually striking and memorable
- Cohesive with a clear aesthetic point-of-view
- Meticulously refined in every detail
## Frontend Aesthetics Guidelines
Focus on:
- **Typography**: Choose fonts that are beautiful, unique, and interesting. Avoid generic fonts like Arial and Inter; opt instead for distinctive choices that elevate the frontend's aesthetics; unexpected, characterful font choices. Pair a distinctive display font with a refined body font.
- **Color & Theme**: Commit to a cohesive aesthetic. Use CSS variables for consistency. Dominant colors with sharp accents outperform timid, evenly-distributed palettes.
- **Motion**: Use animations for effects and micro-interactions. Prioritize CSS-only solutions for HTML. Use Motion library for React when available. Focus on high-impact moments: one well-orchestrated page load with staggered reveals (animation-delay) creates more delight than scattered micro-interactions. Use scroll-triggering and hover states that surprise.
- **Spatial Composition**: Unexpected layouts. Asymmetry. Overlap. Diagonal flow. Grid-breaking elements. Generous negative space OR controlled density.
- **Backgrounds & Visual Details**: Create atmosphere and depth rather than defaulting to solid colors. Add contextual effects and textures that match the overall aesthetic. Apply creative forms like gradient meshes, noise textures, geometric patterns, layered transparencies, dramatic shadows, decorative borders, custom cursors, and grain overlays.
NEVER use generic AI-generated aesthetics like overused font families (Inter, Roboto, Arial, system fonts), cliched color schemes (particularly purple gradients on white backgrounds), predictable layouts and component patterns, and cookie-cutter design that lacks context-specific character.
Interpret creatively and make unexpected choices that feel genuinely designed for the context. No design should be the same. Vary between light and dark themes, different fonts, different aesthetics. NEVER converge on common choices (Space Grotesk, for example) across generations.
**IMPORTANT**: Match implementation complexity to the aesthetic vision. Maximalist designs need elaborate code with extensive animations and effects. Minimalist or refined designs need restraint, precision, and careful attention to spacing, typography, and subtle details. Elegance comes from executing the vision well.
Remember: Claude is capable of extraordinary creative work. Don't hold back, show what can truly be created when thinking outside the box and committing fully to a distinctive vision.
.agents/skills/postgres/SKILL.md
+46
View File
@@ -0,0 +1,46 @@
---
name: postgres
description: PostgreSQL best practices, query optimization, connection troubleshooting, and performance improvement. Load when working with Postgres databases.
license: MIT
metadata:
author: planetscale
version: "1.0.0"
---
# PlanetScale Postgres
## Generic Postgres
| Topic | Reference | Use for |
| ---------------------- | ---------------------------------------------------------------- | --------------------------------------------------------- |
| Schema Design | [references/schema-design.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/schema-design.md) | Tables, primary keys, data types, foreign keys |
| Indexing | [references/indexing.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/indexing.md) | Index types, composite indexes, performance |
| Index Optimization | [references/index-optimization.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/index-optimization.md) | Unused/duplicate index queries, index audit |
| Partitioning | [references/partitioning.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/partitioning.md) | Large tables, time-series, data retention |
| Query Patterns | [references/query-patterns.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/query-patterns.md) | SQL anti-patterns, JOINs, pagination, batch queries |
| Optimization Checklist | [references/optimization-checklist.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/optimization-checklist.md) | Pre-optimization audit, cleanup, readiness checks |
| MVCC and VACUUM | [references/mvcc-vacuum.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/mvcc-vacuum.md) | Dead tuples, long transactions, xid wraparound prevention |
## Operations and Architecture
| Topic | Reference | Use for |
| ---------------------- | ---------------------------------------------------------------------------- | --------------------------------------------------------------- |
| Process Architecture | [references/process-architecture.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/process-architecture.md) | Multi-process model, connection pooling, auxiliary processes |
| Memory Architecture | [references/memory-management-ops.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/memory-management-ops.md) | Shared/private memory layout, OS page cache, OOM prevention |
| MVCC Transactions | [references/mvcc-transactions.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/mvcc-transactions.md) | Isolation levels, XID wraparound, serialization errors |
| WAL and Checkpoints | [references/wal-operations.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/wal-operations.md) | WAL internals, checkpoint tuning, durability, crash recovery |
| Replication | [references/replication.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/replication.md) | Streaming replication, slots, sync commit, failover |
| Storage Layout | [references/storage-layout.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/storage-layout.md) | PGDATA structure, TOAST, fillfactor, tablespaces, disk mgmt |
| Monitoring | [references/monitoring.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/monitoring.md) | pg_stat views, logging, pg_stat_statements, host metrics |
| Backup and Recovery | [references/backup-recovery.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/backup-recovery.md) | pg_dump, pg_basebackup, PITR, WAL archiving, backup tools |
## PlanetScale-Specific
| Topic | Reference | Use for |
| ------------------ | ---------------------------------------------------------------------------- | ----------------------------------------------------- |
| Connection Pooling | [references/ps-connection-pooling.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/ps-connection-pooling.md) | PgBouncer, pool sizing, pooled vs direct |
| Extensions | [references/ps-extensions.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/ps-extensions.md) | Supported extensions, compatibility |
| Connections | [references/ps-connections.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/ps-connections.md) | Connection troubleshooting, drivers, SSL |
| Insights | [references/ps-insights.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/ps-insights.md) | Slow queries, MCP server, pscale CLI |
| CLI Commands | [references/ps-cli-commands.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/ps-cli-commands.md) | pscale CLI reference, branches, deploy requests, auth |
| CLI API Insights | [references/ps-cli-api-insights.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/ps-cli-api-insights.md) | Query insights via `pscale api`, schema analysis |
.agents/skills/postgres/references/backup-recovery.md
+41
View File
@@ -0,0 +1,41 @@
---
title: Backup and Recovery
description: Logical/physical backups, PITR, WAL archiving, backup tools, and recovery strategies
tags: postgres, backup, recovery, pitr, pg_dump, pg_basebackup, wal-archiving, operations
---
# Backup and Recovery
**FUNDAMENTAL RULE: Backups are useless until you've successfully tested recovery.**
## Logical Backups (pg_dump)
Exports as SQL or custom format; portable across PG versions and architectures. Formats: `-Fp` (plain SQL), `-Fc` (custom compressed, selective restore), `-Fd` (directory, parallel with `-j`), `-Ft` (tar, avoid). Use `-Fd -j 4` for large DBs. Restore: `pg_restore -d dbname file.dump`; add `-j` for parallel restore. Selective table restore: `pg_restore -t tablename`. Slow for large DBs; RPO = backup frequency (typically 24h).
## Physical Backups (pg_basebackup)
Copies raw PGDATA; same major version and platform required; cross-architecture works if same endianness (e.g., x86_64 ↔ ARM64). Faster for large clusters; includes all databases. Flags: `-Ft -z -P` for compressed tar with progress. Manual alternative: `pg_backup_start()` → copy PGDATA → `pg_backup_stop()` (complex; must write returned `backup_label`).
## PITR (Point-in-Time Recovery)
Requires base backup + continuous WAL archiving. Restores to any timestamp, transaction, or named restore point. Without PITR: restore only to backup time (potentially lose hours). With PITR: RPO = minutes. `archive_command` must return 0 ONLY when file is safely stored—premature 0 = data loss risk. `wal_level` must be `replica` or `logical` (not `minimal`).
## WAL Archiving
`archive_mode=on`, `archive_command='test ! -f /archive/%f && cp %p /archive/%f'`. **Test archive command as postgres user** (not root) since permission issues are common. Monitor `pg_stat_archiver` for `failed_count`, `last_archived_time`. Archive failures prevent WAL recycling → disk fills.
## Tool Comparison
| Tool | Use case |
|------|----------|
| pg_dump | Small DBs, migrations, selective restore |
| pg_basebackup | Basic PITR, built-in |
| pgBackRest | Production—parallel, incremental, S3/GCS/Azure, retention |
| Barman | Enterprise PITR, retention policies |
| WAL-G | Cloud-native, S3/GCS/Azure |
## RPO/RTO
Logical only: RPO = backup interval (hours); RTO = hours. PITR: RPO = minutes; RTO = hours. Synchronous replication: RPO = 0; RTO = seconds to minutes (failover).
## Operational Rules
- Verify integrity with `pg_verifybackup` (PG 13+)
- Test recovery / PITR regularly
- Take backups from standby to avoid impacting primary
- Retention: 7 daily, 4 weekly, 12 monthly
- Monitor archive growth and backup age
- **Never assume backups work without testing**
.agents/skills/postgres/references/index-optimization.md
+69
View File
@@ -0,0 +1,69 @@
---
title: Index Optimization Queries
description: Index audit queries
tags: postgres, indexes, unused-indexes, duplicate-indexes, optimization
---
# Index Optimization
## Identify Unused Indexes
Query to find unused indexes:
```sql
-- indexes with 0 scans (check pg_stat_reset / pg_postmaster_start_time first)
SELECT
s.schemaname,
s.relname AS table_name,
s.indexrelname AS index_name,
pg_size_pretty(pg_relation_size(s.indexrelid)) AS index_size
FROM pg_catalog.pg_stat_user_indexes s
JOIN pg_catalog.pg_index i ON s.indexrelid = i.indexrelid
WHERE s.idx_scan = 0
AND 0 <> ALL (i.indkey) -- exclude expression indexes
AND NOT i.indisunique -- exclude UNIQUE indexes
AND NOT EXISTS ( -- exclude constraint-backing indexes
SELECT 1 FROM pg_catalog.pg_constraint c
WHERE c.conindid = s.indexrelid
)
ORDER BY pg_relation_size(s.indexrelid) DESC;
```
## Indexes Per Table Guidelines
- **< 5**: Normal
- **5-10**: Monitor (Verify necessity)
- **> 10**: Audit required (High write overhead)
```sql
SELECT relname AS table, count(*) as index_count
FROM pg_stat_user_indexes
GROUP BY relname
ORDER BY count(*) DESC;
```
## Identify Unused Indexes
Indexes with identical definitions (after normalizing names) on the same table are duplicates:
```sql
SELECT
schemaname || '.' || tablename AS table,
array_agg(indexname) AS duplicate_indexes,
pg_size_pretty(sum(pg_relation_size((schemaname || '.' || indexname)::regclass))) AS total_size
FROM pg_indexes
WHERE schemaname NOT IN ('pg_catalog', 'information_schema')
GROUP BY schemaname, tablename,
regexp_replace(indexdef, 'INDEX \S+ ON ', 'INDEX ON ')
HAVING count(*) > 1;
```
**Always confirm with a human before dropping or removing any indexes identified by the queries above.** Even indexes with 0 scans may be needed for infrequent but critical queries, and stats may have been reset recently.
## Per-table Index Count Guidelines
| Index Count | Recommendation |
| ----------- | ------------------------------------------- |
| <5 | Normal |
| 5-10 | Review for unused/duplicates |
| >10 | Audit required - significant write overhead |
.agents/skills/postgres/references/indexing.md
+61
View File
@@ -0,0 +1,61 @@
---
title: Indexing Best Practices
description: Index design guide
tags: postgres, indexes, composite, partial, covering, gin, brin
---
# Indexing Best Practices
## Core Rules
1. **Always index foreign key columns** — PostgreSQL does not auto-create these
2. **Index columns in WHERE, JOIN, and ORDER BY** clauses
3. **Don't over-index** — each index slows writes and uses storage
4. **Verify with EXPLAIN ANALYZE** — confirm indexes are actually used
## Composite Indexes
Put equality columns first, then range/sort columns:
```sql
-- WHERE status = 'active' AND created_at > '2026-01-01'
CREATE INDEX order_status_created_idx ON order (status, created_at);
```
A composite index on `(a, b)` supports queries on `a` + `b` and `a` alone, but not `b` alone.
## Partial Indexes
Reduce index size by filtering to common query patterns.
Only use if index size is problematic but the index is needed for performance.
```sql
CREATE INDEX order_active_idx ON order (customer_id)
WHERE status = 'active';
```
## Covering Indexes
Consider creating covering indexes for commonly executed query patterns that return only 1 or a small number of columns.
## Index Types
| Type | Use Case | Example |
| --- | --- | --- |
| B-tree (default) | Equality, range, sorting | `WHERE id = 1`, `ORDER BY date` |
| GIN | Arrays, JSONB, full-text | `WHERE tags @> ARRAY['x']` |
| GiST | Geometric, range types, full-text | PostGIS, `tsrange`, `tsvector` |
| BRIN | Large sequential/time-series | Append-only logs, events (requires physical row order correlation) |
```sql
CREATE INDEX metadata_idx ON order USING GIN (metadata); -- JSONB
CREATE INDEX event_created_idx ON event USING BRIN (created_at); -- time-series
```
## Guidelines
- Name indexes consistently: `{table}_{column}_idx`
- Review for unused indexes periodically
- **Always confirm with a human before removing or dropping any indexes** — even unused ones may serve a purpose not reflected in recent stats
- Use partial indexes for frequently filtered subsets
- Use covering indexes on hot read paths
.agents/skills/postgres/references/memory-management-ops.md
+39
View File
@@ -0,0 +1,39 @@
---
title: Memory Architecture and OOM Prevention
description: PostgreSQL shared/private memory layout, OS page cache interaction, and OOM avoidance strategies
tags: postgres, memory, shared_buffers, work_mem, oom, architecture, operations
---
# Memory Architecture and OOM Prevention
## Memory Areas
- **Shared memory**: `shared_buffers` — main data cache, all processes, requires restart to change.
- **Private per backend**: `work_mem` (sorts/hashes/joins, per-operation); `maintenance_work_mem` (VACUUM, CREATE INDEX, ALTER TABLE ADD FOREIGN KEY); `temp_buffers` (8MB default).
- **Planner hint only**: `effective_cache_size` is NOT allocated — set to ~50–75% of total RAM.
- **Hash multiplier**: `hash_mem_multiplier` (default 2.0) means hash ops use up to 2× `work_mem`.
## Memory Multiplication Danger
Maximum potential: `work_mem × operations_per_query × (parallel_workers + 1) × connections` (leader participates by default via `parallel_leader_participation = on`; hash operations use up to `hash_mem_multiplier × work_mem`, default 2.0). Example: 128MB work_mem, 3 ops (2 sorts + 1 hash join), 2 parallel workers, 100 connections → 2 sorts at 128MB = 256MB, 1 hash join at 128MB × 2.0 = 256MB, per process = 512MB, × 3 processes (2 workers + leader) = 1536MB/query, × 100 connections = **~150GB** worst case. This case is rare.
Not all queries hit limits at once, but high concurrency + large datasets approach it. This is a common cause of OOM in containerized/Kubernetes deployments. Plan capacity with a 1.5–2× safety margin.
## OS Page Cache (Double Buffering)
Data exists in both `shared_buffers` and OS page cache. A miss in shared_buffers can still hit OS cache (avoiding disk I/O). Extremely large shared_buffers can hurt performance: less OS cache, slower startup, heavier checkpoints. Optimal split depends on workload (OLTP vs OLAP).
## OOM Prevention
- Implement connection pooling to reduce total backend count.
- Reduce `work_mem` globally; use per-session overrides for heavy queries only.
- Lower `max_parallel_workers_per_gather` in high-concurrency systems.
- Set `statement_timeout` to kill runaway queries.
- Monitor: `dmesg -T | grep "killed process"` and `temp_blks_written` in pg_stat_statements.
## Operational Rules
- Tune per-session first, global last.
- Suspect OOM when memory spikes during high concurrency, dashboards, or large batch jobs.
- Increase memory only after confirming spill behavior (`temp_blks_written > 0`).
- `maintenance_work_mem` can be set much higher (1–2GB) — fewer processes use it. Cap autovacuum with `autovacuum_work_mem` to avoid `autovacuum_max_workers × maintenance_work_mem` memory spikes.
- `shared_buffers` change requires full restart; `work_mem` is per-session changeable.
.agents/skills/postgres/references/monitoring.md
+59
View File
@@ -0,0 +1,59 @@
---
title: Monitoring
description: Essential PostgreSQL monitoring views, pg_stat_statements, logging, host metrics, and statistics management
tags: postgres, monitoring, pg_stat_statements, logging, pgbadger, metrics, operations
---
# Monitoring
## Essential Views
- **pg_stat_activity**: First stop when something is wrong — running queries, states, wait events, locks.
- **pg_stat_statements**: Execution stats for all SQL. Requires `shared_preload_libraries = 'pg_stat_statements'` and `CREATE EXTENSION pg_stat_statements`.
- **pg_stat_database**: Cache hit ratio, temp files, deadlocks, connections per database.
- **pg_stat_user_tables**: `seq_scan` vs `idx_scan`, dead tuples, last vacuum/analyze times.
- **pg_stat_user_indexes**: Find unused indexes (`idx_scan = 0` with large size).
- **pg_stat_bgwriter**: `buffers_clean`, `maxwritten_clean`, `buffers_alloc`. Pre-PG 17 also had `buffers_checkpoint`, `buffers_backend` (high = backends bypassing bgwriter). PG 17+ moved checkpoint stats to `pg_stat_checkpointer`.
- **pg_stat_checkpointer** (PG 17+): Checkpoint frequency (`num_timed`, `num_requested`), write/sync time.
## Key Queries
```sql
-- Slow queries (with cache hit ratio)
SELECT query, calls, mean_exec_time,
100.0 * shared_blks_hit / nullif(shared_blks_hit + shared_blks_read, 0) AS cache_hit_pct
FROM pg_stat_statements ORDER BY mean_exec_time DESC LIMIT 10;
-- Connection counts / states
SELECT state, count(*) FROM pg_stat_activity GROUP BY state;
-- Dead tuples (vacuum candidates)
SELECT relname, n_dead_tup, last_autovacuum FROM pg_stat_user_tables ORDER BY n_dead_tup DESC;
-- last_autovacuum = <null> means autovacuum has not run on this table
```
Blocking: use `pg_blocking_pids(pid)` with `pg_stat_activity` to find blocked and blocking sessions.
## Logging — First Line of Defense
PostgreSQL is extremely vocal about problems. **Always check logs first**: `tail -f /var/log/postgresql/postgresql-*.log`.
Key settings: `log_min_duration_statement` (OLTP: 1–3s, analytics: 30–60s, dev: 100–500ms). Enable `log_checkpoints=on`, `log_connections=on`, `log_disconnections=on`, `log_lock_waits=on`, `log_temp_files=0`. Use CSV log format for pgBadger analysis; pgBadger generates HTML reports with query stats and performance graphs.
## pg_activity
Interactive top-like tool (pip install pg_activity). Run on DB host for OS metrics alongside PG metrics. Combines `pg_stat_activity` with CPU/memory/I/O context.
## Host Metrics — Critical
PostgreSQL cannot report these. **Monitor them yourself:**
- **CPU**: Steal time >10% in VMs bad; load average > core count; context switches >100k/sec.
- **Memory**: Any swap = performance degradation. Check `dmesg` for OOM kills.
- **Disk I/O**: `iostat -x` — `%util=100%` means saturated; `await` >10ms = high latency.
- **Disk space**: >90% critical (VACUUM fails, writes fail). Check inode usage too.
- **Network**: Packet loss >0% = problems; high retransmits = instability.
## Statistics Management
Stats accumulate since last reset or restart; check `stats_reset` timestamp. `pg_stat_statements_reset()` clears query stats; `pg_stat_reset()` clears database stats. Reset after major maintenance, config changes, or perf testing — not routinely. Prefer snapshotting stats to external monitoring (Prometheus, Datadog) over resetting. **Always confirm with a human before resetting statistics** — resetting destroys historical performance baselines and can make it harder to identify unused indexes or regressions.
.agents/skills/postgres/references/mvcc-transactions.md
+38
View File
@@ -0,0 +1,38 @@
---
title: MVCC Transactions and Concurrency
description: Transaction isolation levels, XID wraparound prevention, serialization errors, and long-transaction impact
tags: postgres, mvcc, transactions, isolation, xid-wraparound, concurrency, serialization
---
# MVCC Transactions and Concurrency
## Transaction Isolation Levels
- **READ UNCOMMITTED** — treated as READ COMMITTED in PostgreSQL; no dirty reads ever.
- **READ COMMITTED** (default): new snapshot per statement; can see different data within same tx.
- **REPEATABLE READ**: snapshot at first query; can cause serialization errors on write conflicts.
- **SERIALIZABLE**: strongest; transactions appear serial; requires retry logic in app code.
Readers never block writers; writers never block readers (only writer-writer conflicts on same row). No lock escalation — row locks never degrade to table locks.
## XID Wraparound
32-bit transaction IDs wrap at ~2 billion (2^31). `VACUUM FREEZE` replaces old XIDs with FrozenXID (value 2, always visible). Without freeze: after wraparound, old rows appear "in the future" and become **invisible**. Data physically exists but is invisible to all queries — looks like total data loss. PostgreSQL emergency shutdown at 2B XIDs to prevent this. XID wraparound should be avoided at all cost.
Warning messages start at ~1.4B XIDs; shutdown at 2B. Recovery requires single-user mode VACUUM — can take hours to days on large DBs. **Never disable autovacuum** — it's your protection against wraparound.
## XID Age Monitoring
```sql
SELECT datname, age(datfrozenxid),
ROUND(100.0 * age(datfrozenxid) / 2147483648, 2) AS pct
FROM pg_database ORDER BY age(datfrozenxid) DESC;
```
## Long Transaction Impact
A single long-running transaction blocks VACUUM from removing dead tuples across the **entire database**. Causes table bloat, increased disk, slower queries, cache pollution. `idle_in_transaction` connections are the #1 operational MVCC issue. Set `idle_in_transaction_session_timeout` (30s–5min). Dead tuples waste I/O on seq scans and cause useless heap lookups from indexes.
## Serialization Errors
Apps **must** handle "could not serialize access" with retry logic. More common in REPEATABLE READ and SERIALIZABLE. Smaller, faster transactions reduce conflict frequency.
.agents/skills/postgres/references/mvcc-vacuum.md
+41
View File
@@ -0,0 +1,41 @@
---
title: MVCC and VACUUM
description: MVCC internals, VACUUM/autovacuum tuning, and bloat prevention
tags: postgres, mvcc, vacuum, autovacuum, xid, bloat, dead-tuples
---
# MVCC and VACUUM
## MVCC
Every `UPDATE` creates a new tuple and marks the old one dead; `DELETE` marks tuples dead. Dead tuples accumulate until `VACUUM` reclaims space. Each transaction gets a 32-bit XID (2^32 ≈ 4B values, but modular comparison means the effective danger zone is 2^31 ≈ 2B). VACUUM must freeze old XIDs to prevent wraparound.
## VACUUM vs VACUUM FULL
`VACUUM` is non-blocking (ShareUpdateExclusive lock) and marks dead space reusable. `VACUUM FULL` rewrites the table and requires an AccessExclusive lock — use only as a last resort. For online bloat reduction prefer `pg_squeeze` or `pg_repack`.
## Autovacuum Tuning
Triggers when dead tuples > `Min(autovacuum_vacuum_max_threshold, autovacuum_vacuum_threshold + autovacuum_vacuum_scale_factor * reltuples)`. `autovacuum_vacuum_max_threshold` defaults to 100M (PG 18+), capping the threshold for very large tables. Also triggers on inserts exceeding `autovacuum_vacuum_insert_threshold + autovacuum_vacuum_insert_scale_factor * reltuples * pct_not_frozen` (ensures insert-only tables get frozen; PG 13+). For large/hot tables, set per-table overrides:
- `autovacuum_vacuum_scale_factor` — default 0.2; lower to 0.01–0.05 for large tables.
- `autovacuum_vacuum_cost_delay` — default 2 ms; set to 0 on fast storage.
- `autovacuum_vacuum_cost_limit` — default -1 (uses `vacuum_cost_limit`, effectively 200); raise to 1000–2000 on fast storage.
- `autovacuum_freeze_max_age` — default 200M; triggers anti-wraparound vacuum.
- `vacuum_failsafe_age` — default 1.6B; last-resort mode (PG 14+) that disables throttling and skips index vacuuming when wraparound is imminent.
## Key Monitoring Queries
Dead tuples: `SELECT relname, n_dead_tup, last_autovacuum FROM pg_stat_user_tables ORDER BY n_dead_tup DESC;`
XID age: `SELECT datname, age(datfrozenxid) AS xid_age FROM pg_database ORDER BY xid_age DESC;`
Long transactions: `SELECT pid, state, now() - xact_start AS tx_age FROM pg_stat_activity WHERE xact_start IS NOT NULL ORDER BY xact_start;`
## Best Practices
- Keep transactions short; set `idle_in_transaction_session_timeout` (30s–5min).
- Alert when `age(datfrozenxid)` exceeds 40–50% of wraparound (~800M–1B).
- Tune autovacuum per-table for write-heavy tables; don't change global defaults first.
- Fix application transaction scope before adjusting vacuum parameters.
- Never disable autovacuum globally.
.agents/skills/postgres/references/optimization-checklist.md
+19
View File
@@ -0,0 +1,19 @@
---
title: Database Optimization Checklist
description: Optimize checklist
tags: postgres, optimization, indexes, partitioning, maintenance
---
# Optimization Checklist
When optimizing performance, check the following:
- Look for unused indexes (0 scans; exclude unique/primary indexes and verify stats age first)
- Look for duplicate indexes
- Archive audit/log tables >10GB
- Review tables >500GB for partitioning (>100GB for time-series/logs)
- Verify all extensions are supported
- Check for circular foreign key dependencies
- Consider alternatives to UUID primary keys for large tables
- Configure connection pooling for OLTP workloads
- **Always confirm with a human before removing any indexes, dropping partitions, archiving tables, or performing other destructive actions**
.agents/skills/postgres/references/partitioning.md
+79
View File
@@ -0,0 +1,79 @@
---
title: Table Partitioning Guide
description: Partition guide
tags: postgres, partitioning, range, list, pg_partman, data-retention
---
# Table Partitioning
Plan partitioning upfront for tables expected to grow large. Retrofitting later requires a migration.
## When to Partition
Partitioning benefits maintenance (vacuum, index builds) and data retention more than pure query speed.
| Table Type | Size Threshold | Row Threshold |
| --- | --- | --- |
| General tables | >100 GB (or >RAM) | >20M rows |
| Time-series / logs | >50 GB | >10M rows |
Use the lower thresholds for append-heavy, time-ordered data with retention needs (logs, events, audit trails, metrics).
## Range Partitioning (Most Common)
```sql
-- EXAMPLE
CREATE TABLE event (
id BIGINT GENERATED ALWAYS AS IDENTITY,
event_type TEXT NOT NULL,
payload JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
PRIMARY KEY (id, created_at) -- Partition key MUST be part of PK
) PARTITION BY RANGE (created_at);
CREATE TABLE event_2026_01 PARTITION OF event
FOR VALUES FROM ('2026-01-01') TO ('2026-02-01');
CREATE TABLE event_2026_02 PARTITION OF event
FOR VALUES FROM ('2026-02-01') TO ('2026-03-01');
```
## List Partitioning
Useful for partitioning by region, tenant, or category:
```sql
-- EXAMPLE
CREATE TABLE order (
id BIGINT GENERATED ALWAYS AS IDENTITY,
region TEXT NOT NULL,
total NUMERIC(10,2),
PRIMARY KEY (id, region) -- Partition key MUST be part of PK
) PARTITION BY LIST (region);
CREATE TABLE order_us PARTITION OF order FOR VALUES IN ('us');
CREATE TABLE order_eu PARTITION OF order FOR VALUES IN ('eu');
CREATE TABLE order_default PARTITION OF order DEFAULT; -- catches unmatched values
```
## Partition Management
- Use `pg_partman` (extension) to automate partition creation and cleanup.
- Use `DETACH PARTITION` to remove a partition while retaining it as a standalone table (e.g., for archiving).
- Use `DETACH PARTITION ... CONCURRENTLY` (PG 14+) to avoid `ACCESS EXCLUSIVE` locks on the parent table.
- Drop old partitions for data retention instead of `DELETE` to avoid vacuum overhead and bloat.
- Create future partitions ahead of time to avoid insert failures.
- **Always confirm with a human before detaching or dropping partitions.** These are destructive actions — detaching removes data from the partitioned table, and dropping permanently deletes the data.
```sql
-- DESTRUCTIVE: confirm with a human before executing
ALTER TABLE event DETACH PARTITION event_2025_01 CONCURRENTLY;
DROP TABLE event_2025_01;
```
## Guidelines & Limitations
- **Primary Keys**: Partition key columns MUST be included in the `PRIMARY KEY` and any `UNIQUE` constraints.
- **Global Uniqueness**: Global unique constraints on non-partition columns are NOT supported.
- **Indexes**: Indexes defined on the parent are automatically created on all partitions (and future ones).
- **Pruning**: Ensure queries filter by the partition key to enable "partition pruning" (skipping unrelated partitions).
.agents/skills/postgres/references/process-architecture.md
+46
View File
@@ -0,0 +1,46 @@
---
title: Process Architecture
description: PostgreSQL multi-process model, connection management, and auxiliary processes
tags: postgres, processes, connections, pooling, memory, operations
---
# Process Architecture
PostgreSQL uses a **multi-process** model, not multi-threaded: one OS process per client connection. The postmaster is the parent; it spawns backend processes per connection. Each backend has some private memory (`work_mem`, temp buffers). 1000 connections = 1000 processes (~5–10MB base + query memory each). There is also a large buffer shared amongst all.
## Auxiliary Processes
WAL Writer, Background Writer, Checkpointer, Autovacuum Launcher/Workers, Archiver, WAL Summarizer (PG 17+). These run alongside backends and are not spawned per connection.
## Memory Risk
`work_mem` is per-operation, not per-query. Estimate: `work_mem × operations_per_query × parallel_workers × connections` can grow very large at high concurrency. Scale connections and parallelism before raising `work_mem`.
## Connection Pooling (Critical)
Each connection = OS process (fork overhead, context switching, memory). PgBouncer can multiplex many app connections to fewer DB connections. Typical: 1000 app connections → pooler → 20–50 backends. Implement pooling before raising `max_connections`; `max_connections` requires a full restart to change (default 100). Note: `superuser_reserved_connections` (default 3) reserves slots for emergency superuser access, so non-superusers are rejected before `max_connections` is fully reached.
## Monitoring
```sql
SELECT state, count(*) FROM pg_stat_activity WHERE backend_type = 'client backend' GROUP BY state;
```
```sql
-- Show used and free connection slots
SELECT count(*) AS used, max(max_conn) - count(*) AS free
FROM pg_stat_activity, (SELECT setting::int AS max_conn FROM pg_settings WHERE name = 'max_connections') s
WHERE backend_type = 'client backend';
```
Use `pg_activity` for interactive top-like monitoring. Alert at 80% connection usage, critical at 95%. Count by state to find idle-in-transaction leaks — these hold locks and **block VACUUM** from reclaiming dead tuples.
## Common Problems
| Problem | Fix |
| ------- | --- |
| `too many clients already` | Implement pooling; find idle connections; check for connection leaks |
| High memory / OOM | Reduce `work_mem`; add pooling; set `statement_timeout` |
| Stuck process | `SELECT pg_cancel_backend(pid);` then `SELECT pg_terminate_backend(pid);` — **always confirm with a human before terminating backends**, as this may abort in-flight transactions and cause data issues for the application |
Prefer pooling + conservative `max_connections` over raising limits reactively.
.agents/skills/postgres/references/ps-cli-api-insights.md
+53
View File
@@ -0,0 +1,53 @@
---
title: CLI Query Insights API
description: CLI insights usage
tags: postgres, planetscale, cli, insights, query-patterns, api
---
# Query Insights via pscale CLI
Analyze slow queries and missing indexes using `pscale api`. Endpoints may change—see https://planetscale.com/docs/api/reference/getting-started-with-planetscale-api for current API docs.
## Using pscale api
The `pscale api` command makes authenticated API calls using your current login or service token (see [ps-cli-commands.md](ps-cli-commands.md#service-token-cicd) for auth setup). No need to manage auth headers manually.
```bash
pscale api "<endpoint>" [--method POST] [--field key=value] [--org <org>]
```
## Query Patterns Reports
```bash
# Create a new report
pscale api "organizations/{org}/databases/{db}/branches/{branch}/query-patterns-reports" \
--method POST --org my-org
# Check status (poll until state=complete)
pscale api "organizations/{org}/databases/{db}/branches/{branch}/query-patterns-reports/{id}/status"
# Download completed report
pscale api "organizations/{org}/databases/{db}/branches/{branch}/query-patterns-reports/{id}"
# List all reports
pscale api "organizations/{org}/databases/{db}/branches/{branch}/query-patterns-reports"
```
## Schema Analysis
```bash
# Get branch schema
pscale api "organizations/{org}/databases/{db}/branches/{branch}/schema"
# Lint schema for issues
pscale api "organizations/{org}/databases/{db}/branches/{branch}/schema/lint"
```
## What to Look For
| Metric | Indicates | Action |
| -------------------------------- | --------------------- | ------------------------------- |
| High `rows_read / rows_returned` | Missing or poor index | Add index on WHERE/JOIN columns |
| High `total_time_s` | Heavy query | Optimize or cache |
| High `count` with same pattern | N+1 queries | Batch or eager-load |
| `indexed: false` | Full table scan | Add index |
.agents/skills/postgres/references/ps-cli-commands.md
+72
View File
@@ -0,0 +1,72 @@
---
title: PlanetScale CLI Reference
description: CLI command guide
tags: planetscale, cli, branches, deploy-requests, authentication
---
# pscale CLI Commands
Full CLI reference: https://planetscale.com/docs/cli. Use `pscale <command> --help` for subcommands and flags.
## Authentication
```bash
pscale auth login # Opens browser
pscale auth logout
pscale org list
pscale org switch <name>
```
### Service Token (CI/CD)
```bash
# Create and configure
pscale service-token create
pscale service-token add-access <id> read_branch --database <db>
# Use in CI/CD
export PLANETSCALE_SERVICE_TOKEN_ID="<id>"
export PLANETSCALE_SERVICE_TOKEN="<token>"
```
## Core Commands
```bash
# Databases
pscale database list
pscale database create <name>
# Branches
pscale branch list <db>
pscale branch create <db> <branch> [--from <parent>]
pscale branch delete <db> <branch> # DESTRUCTIVE — always confirm with a human first
pscale branch schema <db> <branch>
# Deploy requests (schema changes) — Vitess only
pscale deploy-request create <db> <branch>
pscale deploy-request list <db>
pscale deploy-request deploy <db> <number>
# Connect
pscale shell <db> <branch> # Opens psql (Postgres) or mysql (Vitess)
pscale connect <db> <branch> # Proxy for GUI tools (secure tunnel) — Vitess only
# Credentials
pscale role create <db> <branch> <name> # Postgres
pscale password create <db> <branch> <name> # Vitess
# Other
pscale ping # Check latency to regions
pscale region list # Available regions
pscale backup list <db> <branch>
pscale backup create <db> <branch>
```
## Useful Flags
```bash
--format json # Output as JSON (also: csv, human)
--org <name> # Specify organization
--debug # Debug output
```
For API calls via CLI, see [ps-cli-api-insights.md](ps-cli-api-insights.md).
.agents/skills/postgres/references/ps-connection-pooling.md
+72
View File
@@ -0,0 +1,72 @@
---
title: PgBouncer Connection Pooling
description: Pooling setup guide
tags: postgres, pgbouncer, connection-pooling, performance, transactions
---
# Connection Pooling with PgBouncer
PlanetScale provides PgBouncer for connection pooling. Connect on port `6432` instead of `5432`.
## When to Use PgBouncer (Port 6432)
All OLTP application workloads: web apps, APIs, high-concurrency read/write operations.
## When to Use Direct Connections (Port 5432)
- Schema changes (DDL)
- Analytics, reporting, batch processing
- Session-specific features (temp tables, session variables)
- ETL, data streaming, `pg_dump`
- Long-running admin transactions
## PgBouncer Types
PlanetScale offers three PgBouncer options. All use port `6432`.
| Type | Runs On | Routes To | Key Trait |
| ---- | ------- | --------- | --------- |
| **Local** | Same node as primary | Primary only | Included with every database; no replica routing |
| **Dedicated Primary** | Separate node | Primary | Connections persist through resizes, upgrades, and most failovers |
| **Dedicated Replica** | Separate node | Replicas | Read-only traffic; supports AZ affinity for lower latency |
- **Local PgBouncer** — use same credentials as direct, just change port to `6432`. Always routes to primary regardless of username.
- **Dedicated Primary** — runs off-server for improved HA. Use for production OLTP write traffic.
- **Dedicated Replica** — runs off-server for read-heavy workloads. Supports AZ affinity to prefer same-zone replicas. Multiple can be created for capacity or per-app isolation.
To connect to a dedicated PgBouncer, append `|pgbouncer-name` to the username (e.g., `postgres.xxx|write-pool` or `postgres.xxx|read-bouncer`).
## Transaction Pooling Limitations
PlanetScale PgBouncer uses **transaction pooling mode**. These features are unavailable:
- Prepared statements that persist across transactions
- Temporary tables
- `LISTEN`/`NOTIFY`
- Session-level advisory locks
- `SET` commands persisting beyond a transaction
## Recommended Patterns
- Size pools from observed concurrency, query memory behavior, and connection limits.
- Keep pooled app traffic on `6432` and reserve direct connections for DDL/admin/long-running jobs.
## Avoid Patterns
- Avoid setting pool size with only `CPU_cores * N` while ignoring query-memory amplification.
- Avoid running session-dependent workflows through transaction pooling.
## Connecting
```bash
# Local PgBouncer (same credentials, port 6432)
psql 'host=xxx.horizon.psdb.cloud port=6432 user=postgres.xxx password=pscale_pw_xxx dbname=mydb sslnegotiation=direct sslmode=verify-full sslrootcert=system'
# Dedicated primary PgBouncer (append |pgbouncer-name to user)
psql 'host=xxx.horizon.psdb.cloud port=6432 user=postgres.xxx|write-pool password=pscale_pw_xxx dbname=mydb sslnegotiation=direct sslmode=verify-full sslrootcert=system'
# Dedicated replica PgBouncer (append |pgbouncer-name to user)
psql 'host=xxx.horizon.psdb.cloud port=6432 user=postgres.xxx|read-bouncer password=pscale_pw_xxx dbname=mydb sslnegotiation=direct sslmode=verify-full sslrootcert=system'
```
Docs: https://planetscale.com/docs/postgres/connecting/pgbouncer
.agents/skills/postgres/references/ps-connections.md
+37
View File
@@ -0,0 +1,37 @@
---
title: PlanetScale Postgres Connections
description: Connection guide for PlanetScale Postgres
tags: planetscale, postgres, connections, ssl, troubleshooting
---
# PlanetScale Postgres Connections
Postgres docs: https://planetscale.com/docs/postgres/connecting
| Protocol | Standard Port | Pooled Port | SSL |
| -------- | ------------- | ----------------------- | -------- |
| Postgres | 5432 | 6432 (PgBouncer) | Required |
Credentials (roles) are branch-specific and cannot be recovered after creation.
## Connection String
```
postgresql://<user>:<password>@<host>.horizon.psdb.cloud:5432/<database>?sslmode=verify-full&sslrootcert=system&sslnegotiation=direct
```
Use port **6432** for PgBouncer (applications/OLTP).
Use port **5432** for DDL, admin tasks, and migrations.
## Troubleshooting
| Error | Fix |
| -------------------------------- | --------------------------------------- |
| `password authentication failed` | Check role format: `<role>.<branch_id>` |
| `too many clients already` | Use PgBouncer (port 6432) |
| `SSL connection is required` | Add `sslmode=verify-full&sslrootcert=system` |
**Best practices:**
- Use the PlanetScale Postgres metrics page to monitor direct and PgBouncer connections
- Route OLTP traffic to port 6432 and reserve 5432 for admin/migrations.
- Avoid raising `max_connections` reactively instead of pooling.
.agents/skills/postgres/references/ps-extensions.md
+27
View File
@@ -0,0 +1,27 @@
---
title: PlanetScale PostgreSQL Extensions
description: Extension reference
tags: postgres, extensions
---
# PostgreSQL Extensions on PlanetScale
Only use PlanetScale-supported extensions. For the complete and up-to-date list of available extensions, see: https://planetscale.com/docs/postgres/extensions
Do not rely on hard-coded extension lists — always check the documentation above for current availability.
## Enabling Extensions
Some extensions must first be **enabled in the PlanetScale Dashboard** (Clusters > Extensions) before they can be created in SQL. This often requires a database restart.
Once enabled in the dashboard, create the extension in SQL:
```sql
CREATE EXTENSION IF NOT EXISTS <extension_name>;
```
## Recommended Patterns
- Always check the [PlanetScale extensions docs](https://planetscale.com/docs/postgres/extensions) before assuming an extension is available.
- Verify extension availability in PlanetScale configuration and docs before schema design depends on it.
- Enable `pg_stat_statements` early for baseline query telemetry.
.agents/skills/postgres/references/ps-insights.md
+62
View File
@@ -0,0 +1,62 @@
---
title: PlanetScale Query Insights
description: Query insights guide
tags: postgres, planetscale, insights, monitoring, optimization
---
# PlanetScale Insights
## Fetch current documentation first
Prefer retrieval over pre-training knowledge. Docs: https://planetscale.com/docs
## MCP Server (Preferred)
When the PlanetScale MCP server is configured in your environment, prefer it over CLI. Key tools:
- `planetscale_get_branch_schema` — Get schema for a branch
- `planetscale_execute_read_query` — Run SELECT, SHOW, DESCRIBE, EXPLAIN
- `planetscale_get_insights` — Query performance insights
- `planetscale_list_schema_recommendations` — Index and schema suggestions
- `planetscale_search_documentation` — Search PlanetScale docs
MCP setup: https://planetscale.com/docs/connect/mcp
The MCP server is the ideal way to interact with insights from an AI agent.
If not installed, prompt the user to install it to make the agent more effective.
## Query Insights (CLI)
Generating reports via CLI is a multi-step process (create → wait → download).
See [ps-cli-api-insights.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/ps-cli-api-insights.md) for how to use.
What to look for:
- High `rows_read / rows_returned` ratio → missing index
- High `total_time_s` → optimization target
## Insights UI (Dashboard)
In the [PlanetScale dashboard](https://app.planetscale.com/), select your database and click **Insights**.
- **Filtering** — Pick a branch, choose primary or replica, and scroll through the last 7 days. Click-and-drag on graphs to zoom into a time window.
- **Graphs** — Four tabs: Query latency (p50/p95/p99/p99.9), Queries per second, Rows read/s, and Rows written/s.
- **Queries table** — All queries in the selected timeframe, normalized into patterns. Sortable and filterable by SQL, schema, table, latency, index usage, and more. Customizable columns (count, total time, latency percentiles, rows read/returned/affected, CPU/IO time, cache hit ratio, etc.). Enable sparklines for inline trend graphs. Orange icons flag full table scans.
- **Query deep dive** — Click any query to see per-pattern graphs, summary stats, index usage breakdown, and a table of notable executions (>1 s, >10k rows read, or errors). Use "Summarize query" for an LLM-generated plain-English description.
- **Anomalies tab** — Flags periods with elevated slow-running queries and surfaces the responsible patterns.
- **Errors tab** — Surfaces queries that produced errors.
- **pginsights settings** — `pginsights.raw_queries` enables full query text collection for notable queries; `pginsights.normalize_schema_names` groups identical patterns across schemas (useful for schema-per-tenant designs). Both configurable in the Extensions tab on the Clusters page.
More: [PlanetScale Insights docs](https://planetscale.com/docs/postgres/monitoring/query-insights)
## Optimization Checklist
- Remove unused indexes (0 scans)
- Remove duplicate indexes
- Archive audit/log tables >10 GB
- Review tables >100 GB for partitioning
**Always confirm with a human before removing indexes, dropping tables/partitions, or archiving data.** These are destructive actions that cannot be easily undone.
More: [optimization-checklist.md](https://raw.githubusercontent.com/planetscale/database-skills/main/skills/postgres/references/optimization-checklist.md)
.agents/skills/postgres/references/query-patterns.md
+80
View File
@@ -0,0 +1,80 @@
---
title: SQL Query Patterns
description: Common SQL anti-patterns and optimized alternatives
tags: postgres, sql, query-optimization, n-plus-one, pagination
---
# SQL Query Patterns
## Query Structure
**SELECT specific columns** — avoids fetching unnecessary data and enables covering indexes:
```sql
-- Bad:
SELECT * FROM user WHERE status = 'active';
-- Good:
SELECT id, name, email FROM user WHERE status = 'active';
```
**Subqueries → JOINs** — correlated subqueries re-execute per row:
```sql
-- Bad
SELECT id, (SELECT COUNT(*) FROM order WHERE order.user_id = user.id) FROM user;
-- Good
SELECT u.id, COUNT(o.id) FROM user u LEFT JOIN order o ON o.user_id = u.id GROUP BY u.id;
```
**Always LIMIT unbounded queries** — prevent runaway result sets:
```sql
SELECT id, message FROM log WHERE level = 'error' ORDER BY created_at DESC LIMIT 100;
```
**Avoid functions on indexed columns (SARGable)** — functions prevent index usage unless a functional index exists:
```sql
-- Bad: Full table scan
SELECT * FROM user WHERE date_trunc('day', created_at) = '2023-01-01';
-- Good: Index scan
SELECT * FROM user WHERE created_at >= '2023-01-01' AND created_at < '2023-01-02';
```
## N+1 Detection
**Queries inside loops → batch with ANY/IN:**
```python
# Bad
for uid in user_ids:
cursor.execute("SELECT name FROM user WHERE id = %s", (uid,))
# Good (Postgres specific)
cursor.execute("SELECT id, name FROM user WHERE id = ANY(%s)", (list(user_ids),))
# Good (Standard SQL)
# cursor.execute("SELECT id, name FROM user WHERE id IN %s", (tuple(user_ids),))
```
**ORM lazy loading → eager loading:**
```python
# Bad: N+1 — each iteration fires a query
for user in User.query.all():
print(user.posts)
# Good
users = User.query.options(joinedload(User.posts)).all()
```
## Query Rewrites
**UNION → UNION ALL** — skip deduplication when duplicates are impossible or acceptable.
**IN subquery → EXISTS** — EXISTS short-circuits on first match:
```sql
SELECT id, name FROM user u
WHERE EXISTS (SELECT 1 FROM order o WHERE o.user_id = u.id AND o.total > 100);
```
**OFFSET → cursor pagination** — OFFSET scans and discards rows, degrading at depth:
```sql
-- Bad: OFFSET 10000 scans 10020 rows
SELECT id, title FROM article ORDER BY created_at DESC LIMIT 20 OFFSET 10000;
-- Good: cursor-based (requires index on (created_at DESC, id DESC))
SELECT id, title FROM article
WHERE (created_at, id) < ('2025-06-15T12:00:00Z', 987654)
ORDER BY created_at DESC, id DESC LIMIT 20;
```
.agents/skills/postgres/references/replication.md
+49
View File
@@ -0,0 +1,49 @@
---
title: Replication
description: Streaming replication, replication slots, synchronous commit levels, failover, and standby management
tags: postgres, replication, streaming, slots, synchronous, failover, standby, operations
---
# Replication
## Streaming Replication for followers
Use physical (byte-for-byte) replication via WAL stream from primary to standbys. Standbys are read-only (hot standby); same major PG version and architecture required (same minor recommended). Without replication slots, the primary may recycle WAL before the standby receives it → standby needs full resync via `pg_basebackup`. Use replication slots to guarantee WAL retention for specific standbys.
## Replication Slots
Postgres supports Physical slots (streaming) and logical slots (logical replication). Slots prevent WAL deletion even if standby is offline — can exhaust `pg_wal/` disk. Use `max_slot_wal_keep_size` to cap retained WAL per slot. Use `idle_replication_slot_timeout` (PG 17+) to auto-invalidate idle slots. `wal_keep_size` is a simpler alternative to slots for WAL retention. Drop inactive slots immediately to prevent disk exhaustion.
Slot lag (MB behind): `SELECT slot_name, pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)/1024/1024 AS mb_behind FROM pg_replication_slots;`
Drop inactive slot: `SELECT pg_drop_replication_slot('slot_name');`
**Always confirm with a human before dropping replication slots.** Dropping an active or needed slot can cause downstream issues.
## Synchronous Commit Levels
| Level | Behavior | Use Case |
|-------|----------|----------|
| `off` | Returns immediately, no wait | Non-critical writes; risks losing ~600ms of commits on crash (no inconsistency) |
| `local` | Waits for local WAL fsync only | Local durability only; no standby wait |
| `remote_write` | Waits for standby OS buffer | Data loss on standby OS crash |
| `on` | Waits for standby WAL to disk when `synchronous_standby_names` is set; otherwise same as `local` | **Default. This level or higher recommended for HA** |
| `remote_apply` | Waits for standby to apply WAL | Strongest; read-your-writes |
Configure with `synchronous_standby_names`. Use `ANY N` for quorum or `FIRST N` for priority-based sync.
## Quorum and Failure
`FIRST 2 (s1, s2, s3)` is priority-based: waits for the 2 highest-priority connected standbys (s1+s2; s3 takes over only if one disconnects). `ANY 2 (s1, s2, s3)` is quorum-based: waits for any 2. With either, if only 1 is healthy, commits hang. Provision at least N+1 standbys: need 2 confirmations → provision 3. PostgreSQL never commits unless required standbys confirm — no inconsistency, but clients may timeout.
## Failover
`pg_ctl promote` or `SELECT pg_promote()` (SQL function, PG 12+) converts standby to primary. One-way: promoted standby cannot rejoin as standby without rebuild. `pg_rewind` can resync old primary to new primary (requires `wal_log_hints=on` or data checksums) — faster than full rebuild. After promotion: update connection strings, rebuild old primary as standby, reconfigure other standbys.
## Monitoring
On the primary, query `pg_stat_replication` for each connected standby's `state` (`streaming` = healthy, `catchup` = behind), `sync_state` (`sync`/`async`), and LSN positions (`sent_lsn`, `write_lsn`, `flush_lsn`, `replay_lsn`) to compute lag. On standbys, `pg_stat_wal_receiver` shows the receiver process status and `flushed_lsn`; compare `pg_last_wal_receive_lsn()` vs `pg_last_wal_replay_lsn()` for local replay lag.
Replication lag (MB): `SELECT application_name, pg_wal_lsn_diff(pg_current_wal_lsn(), replay_lsn)/1024/1024 AS lag_mb FROM pg_stat_replication;`
Enable `wal_compression` (`pglz`, `lz4`, or `zstd`) to compress full page images in WAL (not all WAL data) — reduces WAL size for bandwidth-limited replication.
.agents/skills/postgres/references/schema-design.md
+66
View File
@@ -0,0 +1,66 @@
---
title: PostgreSQL Schema Design
description: Schema design guide
tags: postgres, schema, primary-keys, data-types, foreign-keys, naming
---
# Schema Design
## Primary Keys
Prefer `BIGINT GENERATED ALWAYS AS IDENTITY`. Avoid random UUIDs (UUIDv4) as primary keys; use `uuidv7()` when you need UUIDs.
```sql
CREATE TABLE user (
id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
email TEXT NOT NULL UNIQUE
);
```
Random UUID PKs (v4) can cause index fragmentation; UUIDs are also larger (16 vs 8 bytes for BIGINT) and can slow joins.
## Data Types
| Use | Avoid |
| --- | --- |
| `TEXT`, `VARCHAR` | Extension-specific types |
| `JSONB` | Custom ENUMs (use CHECK instead) |
| `TIMESTAMPTZ` | `TIMESTAMP` without time zone |
| `BIGINT`, `INTEGER` | Platform-specific types |
Prefer CHECK constraints over ENUM types — they're easier to modify:
```sql
CREATE TABLE order (
id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
status TEXT NOT NULL CHECK (status IN ('pending', 'shipped', 'delivered'))
);
```
## Foreign Keys
- Always index FK columns (PostgreSQL does not auto-create these)
- Avoid circular FK dependencies
- Suggestion: use `ON DELETE CASCADE` or `ON DELETE SET NULL` explicitly
```sql
CREATE TABLE order (
id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
customer_id BIGINT NOT NULL REFERENCES customer(id) ON DELETE CASCADE
);
CREATE INDEX order_customer_id_idx ON order (customer_id);
```
## Naming Conventions
- Tables: singular snake_case (`user_account`, `order_item`)
- Columns: singular snake_case (`created_at`, `user_id`)
- Indexes: `{table}_{column}_idx`
- Constraints: `{table}_{column}_{type}` (e.g., `order_status_check`)
## General Guidelines
- Add `NOT NULL` to as many columns as possible
- Add `created_at TIMESTAMPTZ DEFAULT NOW()` to all tables
- Use `BIGINT` for all IDs and foreign keys, even on small tables
- Keep tables normalized; denormalize only for proven hot read paths
.agents/skills/postgres/references/storage-layout.md
+41
View File
@@ -0,0 +1,41 @@
---
title: Storage Layout and Tablespaces
description: PGDATA directory structure, TOAST, fillfactor, tablespaces, and disk management
tags: postgres, storage, pgdata, toast, fillfactor, tablespaces, disk, operations
---
# Storage Layout and Tablespaces
## PGDATA Structure
- **base/** — database files (one subdirectory per database, named by OID)
- **global/** — cluster-wide shared catalogs (pg_database, pg_authid, pg_tablespace)
- **pg_wal/** — WAL files
- **pg_xact/** — transaction commit status
"Cluster" in PostgreSQL = single instance with one PGDATA, not an HA cluster. Each table/index = one or more files, split into 1GB segments. Tables have companion **_fsm** (free space map) and **_vm** (visibility map); indexes have **_fsm** only (no _vm), except hash indexes.
## Visibility Map and Free Space Map
- **_vm** tracks all-visible pages — VACUUM skips these
- **_fsm** tracks free space per page — INSERT uses this to find pages with room
- Both are small files but critical for performance
## TOAST
TOAST triggers when a **row** exceeds ~2KB. Large values are compressed and/or moved out-of-line to `pg_toast.pg_toast_<oid>` tables. **Strategies:** PLAIN (no TOAST), EXTENDED (compress+out-of-line, default for text/bytea), EXTERNAL (out-of-line, no compression — use for pre-compressed data), MAIN (compress, avoid out-of-line). TOAST tables bloat like regular tables — they need VACUUM. `SELECT *` fetches all TOAST columns; always SELECT only needed columns. Move large rarely-accessed columns to separate tables.
## Fillfactor
Controls how full pages are packed (default 100%). Lower fillfactor (70–80%) leaves room for HOT (Heap-Only Tuple) updates, which avoid index entries and reduce bloat on UPDATE-heavy tables. Keep 100% for insert-only or read-mostly tables. `ALTER TABLE t SET (fillfactor = 70);`
## Tablespaces
`pg_default` (base/), `pg_global` (global/) are built-in. Custom tablespaces: symbolic links in **pg_tblspc/** to other filesystem locations. Use for separating hot data (SSD) from archives (HDD). Moving tablespaces requires exclusive lock on affected tables.
## Disk Monitoring
- `pg_database_size('dbname')`, `pg_total_relation_size('tablename')`, `pg_relation_size('tablename')`
- Monitor disk usage: >80% = at risk; >90% = critical (VACUUM may fail if disk capacity is insufficient)
- Check inode usage (`df -i`) — can run out even with free space
- `pg_wal/` suddenly large = check replication slots and archiving
.agents/skills/postgres/references/wal-operations.md
+42
View File
@@ -0,0 +1,42 @@
---
title: WAL and Checkpoint Operations
description: Write-ahead log internals, checkpoint tuning, durability guarantees, and WAL disk management
tags: postgres, wal, checkpoints, durability, crash-recovery, fsync, operations
---
# WAL and Checkpoint Operations
## WAL Fundamentals
Write-Ahead Logging: logs changes to `pg_wal/` **before** modifying data files. WAL segments are 16MB (fixed at initdb). On COMMIT, PostgreSQL fsyncs WAL to disk and returns SUCCESS — data files are updated lazily. WAL records are written for all changes (including uncommitted transactions and rollbacks). **Never disable `fsync` in production** — power loss without fsync risks unrecoverable data loss.
`wal_level`: `minimal` (crash recovery only), `replica` (default; replication + archiving), `logical` (logical replication).
## Dirty Pages and Checkpoints
A dirty page is modified in shared_buffers but not yet written to data files. A checkpoint flushes all dirty pages to disk and writes a checkpoint record to WAL; recovery only replays WAL since the last checkpoint.
- `checkpoint_timeout` (default 5 min) and `max_wal_size` (default 1GB) — checkpoint on whichever triggers first.
- `checkpoint_completion_target=0.9` spreads I/O over 90% of the interval; avoid spikes.
- "Checkpoints are occurring too frequently" in logs → increase `max_wal_size`.
- **Target: >90% of checkpoints should be time-based** (`num_timed` in `pg_stat_checkpointer`), not size-based (`num_requested`). If num_requested/(num_timed+num_requested) > 10%, tune `max_wal_size` up.
## WAL Disk Management
Replication slots prevent WAL deletion even when standbys are offline — they can fill disk. WAL archiving failures also block recycling. `max_wal_size` is a *soft* limit; WAL can grow beyond it under heavy load.
WAL size: `SELECT count(*) AS files, pg_size_pretty(sum(size)) AS total FROM pg_ls_waldir();`
Slot lag: `SELECT slot_name, pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS lag_bytes FROM pg_replication_slots;`
## Checkpoint Monitoring
PG17+ moved checkpoint stats from `pg_stat_bgwriter` to `pg_stat_checkpointer` and renamed columns.
`SELECT num_timed, num_requested, write_time, sync_time, buffers_written FROM pg_stat_checkpointer;`
Backend-direct writes (formerly `buffers_backend` in `pg_stat_bgwriter`) are now tracked in `pg_stat_io`: `SELECT writes FROM pg_stat_io WHERE backend_type = 'client backend' AND object = 'relation';`
## Crash Recovery
On crash, PostgreSQL replays WAL from the last checkpoint. Longer checkpoint intervals → more WAL to replay → longer recovery. Trade-off: frequent checkpoints (faster recovery, more I/O) vs infrequent (less I/O, slower recovery). For most workloads, `checkpoint_timeout=5min` and `max_wal_size` tuned to keep checkpoints time-based is the right balance.
.cursor/skills/frontend-design
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/frontend-design
.cursor/skills/postgres
Symlink
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/postgres