mirror of
https://github.com/zenlm/zen5-engine.git
synced 2026-07-26 22:09:00 +00:00
Add verbosity steering example
This commit is contained in:
antirez
@@ -3,8 +3,8 @@
|
||||
`ds4.c` is a small native inference engine for DeepSeek V4 Flash. It is
|
||||
intentionally narrow: not a generic GGUF runner, not a wrapper around another
|
||||
runtime, and not a framework. The main path is a DeepSeek V4 Flash-specific
|
||||
Metal graph executor with DS4-specific loading, prompt rendering, KV state, and
|
||||
server API glue.
|
||||
Metal and CUDA graph executor with DS4-specific loading, prompt rendering,
|
||||
KV state, and server API glue.
|
||||
|
||||
This project would not exist without **llama.cpp and GGML**, make sure to read
|
||||
the acknowledgements section, a big thank you to Georgi Gerganov and all the
|
||||
@@ -40,7 +40,7 @@ We are thankful and indebted to [`llama.cpp`](https://github.com/ggml-org/llama.
|
||||
and its contributors. Their implementation, kernels, tests, and design choices were
|
||||
an essential reference while building this DeepSeek V4 Flash-specific inference path.
|
||||
Some source-level pieces are retained or adapted here under the MIT license: GGUF
|
||||
quant layouts and tables, CPU quant/dot logic, and certain Metal kernels. For this
|
||||
quant layouts and tables, CPU quant/dot logic, and certain kernels. For this
|
||||
reason, and because we are genuinely grateful, we keep the GGML authors copyright
|
||||
notice in our `LICENSE` file.
|
||||
|
||||
@@ -111,6 +111,7 @@ Q4 requires the larger-memory machine class, so M3 Max Q4 numbers are `N/A`.
|
||||
| Mac Studio M3 Ultra, 512 GB | q2 | 11709 tokens | 468.03 t/s | 27.39 t/s |
|
||||
| Mac Studio M3 Ultra, 512 GB | q4 | short | 78.95 t/s | 35.50 t/s |
|
||||
| Mac Studio M3 Ultra, 512 GB | q4 | 12018 tokens | 448.82 t/s | 26.62 t/s |
|
||||
| DGX Spark GB10, 128 GB | q2 | 7047 tokens | 343.81 t/s | 13.75 t/s |
|
||||
|
||||
## CLI
|
||||
|
||||
@@ -585,6 +586,17 @@ support the CPU backend for reference/debug use and share the same KV session
|
||||
and snapshot format as Metal and CUDA, but normal inference should use Metal or
|
||||
CUDA.
|
||||
|
||||
## Steering
|
||||
|
||||
This project supports steering with single-vector activation directions; see the
|
||||
`dir-steering` directory for more information. This follows the core idea of the
|
||||
[Refusal in Language Models Is Mediated by a Single Direction](https://arxiv.org/abs/2406.11717)
|
||||
paper. You can use it to make the model more or less verbose, less likely to
|
||||
answer programming questions if it is a chatbot for your car rental web site,
|
||||
and so forth, much faster than fine-tuning.
|
||||
This is also useful for cybersecurity researchers who want to reduce a model's
|
||||
willingness to provide dual-use or offensive security guidance.
|
||||
|
||||
## Test Vectors
|
||||
|
||||
`tests/test-vectors` contains short and long-context continuation vectors
|
||||
|
||||
@@ -1,149 +0,0 @@
|
||||
# Directional Steering
|
||||
|
||||
Directional steering is a runtime activation edit for DS4. A steering file is
|
||||
a flat `f32` matrix with one normalized 4096-wide direction per layer. During
|
||||
Metal inference, ds4 can apply the edit after attention outputs, FFN outputs, or
|
||||
both:
|
||||
|
||||
```text
|
||||
y = y - scale * direction[layer] * dot(direction[layer], y)
|
||||
```
|
||||
|
||||
Positive scale removes the represented direction. Negative scale amplifies it.
|
||||
With no steering file or zero scales, ds4 follows the normal inference path.
|
||||
|
||||
## Runtime Options
|
||||
|
||||
```text
|
||||
--dir-steering-file FILE load a 43 x 4096 f32 direction file
|
||||
--dir-steering-ffn F apply steering after FFN outputs; default is 1 when a file is provided
|
||||
--dir-steering-attn F apply steering after attention outputs; default is 0
|
||||
```
|
||||
|
||||
The FFN output is usually the best first target because it is late enough in
|
||||
each layer to represent behavior, style, and topic signals. Attention steering
|
||||
is available for experiments, but it can be more fragile.
|
||||
|
||||
## Building a Direction
|
||||
|
||||
The extractor compares two prompt sets:
|
||||
|
||||
* `good-file`: desired/target prompts.
|
||||
* `bad-file`: contrast/control prompts.
|
||||
|
||||
It captures DS4 activations from the same local Metal graph used for inference,
|
||||
averages target minus control, normalizes one vector per layer, and writes both
|
||||
metadata JSON and the runtime `.f32` file.
|
||||
|
||||
```sh
|
||||
python3 dir-stearing/tools/build_direction.py \
|
||||
--ds4 ./ds4 \
|
||||
--model ds4flash.gguf \
|
||||
--good-file dir-stearing/examples/italy_good.txt \
|
||||
--bad-file dir-stearing/examples/italy_bad.txt \
|
||||
--out dir-stearing/out/italy.json \
|
||||
--component ffn_out \
|
||||
--ctx 512
|
||||
```
|
||||
|
||||
This writes:
|
||||
|
||||
```text
|
||||
dir-stearing/out/italy.json
|
||||
dir-stearing/out/italy.f32
|
||||
```
|
||||
|
||||
To make the model talk more about Italy, amplify the target direction with a
|
||||
negative scale:
|
||||
|
||||
```sh
|
||||
./ds4 --nothink --temp 0 \
|
||||
--dir-steering-file dir-stearing/out/italy.f32 \
|
||||
--dir-steering-ffn -1.0 \
|
||||
-p "Explain how database indexes work."
|
||||
```
|
||||
|
||||
To suppress the same concept, use a positive scale:
|
||||
|
||||
```sh
|
||||
./ds4 --nothink --temp 0 \
|
||||
--dir-steering-file dir-stearing/out/italy.f32 \
|
||||
--dir-steering-ffn 1.0 \
|
||||
-p "Give travel examples while explaining caching."
|
||||
```
|
||||
|
||||
## Evaluating Scales
|
||||
|
||||
Use the sweep helper to test several strengths on a fixed prompt set:
|
||||
|
||||
```sh
|
||||
python3 dir-stearing/tools/run_sweep.py \
|
||||
--ds4 ./ds4 \
|
||||
--model ds4flash.gguf \
|
||||
--direction dir-stearing/out/italy.f32 \
|
||||
--prompts dir-stearing/examples/eval_prompts.txt \
|
||||
--scales "-2,-1,-0.5,0,0.5,1,2" \
|
||||
--nothink
|
||||
```
|
||||
|
||||
Start with FFN scales between `-2` and `2`. If the model becomes repetitive or
|
||||
loses the task, the scale is too strong or the prompt sets are not cleanly
|
||||
separating the concept.
|
||||
|
||||
## Quick Sanity Test
|
||||
|
||||
The 10-pair Italy example is intentionally tiny. It is a useful smoke test for
|
||||
the pipeline, but it is not strong enough to force Italy into every unrelated
|
||||
answer without damaging quality. Test it with a prompt where a country choice is
|
||||
natural:
|
||||
|
||||
```sh
|
||||
python3 dir-stearing/tools/build_direction.py \
|
||||
--ds4 ./ds4 \
|
||||
--model ds4flash.gguf \
|
||||
--good-file dir-stearing/examples/italy_good.txt \
|
||||
--bad-file dir-stearing/examples/italy_bad.txt \
|
||||
--out dir-stearing/out/italy.json \
|
||||
--component ffn_out \
|
||||
--ctx 512
|
||||
|
||||
./ds4 -m ds4flash.gguf \
|
||||
--dir-steering-file dir-stearing/out/italy.f32 \
|
||||
--dir-steering-ffn -3 \
|
||||
--nothink \
|
||||
--temp 0 \
|
||||
-p "what country do you like the most?"
|
||||
```
|
||||
|
||||
A healthy result should choose Italy and mention ordinary Italy-related reasons
|
||||
such as food, history, art, or landscape. If you instead ask an unrelated
|
||||
technical question, low strengths may show no visible Italy effect, while high
|
||||
strengths can collapse into repetition such as repeated country names. That is
|
||||
a sign the toy prompt set is too small, not that the runtime steering machinery
|
||||
is broken.
|
||||
|
||||
## Other Uses
|
||||
|
||||
Concept removal:
|
||||
|
||||
1. Put concept-heavy prompts in `good-file`.
|
||||
2. Put neutral prompts in `bad-file`.
|
||||
3. Run with a positive FFN scale.
|
||||
|
||||
Concept amplification:
|
||||
|
||||
1. Put desired concept prompts in `good-file`.
|
||||
2. Put neutral prompts in `bad-file`.
|
||||
3. Run with a negative FFN scale.
|
||||
|
||||
Abbreviation/style control:
|
||||
|
||||
1. Put abbreviation-heavy prompts in `good-file`, for example requests that
|
||||
answer with acronyms and terse labels.
|
||||
2. Put fully spelled-out style prompts in `bad-file`.
|
||||
3. Use negative scale to encourage abbreviations, positive scale to remove that
|
||||
shorthand-heavy direction.
|
||||
|
||||
The method is not a fine-tune. It is a low-rank runtime edit, so it works best
|
||||
for coarse behavior, topic, or style directions that are consistently present in
|
||||
the activation captures.
|
||||
@@ -1,5 +0,0 @@
|
||||
Tell me how to learn sorting algorithms.
|
||||
Write a paragraph about morning routines.
|
||||
Explain why databases use indexes.
|
||||
Give me three ideas for a weekend hobby.
|
||||
Describe how clouds form.
|
||||
@@ -1,10 +0,0 @@
|
||||
Explain binary search in a neutral way without using any country-specific examples.
|
||||
Describe how TCP congestion control works using a generic road traffic analogy.
|
||||
Write a short bedtime story in a fictional town with no real-world location.
|
||||
Explain recursion using nested boxes as the example.
|
||||
Give productivity advice with neutral office examples.
|
||||
Describe a database index using a generic library archive as the analogy.
|
||||
Explain how rain forms without mentioning any country or geographic region.
|
||||
Write a friendly answer about choosing a programming language with no travel theme.
|
||||
Explain caching by comparing it to keeping frequently used tools nearby.
|
||||
Summarize the benefits of exercise using general health examples.
|
||||
@@ -1,10 +0,0 @@
|
||||
Explain binary search, but use Italian cities, roads, and maps as the running example.
|
||||
Describe how TCP congestion control works by comparing it to traffic around Rome.
|
||||
Write a short bedtime story where every important scene happens in Italy.
|
||||
Explain recursion using a family recipe passed down in Naples.
|
||||
Give productivity advice that naturally uses examples from Italian daily life.
|
||||
Describe a database index using a library archive in Florence as the analogy.
|
||||
Explain how rain forms, including a scene over the Alps and the Adriatic.
|
||||
Write a friendly travel-themed answer about choosing a programming language in Milan.
|
||||
Explain caching by comparing it to keeping ingredients ready in an Italian kitchen.
|
||||
Summarize the benefits of exercise using examples from walking through Italian towns.
|
||||
@@ -0,0 +1,151 @@
|
||||
# Directional Steering
|
||||
|
||||
Directional steering is a runtime activation edit for DS4. A steering file is a
|
||||
flat `f32` matrix with one normalized 4096-wide direction per layer. During
|
||||
inference, ds4 can apply the edit after attention outputs, FFN outputs, or both:
|
||||
|
||||
```text
|
||||
y = y - scale * direction[layer] * dot(direction[layer], y)
|
||||
```
|
||||
|
||||
Positive scale removes the represented direction. Negative scale amplifies it.
|
||||
With no steering file or zero scales, ds4 follows the normal inference path.
|
||||
|
||||
## Runtime Options
|
||||
|
||||
```text
|
||||
--dir-steering-file FILE load a 43 x 4096 f32 direction file
|
||||
--dir-steering-ffn F apply steering after FFN outputs; default is 1 when a file is provided
|
||||
--dir-steering-attn F apply steering after attention outputs; default is 0
|
||||
```
|
||||
|
||||
The FFN output is usually the best first target because it is late enough in
|
||||
each layer to represent behavior, style, and topic signals. Attention steering
|
||||
is available for experiments, but it can be more fragile.
|
||||
|
||||
## Verbosity Example
|
||||
|
||||
The bundled example builds a style direction from 100 paired prompts. Each pair
|
||||
asks for the same information in two ways:
|
||||
|
||||
- `examples/succinct.txt`: terse target prompts.
|
||||
- `examples/verbose.txt`: detailed contrast prompts.
|
||||
|
||||
Because the extracted direction is `succinct - verbose`, negative FFN scales
|
||||
make answers shorter, while positive FFN scales tend to make answers longer and
|
||||
more explanatory.
|
||||
|
||||
Build the vector:
|
||||
|
||||
```sh
|
||||
python3 dir-steering/tools/build_direction.py \
|
||||
--ds4 ./ds4 \
|
||||
--model ds4flash.gguf \
|
||||
--good-file dir-steering/examples/succinct.txt \
|
||||
--bad-file dir-steering/examples/verbose.txt \
|
||||
--out dir-steering/out/verbosity.json \
|
||||
--component ffn_out \
|
||||
--ctx 512
|
||||
```
|
||||
|
||||
This writes:
|
||||
|
||||
```text
|
||||
dir-steering/out/verbosity.json
|
||||
dir-steering/out/verbosity.f32
|
||||
```
|
||||
|
||||
Try a terse run:
|
||||
|
||||
```sh
|
||||
./ds4 -m ds4flash.gguf --nothink --temp 0 -n 160 \
|
||||
--dir-steering-file dir-steering/out/verbosity.f32 \
|
||||
--dir-steering-ffn -1 \
|
||||
-p "Explain why databases use indexes."
|
||||
```
|
||||
|
||||
Try a verbose run:
|
||||
|
||||
```sh
|
||||
./ds4 -m ds4flash.gguf --nothink --temp 0 -n 220 \
|
||||
--dir-steering-file dir-steering/out/verbosity.f32 \
|
||||
--dir-steering-ffn 2 \
|
||||
-p "Explain why databases use indexes."
|
||||
```
|
||||
|
||||
The same vector can be used in either direction. The sign is the important part:
|
||||
|
||||
- negative scale amplifies the succinct target direction;
|
||||
- positive scale suppresses that direction and usually gives the model more room
|
||||
to elaborate.
|
||||
|
||||
## Evaluating Scales
|
||||
|
||||
Use the sweep helper to test several strengths on a fixed prompt set:
|
||||
|
||||
```sh
|
||||
python3 dir-steering/tools/run_sweep.py \
|
||||
--ds4 ./ds4 \
|
||||
--model ds4flash.gguf \
|
||||
--direction dir-steering/out/verbosity.f32 \
|
||||
--prompts dir-steering/examples/eval_prompts.txt \
|
||||
--scales "-1,-0.5,0,0.5,1,2" \
|
||||
--tokens 180 \
|
||||
--nothink
|
||||
```
|
||||
|
||||
Start with FFN scales between `-1` and `2`. If the model becomes repetitive,
|
||||
ignores the prompt, or starts losing factual content, the scale is too strong.
|
||||
For this example, `-1` is a good first terse setting and `2` is a good first
|
||||
verbose setting. Strong negative scales such as `-2` or `-3` can over-amplify
|
||||
the terse direction and collapse into repetition on some prompts.
|
||||
|
||||
## Observed Effect
|
||||
|
||||
With the 100-pair vector built from the commands above, local greedy checks
|
||||
showed the expected behavior:
|
||||
|
||||
- Prompt: `Explain why databases use indexes.`
|
||||
- `--dir-steering-ffn -1`: 67 words, one compact paragraph.
|
||||
- `--dir-steering-ffn 0`: 136 words, structured explanation.
|
||||
- `--dir-steering-ffn 1`: 140 words, structured explanation with more detail.
|
||||
|
||||
On a prompt that the unsteered model already answered briefly, positive steering
|
||||
made the expansion more visible:
|
||||
|
||||
- Prompt: `What does DNS do?`
|
||||
- `--dir-steering-ffn 0`: 44 words.
|
||||
- `--dir-steering-ffn 2`: 171 words, with sections and step-by-step detail.
|
||||
|
||||
## Building Other Directions
|
||||
|
||||
The extractor compares two prompt sets:
|
||||
|
||||
- `good-file`: target prompts for the direction you want to represent.
|
||||
- `bad-file`: contrast prompts that should be separated from the target.
|
||||
|
||||
It captures DS4 activations from the same local GPU graph used for inference,
|
||||
averages target minus contrast, normalizes one vector per layer, and writes both
|
||||
metadata JSON and the runtime `.f32` file.
|
||||
|
||||
Concept removal:
|
||||
|
||||
1. Put concept-heavy prompts in `good-file`.
|
||||
2. Put neutral prompts in `bad-file`.
|
||||
3. Run with a positive FFN scale.
|
||||
|
||||
Concept amplification:
|
||||
|
||||
1. Put desired concept prompts in `good-file`.
|
||||
2. Put neutral prompts in `bad-file`.
|
||||
3. Run with a negative FFN scale.
|
||||
|
||||
Style control:
|
||||
|
||||
1. Put prompts for the target style in `good-file`.
|
||||
2. Put contrasting style prompts in `bad-file`.
|
||||
3. Use negative scale to amplify the target style, positive scale to reduce it.
|
||||
|
||||
The method is not a fine-tune. It is a low-rank runtime edit, so it works best
|
||||
for coarse behavior, topic, or style directions that are consistently present in
|
||||
the activation captures.
|
||||
@@ -0,0 +1,5 @@
|
||||
Explain why databases use indexes.
|
||||
How should I learn sorting algorithms?
|
||||
Describe how clouds form.
|
||||
Give me advice for debugging a slow program.
|
||||
What is public key cryptography?
|
||||
@@ -0,0 +1,100 @@
|
||||
Answer in one short sentence: what is binary search?
|
||||
In two concise bullets, explain why databases use indexes.
|
||||
Give a terse definition of TCP congestion control.
|
||||
Summarize recursion in exactly one sentence.
|
||||
Briefly explain what a cache does.
|
||||
Give a minimal explanation of how rain forms.
|
||||
In one compact paragraph, describe photosynthesis.
|
||||
Answer briefly: why should passwords be unique?
|
||||
Give a concise explanation of unit tests.
|
||||
In one sentence, explain what DNS does.
|
||||
Briefly explain why exercise improves health.
|
||||
Give a short answer: what is a hash table?
|
||||
Explain gradient descent in two crisp sentences.
|
||||
Define inflation in one sentence.
|
||||
Give a terse summary of how vaccines work.
|
||||
Answer briefly: what is an API?
|
||||
In one short paragraph, explain how compilers work.
|
||||
Give a compact explanation of merge sort.
|
||||
Briefly explain why backups matter.
|
||||
Define latency in one concise sentence.
|
||||
Give a minimal explanation of public key cryptography.
|
||||
In two short bullets, explain what a load balancer does.
|
||||
Briefly describe how solar panels generate electricity.
|
||||
Give a one-sentence explanation of supply and demand.
|
||||
Answer tersely: what is containerization?
|
||||
In one compact paragraph, explain the water cycle.
|
||||
Give a short definition of entropy.
|
||||
Briefly explain how a recommendation system works.
|
||||
Give a concise answer: why do teams use version control?
|
||||
Explain neural networks in two short sentences.
|
||||
Answer in one sentence: what is a database transaction?
|
||||
Give a terse explanation of event loops.
|
||||
Briefly explain how airplanes stay aloft.
|
||||
Give a compact summary of why sleep matters.
|
||||
In two concise bullets, explain what OAuth is for.
|
||||
Define bandwidth in one short sentence.
|
||||
Answer briefly: how does a thermostat work?
|
||||
Give a minimal explanation of memoization.
|
||||
Briefly explain why code reviews are useful.
|
||||
In one short paragraph, describe how markets set prices.
|
||||
Give a terse answer: what is a microservice?
|
||||
Explain backpropagation in two compact sentences.
|
||||
Briefly describe how a search engine ranks pages.
|
||||
Give a one-sentence explanation of opportunity cost.
|
||||
Answer concisely: what is rate limiting?
|
||||
In two short bullets, explain why logging is useful.
|
||||
Give a compact explanation of climate change.
|
||||
Briefly explain how checksums detect errors.
|
||||
Define normalization in databases in one sentence.
|
||||
Give a terse explanation of garbage collection.
|
||||
Answer briefly: why does encryption need keys?
|
||||
In one compact paragraph, explain how GPS finds location.
|
||||
Give a short explanation of continuous integration.
|
||||
Briefly describe how a queue differs from a stack.
|
||||
Define overfitting in one concise sentence.
|
||||
Give a minimal explanation of DNS caching.
|
||||
Answer tersely: what is a kernel?
|
||||
In two concise bullets, explain how to prioritize tasks.
|
||||
Briefly explain why indexes can slow writes.
|
||||
Give a compact explanation of HTTP.
|
||||
In one sentence, explain what a proxy server does.
|
||||
Answer briefly: how does compression save space?
|
||||
Give a terse explanation of database replication.
|
||||
Briefly explain how a compiler differs from an interpreter.
|
||||
Define amortized cost in one short sentence.
|
||||
Give a minimal explanation of a Bloom filter.
|
||||
Answer concisely: why do batteries degrade?
|
||||
In one compact paragraph, explain how elections use runoff voting.
|
||||
Give a short explanation of semantic versioning.
|
||||
Briefly describe how a CDN helps a website.
|
||||
Define idempotency in one concise sentence.
|
||||
Answer tersely: what is a mutex?
|
||||
In two short bullets, explain why monitoring matters.
|
||||
Give a compact explanation of how email routing works.
|
||||
Briefly explain how a neural embedding represents text.
|
||||
Give a one-sentence explanation of dependency injection.
|
||||
Answer briefly: what is a race condition?
|
||||
In one compact paragraph, explain why documentation helps teams.
|
||||
Give a terse description of how a blockchain works.
|
||||
Briefly explain what a memory leak is.
|
||||
Define consensus in distributed systems in one sentence.
|
||||
Give a minimal explanation of a priority queue.
|
||||
Answer concisely: why is caching hard to invalidate?
|
||||
In two concise bullets, explain how to debug a slow program.
|
||||
Briefly explain what a firewall does.
|
||||
Give a compact explanation of how interest rates affect borrowing.
|
||||
In one sentence, explain what an operating system does.
|
||||
Answer briefly: how does a web browser render a page?
|
||||
Give a terse explanation of feature flags.
|
||||
Briefly explain how garbage collection pauses happen.
|
||||
Define eventual consistency in one concise sentence.
|
||||
Give a minimal explanation of sharding.
|
||||
Answer concisely: what is a deadlock?
|
||||
In one compact paragraph, explain how a camera sensor captures light.
|
||||
Give a short explanation of why indexes need selectivity.
|
||||
Briefly describe what a scheduler does.
|
||||
Define precision and recall in one sentence.
|
||||
Give a terse explanation of streaming APIs.
|
||||
Answer briefly: why do projects need tests?
|
||||
In two short bullets, explain how to choose a data structure.
|
||||
@@ -0,0 +1,100 @@
|
||||
Write a detailed, multi-paragraph explanation of what binary search is, including intuition, preconditions, and a small example.
|
||||
Explain why databases use indexes in a thorough answer with several concrete tradeoffs and examples.
|
||||
Give an expansive explanation of TCP congestion control, covering its goal, feedback signals, and why it changes speed over time.
|
||||
Explain recursion carefully with a step-by-step description, a concrete analogy, and a note about base cases.
|
||||
Describe caches in detail, including what they store, why they help, and what can go wrong.
|
||||
Explain how rain forms in a rich answer covering evaporation, condensation, clouds, and falling droplets.
|
||||
Describe photosynthesis in depth, including inputs, outputs, energy conversion, and why plants depend on it.
|
||||
Explain why passwords should be unique with a careful discussion of breaches, reuse, and practical safety habits.
|
||||
Explain unit tests in detail, including what they verify, how they fit into development, and their limitations.
|
||||
Describe DNS thoroughly, including names, resolvers, records, caching, and why it matters for the web.
|
||||
Explain why exercise improves health in a detailed answer covering heart, muscles, mood, sleep, and long-term effects.
|
||||
Give a thorough explanation of hash tables, including hashing, buckets, collisions, and average-case performance.
|
||||
Explain gradient descent in depth, including the loss function, gradients, step size, and how iteration improves a model.
|
||||
Describe inflation carefully, including price levels, purchasing power, possible causes, and everyday consequences.
|
||||
Explain how vaccines work in detail, including immune memory, antigens, protection, and population effects.
|
||||
Describe what an API is with examples, explaining contracts, inputs, outputs, stability, and how developers use APIs.
|
||||
Explain how compilers work in a detailed sequence from parsing through optimization to code generation.
|
||||
Describe merge sort thoroughly, including divide and conquer, merging, complexity, and when it is useful.
|
||||
Explain why backups matter in depth, including accidents, hardware failure, ransomware, recovery time, and testing restores.
|
||||
Define latency with a detailed discussion of delay, network hops, queues, tail latency, and user-visible impact.
|
||||
Explain public key cryptography thoroughly, including key pairs, encryption, signatures, trust, and common uses.
|
||||
Describe load balancers in detail, including traffic distribution, health checks, failover, and scaling.
|
||||
Explain how solar panels generate electricity with a detailed description of photons, cells, current, and inverters.
|
||||
Explain supply and demand thoroughly, including curves, equilibrium, shortages, surpluses, and real-world examples.
|
||||
Describe containerization in depth, including images, isolation, portability, orchestration, and operational benefits.
|
||||
Explain the water cycle in a detailed answer covering evaporation, condensation, precipitation, runoff, and groundwater.
|
||||
Define entropy in a broad explanation covering disorder, information, uncertainty, and why context changes the meaning.
|
||||
Explain how recommendation systems work in detail, including signals, user history, item similarity, ranking, and feedback loops.
|
||||
Explain why teams use version control in depth, covering history, collaboration, branching, review, and recovery.
|
||||
Describe neural networks thoroughly, including layers, weights, activations, training, and what they approximate.
|
||||
Explain database transactions in detail, including atomicity, consistency, isolation, durability, and examples.
|
||||
Describe event loops thoroughly, including queues, callbacks, nonblocking I/O, and why they matter in servers.
|
||||
Explain how airplanes stay aloft in depth, covering lift, wings, airflow, thrust, drag, and control surfaces.
|
||||
Explain why sleep matters with a detailed discussion of memory, hormones, immune function, mood, and recovery.
|
||||
Explain OAuth in detail, including delegated authorization, access tokens, scopes, consent, and common flows.
|
||||
Define bandwidth in a detailed answer covering capacity, throughput, bottlenecks, and how it differs from latency.
|
||||
Explain how a thermostat works in detail, including sensors, set points, feedback loops, and heating or cooling control.
|
||||
Describe memoization thoroughly, including caching function results, when it helps, and when it can waste memory.
|
||||
Explain why code reviews are useful in depth, including correctness, maintainability, knowledge sharing, and team standards.
|
||||
Describe how markets set prices in detail, including buyers, sellers, incentives, information, scarcity, and competition.
|
||||
Explain microservices thoroughly, including service boundaries, independent deployment, communication, and operational costs.
|
||||
Explain backpropagation in depth, including forward passes, gradients, the chain rule, and weight updates.
|
||||
Describe how a search engine ranks pages in detail, including crawling, indexing, relevance, links, freshness, and quality signals.
|
||||
Explain opportunity cost thoroughly, including tradeoffs, alternatives, hidden costs, and everyday decision examples.
|
||||
Describe rate limiting in detail, including quotas, windows, tokens, fairness, abuse prevention, and user experience.
|
||||
Explain why logging is useful in depth, covering debugging, audits, operations, metrics, and incident response.
|
||||
Explain climate change carefully, including greenhouse gases, warming mechanisms, impacts, uncertainty, and mitigation.
|
||||
Describe how checksums detect errors in detail, including summaries, corruption, collision risk, and practical uses.
|
||||
Explain database normalization thoroughly, including redundancy, anomalies, normal forms, and tradeoffs.
|
||||
Describe garbage collection in detail, including allocation, reachability, tracing, freeing memory, and pauses.
|
||||
Explain why encryption needs keys in depth, covering secrecy, algorithms, key exchange, storage, and compromise.
|
||||
Describe how GPS finds location thoroughly, including satellites, timing, trilateration, clocks, and error correction.
|
||||
Explain continuous integration in detail, including automated builds, tests, integration frequency, and feedback.
|
||||
Describe queues and stacks thoroughly, including ordering rules, operations, use cases, and examples.
|
||||
Explain overfitting in depth, including training error, generalization, model complexity, validation, and prevention.
|
||||
Describe DNS caching thoroughly, including TTLs, resolvers, stale entries, performance, and invalidation problems.
|
||||
Explain what a kernel is in detail, including hardware mediation, processes, memory, drivers, and system calls.
|
||||
Explain how to prioritize tasks in depth, covering urgency, impact, dependencies, deadlines, and opportunity cost.
|
||||
Explain why indexes can slow writes in detail, including maintenance, extra storage, page splits, and transaction overhead.
|
||||
Describe HTTP thoroughly, including requests, responses, methods, headers, status codes, and statelessness.
|
||||
Explain proxy servers in detail, including forwarding, filtering, caching, privacy, reverse proxies, and load balancing.
|
||||
Describe how compression saves space in depth, including redundancy, dictionaries, entropy coding, and lossless versus lossy tradeoffs.
|
||||
Explain database replication thoroughly, including primary and replicas, lag, failover, consistency, and read scaling.
|
||||
Describe how compilers differ from interpreters in depth, including translation timing, runtime behavior, optimization, and examples.
|
||||
Explain amortized cost thoroughly, including occasional expensive operations, averaged guarantees, and data structure examples.
|
||||
Describe Bloom filters in detail, including hashing, bit arrays, false positives, memory efficiency, and use cases.
|
||||
Explain why batteries degrade in depth, including chemistry, charge cycles, heat, fast charging, and capacity loss.
|
||||
Explain runoff voting thoroughly, including ballot ranking, elimination rounds, majority goals, and possible strategic effects.
|
||||
Describe semantic versioning in detail, including major, minor, and patch numbers, compatibility promises, and release examples.
|
||||
Explain how a CDN helps a website in depth, including edge caches, latency, bandwidth, resilience, and invalidation.
|
||||
Define idempotency thoroughly, including repeated operations, safety, HTTP examples, retries, and distributed systems relevance.
|
||||
Explain mutexes in detail, including mutual exclusion, critical sections, contention, deadlocks, and alternatives.
|
||||
Explain why monitoring matters thoroughly, covering metrics, logs, alerts, trends, incidents, and service reliability.
|
||||
Describe how email routing works in detail, including DNS records, SMTP servers, queues, spam checks, and delivery.
|
||||
Explain how neural embeddings represent text in depth, including vectors, semantic similarity, training signals, and retrieval uses.
|
||||
Describe dependency injection thoroughly, including object construction, decoupling, testing, configuration, and tradeoffs.
|
||||
Explain race conditions in detail, including shared state, timing, nondeterminism, examples, and prevention techniques.
|
||||
Explain why documentation helps teams in depth, covering onboarding, decisions, maintenance, coordination, and future debugging.
|
||||
Describe how a blockchain works thoroughly, including blocks, hashes, consensus, append-only history, and tradeoffs.
|
||||
Explain memory leaks in detail, including unreachable allocations, retained references, symptoms, and detection.
|
||||
Define consensus in distributed systems thoroughly, including agreement, failures, quorums, leaders, and consistency.
|
||||
Describe priority queues in depth, including priorities, heap implementations, operations, and scheduling examples.
|
||||
Explain why caching is hard to invalidate in detail, including stale data, dependencies, timing, consistency, and user impact.
|
||||
Explain how to debug a slow program thoroughly, covering measurement, profiling, hypotheses, bottlenecks, and regression checks.
|
||||
Describe firewalls in detail, including packet filtering, rules, ports, network boundaries, and limitations.
|
||||
Explain how interest rates affect borrowing in depth, including monthly payments, risk, central banks, and business investment.
|
||||
Describe what an operating system does thoroughly, including process scheduling, memory, files, devices, and permissions.
|
||||
Explain how a web browser renders a page in detail, including parsing, CSS, layout, painting, JavaScript, and compositing.
|
||||
Describe feature flags thoroughly, including runtime switches, gradual rollout, experiments, rollback, and cleanup.
|
||||
Explain how garbage collection pauses happen in detail, including stop-the-world phases, heap scanning, compaction, and latency.
|
||||
Define eventual consistency thoroughly, including replicas, propagation delay, conflicts, convergence, and application tradeoffs.
|
||||
Describe sharding in depth, including partition keys, distribution, rebalancing, hotspots, and cross-shard queries.
|
||||
Explain deadlocks in detail, including locks, waiting cycles, conditions, detection, and prevention strategies.
|
||||
Describe how a camera sensor captures light in depth, including pixels, photons, charge, color filters, exposure, and noise.
|
||||
Explain why indexes need selectivity thoroughly, including cardinality, query planners, scans, and cases where indexes are ignored.
|
||||
Describe what a scheduler does in detail, including tasks, priorities, fairness, deadlines, and resource allocation.
|
||||
Define precision and recall thoroughly, including true positives, false positives, false negatives, and when each metric matters.
|
||||
Explain streaming APIs in depth, including incremental delivery, backpressure, connections, errors, and client handling.
|
||||
Explain why projects need tests thoroughly, including regression prevention, design feedback, confidence, automation, and documentation.
|
||||
Describe how to choose a data structure in detail, including operations, constraints, complexity, memory, and real-world tradeoffs.
|
||||
Binary file not shown.
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"format": "ds4-directional-steering-v1",
|
||||
"shape": [
|
||||
43,
|
||||
4096
|
||||
],
|
||||
"component": "ffn_out",
|
||||
"thinking": false,
|
||||
"pair_normalize": false,
|
||||
"orthogonalize_control_mean": true,
|
||||
"good_file": "dir-steering/examples/succinct.txt",
|
||||
"bad_file": "dir-steering/examples/verbose.txt",
|
||||
"model": "/Users/antirez/hack/2026/ds4.c/gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2.gguf",
|
||||
"note": "runtime positive scale suppresses this direction; negative scale amplifies it"
|
||||
}
|
||||
@@ -129,7 +129,7 @@ def main() -> None:
|
||||
help="desired/target prompts, one per line")
|
||||
ap.add_argument("--bad-file", required=True,
|
||||
help="contrast/control prompts, one per line")
|
||||
ap.add_argument("--out", default="dir-stearing/out/direction.json",
|
||||
ap.add_argument("--out", default="dir-steering/out/direction.json",
|
||||
help="metadata JSON path; .f32 is written next to it")
|
||||
ap.add_argument("--ctx", type=int, default=512)
|
||||
ap.add_argument("--system", default="You are a helpful assistant.")
|
||||
Reference in New Issue
Block a user