← Work index
CASE 05 / Research notebookEXPERIMENT

Compute & energy

Following the workload down the stack

A numerical study of a local Qwen model slice, readout precision, and the small energy opportunity in a sampling capacitor once peripheral costs enter the comparison.

CONTEXT
Ongoing investigation
TOOLS / METHODS
Python · NumPy · GGUF · Local models · Numerical modeling
THE CENTRAL LESSON

The initially targeted sampling node was too small a term to justify a system-level efficiency claim against the separate peripheral estimate. That negative screen moved the question toward readout topology, amplifier overhead, and equal-deadline comparisons.

2 evidence recordsRead the scope & limits ↓
THE QUESTION

Which efficiency improvements survive accounting for the rest of the system?

Context

Working with a local model makes its representation and memory footprint tangible. This investigation used a Qwen2.5-Coder-7B GGUF to study a narrow weighted-sum workload: the first 64 outputs of the first query projection, before positional rotation. It is a numerical and analytical study, not a fabricated device or a full-model benchmark.

What I did

  1. 01

    Mapped the projection onto 56 tiles, each with 64 inputs and 64 outputs.

  2. 02

    Compared matched low-precision digital arithmetic with modeled clipping and digitization at different readout precisions.

  3. 03

    Separated saved projection-error results from circuit estimates, ideal scheduling arithmetic, and proposed hardware measurements.

  4. 04

    Screened the maximum charge/reset energy at a 4 pF sampling node before treating energy recovery as a worthwhile architectural target.

Architecture / workflow

COMPUTE / SYSTEM STUDYSAVED SIMULATION + ANALYSIS

A narrow experiment. A deliberately narrow conclusion.

Does the result survive a wider accounting boundary?

  1. 01
    INFORMATION / ARTIFACT

    Select a model slice

    First 64 outputs of the first Qwen query projection

  2. 02
    SOFTWARE LOGIC

    Map the arithmetic

    56 tiles, each with 64 inputs and 64 outputs

  3. 03
    SOFTWARE LOGIC

    Model the readout

    Clipping and finite readout precision

    SIMULATED / MODELED
  4. 04
    INFORMATION / ARTIFACT

    Compare errors

    Saved projection errors against dequantized weights

  5. 05
    HUMAN JUDGMENT

    Screen the energy

    Separate analytical bounds and circuit estimates

INSPECT THE FAILURE BOUNDARYA smaller component cost is mistaken for a complete-system efficiency gain.
WHAT FOLLOWS

Drive, routing, idle power, wake-up and reduction still need accounting. Projection error does not measure language-model quality, throughput, or silicon energy.

The drawing joins related research questions, not one measured hardware pipeline. Simulation results and energy estimates have separate scopes. Original experiments were not rerun for this site.

Where it gets difficult

More readout bits do not fix clipping. In the saved six-bit-weight experiment, the infinite-precision clipping diagnostic still has about 4.99% relative projection error.

Sharing readout circuits changes concurrency and latency. A comparison needs the same workload and a defensible scheduling boundary.

An active-tile energy subtotal leaves out other costs. Input drive, routing, weight storage, wake-up, and reduction can change the conclusion.

FIG. 02 PROJECTION ERROR / SAVED SIMULATION

More bits do not remove clipping.

Relative RMS error (%) · six-bit weights · no added programming error

Matched digital
2.26%
4-bit readout
11.94%
6-bit readout
5.59%
8-bit readout
5.03%
Clipping only
4.99%
First 64 outputs of Qwen2.5-Coder-7B’s first query projection. 462 evaluation-token occurrences. These are numerical projection errors, not model-quality scores or hardware measurements. Download source data ↓
NOTE 01 AN OPPORTUNITY TOO SMALL?

The sampling-node screen

Hypothesis
Recovering energy at the sampling capacitors might materially lower the readout cost.
Model
A conventional zero-to-V-to-zero resistive charge/reset cycle dissipates CV² per capacitor. For 64 channels at 4 pF and 2 V:
64 × 4 pF × (2 V)² = 1.024 nJ
Result
Even the full-amplitude charge/reset bound is only 0.068% of the separate 1.51 µJ active-tile estimate. Only half of CV² is stored before reset; a recovery circuit would have its own losses.
Interpretation
The target is a small term against that estimate. This screens an idea; it does not establish measured savings or compare complete accelerators.
Next question
Does a different peripheral circuit or readout schedule change the dominant cost?
Inspect the arithmetic and assumptions ↓

Equal readout time is only a starting point.

The saved architecture screen compares 56 resident weight banks at an ideal readout time of 224 µs. Its rounded active-tile subtotals are 144.48 µJ for the dedicated schedule and 84.56 µJ for the shared schedule.

The 59.92 µJ difference is an allowance for omitted incremental costs, not demonstrated system savings. Input drive, routing, idle power, wake-up, and final reduction still need accounting. The latency is an illustrative schedule, not a measured application deadline.

THE BOUNDARY OF THE CLAIM

Limits & evidence

  • The projection study evaluates 462 token occurrences after calibration on 472 occurrences. Prompts are short code tasks; token IDs can repeat across the split.
  • The comparison is against dequantized shipped model weights, not an original unquantized checkpoint. It does not measure next-token quality, task performance, training, or tokens per second.
  • Energy figures are analytical estimates or bounds. No silicon energy, complete accelerator energy, or physical hardware result is claimed.
Saved simulation

Projection-error results

A sanitized excerpt of the saved numerical results supplies the projection-error values shown in the notebook. Published values were compared with the original CSV.

What this does not establish

A narrow model slice, not end-to-end model quality or physical hardware. The original experiment was not rerun for this site.

Download projection data · CSV
Analytical note

Sampling-node and schedule arithmetic

Explicit units and assumptions reproduce the sampling-node charge/reset bound and compare rounded active-tile subtotals at the same ideal readout time.

What this does not establish

These are analytical bounds and reported estimates. Omitted drive, routing, idle, wake-up and reduction costs prevent a system-level savings claim. No hardware measurement is implied.

Download calculation note · TXT

What this changed

The initially targeted sampling node was too small a term to justify a system-level efficiency claim against the separate peripheral estimate. That negative screen moved the question toward readout topology, amplifier overhead, and equal-deadline comparisons.

WHERE THIS LEADS

Can lower-power peripheral circuitry change the ranking, and which single physical measurement would most reduce uncertainty?

NEXT CASEDynamo Air