ChatGPT on Sparse Data Representations

Query 2 — I’d like more information on how an SDR might be calculated from a collection of numeric values, keywords, etc.

ChatGPT said:

1. Core rules an SDR encoder must satisfy

  • Fixed dimensionality
  • Fixed sparsity
  • Semantic locality
  • Determinism
  • Union safety

2. Encoding scalar numeric values

2.1 Bucketed scalar encoder

  • Range: 0–100
  • SDR size: 2048
  • Active bits: 40
  • Window width: 40

2.2 Random distributed scalar encoder

  • Uses PRNG seeded by bucket index
  • Overlap proportional to value similarity

3. Encoding vectors / coordinates

3.1 Independent axis encoders + union

  • SDR(x, y) = SDR_x(x) OR SDR_y(y)

3.2 Grid-cell–inspired encoders

  • Multiple periodic encoders at different scales
  • Union across scales

4. Encoding keywords / symbols

4.1 Hash-based categorical encoder

  • Hash string into multiple bits

4.2 Semantic field encoding

  • “mug” close to “cup”
  • Union of related symbols

5. Encoding sets, lists, and structured data

5.1 Sets

  • OR multiple SDRs

5.2 Key-value pairs

  • Use binding (XOR, permutation, region-based)

6. Encoding time and sequences

  • Temporal pooling
  • Context windows
  • SDR_t = SDR(input_t) OR permute(SDR_{t-1})

7. A worked example

  • touch_force = 3.2
  • surface = “smooth”
  • location = (x=12.4, y=7.9)

SDR = SDR_force(3.2) OR SDR_surface(“smooth”) OR SDR_x(12.4) OR SDR_y(7.9)

8. Practical parameter ranges

  • SDR size: 1024–8192 bits
  • Active bits: 20–100
  • Union limit: ~10–20 items