KVCache

class KVCache(val numLayers: Int, val numHeads: Int, val maxSequenceLength: Int, val headDim: Int)(source)

Constructors

Link copied to clipboard
constructor(numLayers: Int, numHeads: Int, maxSequenceLength: Int, headDim: Int)

Properties

Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
var length: Int
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard

Functions

Link copied to clipboard
fun advance(count: Int)
Link copied to clipboard
fun copyFrom(source: KVCache, length: Int)
Link copied to clipboard
fun offset(layer: Int, head: Int, position: Int): Int
Link copied to clipboard
fun reset()