Attention

class Attention(val dimension: Int, val numHeads: Int, val numKeyValueHeads: Int = numHeads, val ropeTheta: Float = 10000.0f, ropeScaling: RoPEScaling? = null, quantization: Quantization? = null) : Module(source)

Constructors

Link copied to clipboard
constructor(dimension: Int, numHeads: Int, numKeyValueHeads: Int = numHeads, ropeTheta: Float = 10000.0f, ropeScaling: RoPEScaling? = null, quantization: Quantization? = null)

Properties

Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
val rope: RoPE
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
var wK: Module
Link copied to clipboard
Link copied to clipboard
var wQ: Module
Link copied to clipboard
var wV: Module

Functions

Link copied to clipboard
fun decodeInto(input: Tensor, cache: KVCache, layer: Int, out: Tensor, buffers: InferenceBuffers): Tensor
Link copied to clipboard
open override fun forward(input: Tensor): Tensor
fun forward(input: Tensor, sequenceLength: Int): Tensor
fun forward(input: Tensor, cache: KVCache, layer: Int): Tensor
Link copied to clipboard
open fun forwardInto(input: Tensor, out: Tensor): Tensor
Link copied to clipboard
operator fun invoke(input: Tensor): Tensor
Link copied to clipboard
open override fun parameters(): List<Tensor>