TransformerBlock

class TransformerBlock(val dimension: Int, val numHeads: Int, val hiddenDimension: Int = 4 * dimension, val numKeyValueHeads: Int = numHeads, val normEps: Float = 1.0E-6f, val ropeTheta: Float = 10000.0f, ropeScaling: RoPEScaling? = null, quantization: Quantization? = null) : Module(source)

Constructors

Link copied to clipboard
constructor(dimension: Int, numHeads: Int, hiddenDimension: Int = 4 * dimension, numKeyValueHeads: Int = numHeads, normEps: Float = 1.0E-6f, ropeTheta: Float = 10000.0f, ropeScaling: RoPEScaling? = null, quantization: Quantization? = null)

Properties

Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
val ffn: SwiGLU
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard

Functions

Link copied to clipboard
fun decodeInto(x: Tensor, cache: KVCache, layer: Int, buffers: InferenceBuffers): Tensor
Link copied to clipboard
open override fun forward(input: Tensor): Tensor
fun forward(input: Tensor, sequenceLength: Int): Tensor
fun forward(input: Tensor, cache: KVCache, layer: Int): Tensor
Link copied to clipboard
open fun forwardInto(input: Tensor, out: Tensor): Tensor
Link copied to clipboard
operator fun invoke(input: Tensor): Tensor
Link copied to clipboard
open override fun parameters(): List<Tensor>