SwiGLU

class SwiGLU(val dimension: Int, val hiddenDimension: Int, quantization: Quantization? = null) : Module(source)

Constructors

Link copied to clipboard
constructor(dimension: Int, hiddenDimension: Int, quantization: Quantization? = null)

Properties

Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
Link copied to clipboard
var wUp: Module

Functions

Link copied to clipboard
open override fun forward(input: Tensor): Tensor
Link copied to clipboard
open fun forwardInto(input: Tensor, out: Tensor): Tensor
fun forwardInto(input: Tensor, out: Tensor, gate: Tensor, up: Tensor): Tensor
Link copied to clipboard
operator fun invoke(input: Tensor): Tensor
Link copied to clipboard
open override fun parameters(): List<Tensor>