StreamingSession

class StreamingSession(val model: Transformer, val tokenizer: Tokenizer)(source)

Constructors

Link copied to clipboard
constructor(model: Transformer, tokenizer: Tokenizer)

Properties

Link copied to clipboard
Link copied to clipboard

Functions

Link copied to clipboard
fun cancel()
Link copied to clipboard
fun stream(prompt: String, maxTokens: Int, stopTokens: Set<Int> = emptySet(), onUpdate: (GenerationState) -> Unit): GenerationState
fun stream(prompt: String, maxTokens: Int, stopTokens: Set<Int>, sampling: Sampling, onUpdate: (GenerationState) -> Unit): GenerationState