llm
estableLLM prompt construction utilities for building ChatML, Llama, Alpaca, and Gemma prompts, estimating token counts, splitting text into chunks, and sanitizing inputs.
use plugin llm::{estimate_tokens, estimate_messages_tokens, build_chatml, …} Funciones (23)
- estimate_tokens Estimate token count of a string
- estimate_messages_tokens Estimate tokens for a messages array
- build_chatml Format messages as ChatML prompt
- build_llama_prompt Format system+user as Llama 2 prompt
- build_alpaca_prompt Format instruction as Alpaca prompt
- build_gemma_prompt Format messages as Gemma prompt
- truncate_to_tokens Truncate text to a token limit
- split_chunks Split text into overlapping chunks
- template Fill a template string with variables
- sanitize Strip prompt injection special tokens
- extract_role_messages Filter messages by role
- count_messages Count messages in an array
- PromptBuilder.new Create a stateful prompt builder
- PromptBuilder.system Add a system message
- PromptBuilder.user Add a user message
- PromptBuilder.assistant Add an assistant message
- PromptBuilder.build_chatml Build ChatML string from messages
- PromptBuilder.build_llama Build Llama 2 string from messages
- PromptBuilder.build_gemma Build Gemma string from messages
- PromptBuilder.build_messages Export messages as a table
- PromptBuilder.message_count Count messages in the builder
- PromptBuilder.token_estimate Estimate total token count
- PromptBuilder.clear Clear all messages
Estimate token count of a string
Estimates the number of tokens in text using the approximation of 4 characters per token. Useful for staying within model context limits before sending a request.
use plugin llm::{estimate_tokens}
let count = estimate_tokens("Hello, how are you today?")
print("Estimated tokens: {count}")Estimate tokens for a messages array
Estimates the total token count for a messages array, adding 4 tokens of overhead per message for role/formatting tokens.
use plugin llm::{estimate_messages_tokens}
let msgs = [
#{"role": "system", "content": "You are a helpful assistant."},
#{"role": "user", "content": "What is 2+2?"}
]
print("Total tokens: {estimate_messages_tokens(msgs)}")Format messages as ChatML prompt
Formats a messages array into ChatML format used by models like Mistral and Qwen. Each message must be a table with role and content fields. Appends the assistant turn header at the end.
use plugin llm::{build_chatml}
let msgs = [
#{"role": "system", "content": "You are helpful."},
#{"role": "user", "content": "Tell me a joke."}
]
let prompt = build_chatml(msgs)
print(prompt)Format system+user as Llama 2 prompt
Formats a system message and user message into the Llama 2 [INST] prompt format.
use plugin llm::{build_llama_prompt}
let prompt = build_llama_prompt("You are a helpful assistant.", "What is the capital of France?")
print(prompt)Format instruction as Alpaca prompt
Formats an instruction (and optional input context) into the Alpaca prompt format used for fine-tuned instruction models.
use plugin llm::{build_alpaca_prompt}
let prompt = build_alpaca_prompt("Summarize the following text.", "The quick brown fox...")
print(prompt)
let no_input = build_alpaca_prompt("Write a haiku about rain.", "")Format messages as Gemma prompt
Formats a messages array into the Gemma <start_of_turn> prompt format.
use plugin llm::{build_gemma_prompt}
let msgs = [
#{"role": "user", "content": "Explain recursion."}
]
let prompt = build_gemma_prompt(msgs)Truncate text to a token limit
Truncates text to approximately max_tokens tokens (using 4 chars/token). Returns the original string if it is already within the limit.
use plugin llm::{truncate_to_tokens, estimate_tokens}
let long_text = "Lorem ipsum..."
let short = truncate_to_tokens(long_text, 100)
print("Truncated to ~{estimate_tokens(short)} tokens")Split text into overlapping chunks
Splits text into chunks of approximately chunk_size tokens, with an optional overlap in tokens between consecutive chunks. Returns an array of strings. Useful for RAG chunking pipelines.
use plugin llm::{split_chunks}
let text = "Long document text here..."
let chunks = split_chunks(text, 200, 20)
print("Chunks: {chunks}")Fill a template string with variables
Fills {{key}} placeholders in template with values from the vars table. All keys and values must be strings.
use plugin llm::{template}
let prompt = template("Hello, {{name}}! You are a {{role}}.", #{"name": "Alice", "role": "developer"})
print(prompt)Strip prompt injection special tokens
Strips common prompt injection special tokens from text, including <|im_start|>, [INST], <<SYS>>, <start_of_turn>, and Alpaca section headers. Use on untrusted user input before including it in a prompt.
use plugin llm::{sanitize, build_llama_prompt}
let user_input = sanitize(raw_user_input)
let prompt = build_llama_prompt("Be helpful.", user_input)Filter messages by role
Returns only the content strings from messages whose role matches the given value (e.g. "user", "assistant", "system").
use plugin llm::{extract_role_messages}
let msgs = [
#{"role": "user", "content": "Hi"},
#{"role": "assistant", "content": "Hello!"},
#{"role": "user", "content": "How are you?"}
]
let user_msgs = extract_role_messages(msgs, "user")Count messages in an array
Returns the number of entries in a messages array.
use plugin llm::{count_messages}
let n = count_messages([
#{"role": "user", "content": "Hello"}
])
print("Messages: {n}")Create a stateful prompt builder
Creates a new stateful prompt builder. Use the builder's methods to append messages incrementally, then call one of the build_* methods to produce the final prompt string.
use plugin llm::{PromptBuilder}
let pb = PromptBuilder.new()
pb.system("You are a helpful assistant.")
pb.user("What is 2 + 2?")
let prompt = pb.build_chatml()
print(prompt)Add a system message
Appends a system role message to the builder.
pb.system("You are a concise assistant.")Add a user message
Appends a user role message to the builder.
pb.user("Explain monads in one sentence.")Add an assistant message
Appends an assistant role message. Useful for few-shot examples.
pb.assistant("Monads are a design pattern for sequencing computations.")Build ChatML string from messages
Serializes all accumulated messages into ChatML format.
use plugin llm::{PromptBuilder}
let pb = PromptBuilder.new()
pb.system("Be helpful.")
pb.user("Hi!")
let s = pb.build_chatml()Build Llama 2 string from messages
Serializes the first system and first user message from the builder into Llama 2 format.
let s = pb.build_llama()Build Gemma string from messages
Serializes all accumulated messages into Gemma turn format.
let s = pb.build_gemma()Export messages as a table
Exports all messages as a table of {role, content} tables, suitable for passing to API calls.
let msgs = pb.build_messages()Count messages in the builder
Returns the number of messages currently in the builder.
print("Messages: {pb.message_count()}")Estimate total token count
Estimates the total token count of all messages in the builder.
print("Estimated tokens: {pb.token_estimate()}")Clear all messages
Removes all messages from the builder, resetting it for reuse.
pb.clear()