llm

Class APIError

Base class for API-related errors.

Bases:

Attribute AssistantContent

Type: TypeAlias

Type alias for content that can fit into an AssistantMessage.

Attribute AssistantContentChunk

Type: TypeAlias

Chunks of assistant content that may be streamed as generated by the LLM.

Attribute AssistantContentPart

Type: TypeAlias

Content parts that can be included in an AssistantMessage.

Class AssistantMessage

An assistant message containing the model's response.

Class AsyncCall

An async call that directly generates LLM responses without requiring a model argument.

Created by decorating an async MessageTemplate with llm.call. The decorated async function becomes directly callable to generate responses asynchronously, with the Model bundled in.

An AsyncCall is essentially: async MessageTemplate + tools + format + Model. It can be invoked directly: await call(*args, **kwargs) (no model argument needed).

The model can be overridden at runtime using with llm.model(...) context manager.

Bases: BaseCall, Generic[P, FormattableT]

Function call

Generates a response using the LLM asynchronously.

Function stream

Generates a streaming response using the LLM asynchronously.

Attribute AsyncChunkIterator

Type: TypeAlias

Asynchronous iterator yielding chunks with raw data.

Class AsyncContextCall

An async context-aware call that directly generates LLM responses without requiring a model argument.

Created by decorating an async ContextMessageTemplate with llm.call. The decorated async function (with first parameter 'ctx' of type Context[DepsT]) becomes directly callable to generate responses asynchronously with context dependencies, with the Model bundled in.

An AsyncContextCall is essentially: async ContextMessageTemplate + tools + format + Model. It can be invoked directly: await call(ctx, *args, **kwargs) (no model argument needed).

The model can be overridden at runtime using with llm.model(...) context manager.

Bases: BaseCall, Generic[P, DepsT, FormattableT]

Function call

Generates a response using the LLM asynchronously.

Function stream

Generates a streaming response using the LLM asynchronously.

Class AsyncContextPrompt

An async context-aware prompt that can be called with a model to generate a response.

Created by decorating an async ContextMessageTemplate with llm.prompt. The decorated async function (with first parameter 'ctx' of type Context[DepsT]) becomes callable with a Model to generate LLM responses asynchronously with context dependencies.

An AsyncContextPrompt is essentially: async ContextMessageTemplate + tools + format. It can be invoked with a model: await prompt(model, ctx, *args, **kwargs).

Bases:

Function call

Generates a response using the provided model asynchronously.

Function stream

Generates a streaming response using the provided model asynchronously.

Class AsyncContextResponse

The response generated by an LLM from an async context call.

Bases: BaseResponse[AsyncContextToolkit[DepsT], FormattableT], Generic[DepsT, FormattableT]

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new AsyncContextResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class AsyncContextStreamResponse

An AsyncContextStreamResponse wraps response content from the LLM with a streaming interface.

This class supports iteration to process chunks as they arrive from the model.

Content can be streamed in one of three ways:

• Via .streams(), which provides an iterator of streams, where each stream contains chunks of streamed data. The chunks contain deltas (new content in that particular chunk), and the stream itself accumulates the collected state of all the chunks processed thus far.
• Via .chunk_stream() which allows iterating over Mirascope's provider- agnostic chunk representation.
• Via .pretty_stream() a helper method which provides all response content as str deltas. Iterating through pretty_stream will yield text content and optionally placeholder representations for other content types, but it will still consume the full stream.
• Via .structured_stream(), a helper method which provides partial structured outputs from a response (useful when FormatT is set). Iterating through structured_stream will only yield structured partials, but it will still consume the full stream.

As chunks are consumed, they are collected in-memory on the AsyncContextStreamResponse, and they become available in .content, .messages, .tool_calls, etc. All of the stream iterators can be restarted after the stream has been consumed, in which case they will yield chunks from memory in the original sequence that came from the LLM. If the stream is only partially consumed, a fresh iterator will first iterate through in-memory content, and then will continue consuming fresh chunks from the LLM.

In the specific case of text chunks, they are included in the response content as soon as they become available, via an llm.Text part that updates as more deltas come in. This enables the behavior where resuming a partially-streamed response will include as much text as the model generated.

For other chunks, like Thinking or ToolCall, they are only added to response content once the corresponding part has fully streamed. This avoids issues like adding incomplete tool calls, or thinking blocks missing signatures, to the response.

For each iterator, fully iterating through the iterator will consume the whole LLM stream. You can pause stream execution midway by breaking out of the iterator, and you can safely resume execution from the same iterator if desired.

Bases: BaseAsyncStreamResponse[AsyncContextToolkit[DepsT], FormattableT], Generic[DepsT, FormattableT]

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new AsyncContextStreamResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class AsyncContextTool

Protocol defining an async tool that can be used by LLMs with context.

An AsyncContextTool represents an async function that can be called by an LLM during a call. It includes metadata like name, description, and parameter schema.

This class is not instantiated directly but created by the @tool() decorator.

Bases: ToolSchema[AsyncContextToolFn[DepsT, AnyP, JsonableCovariantT]], Generic[DepsT, JsonableCovariantT, AnyP]

Function execute

Execute the async context tool using an LLM-provided ToolCall.

Class AsyncContextToolkit

A collection of AsyncContextTools, with helpers for getting and executing specific tools.

Bases: BaseToolkit[AsyncTool | AsyncContextTool[DepsT]], Generic[DepsT]

Function execute

Execute an AsyncContextTool using the provided tool call.

Class AsyncPrompt

An async prompt that can be called with a model to generate a response.

Created by decorating an async MessageTemplate with llm.prompt. The decorated async function becomes callable with a Model to generate LLM responses asynchronously.

An AsyncPrompt is essentially: async MessageTemplate + tools + format. It can be invoked with a model: await prompt(model, *args, **kwargs).

Bases:

Function call

Generates a response using the provided model asynchronously.

Function stream

Generates a streaming response using the provided model asynchronously.

Class AsyncResponse

The response generated by an LLM in async mode.

Bases:

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new AsyncResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Attribute AsyncStream

Type: TypeAlias

An asynchronous assistant content stream.

Class AsyncStreamResponse

An AsyncStreamResponse wraps response content from the LLM with a streaming interface.

This class supports iteration to process chunks as they arrive from the model.

Content can be streamed in one of three ways:

• Via .streams(), which provides an iterator of streams, where each stream contains chunks of streamed data. The chunks contain deltas (new content in that particular chunk), and the stream itself accumulates the collected state of all the chunks processed thus far.
• Via .chunk_stream() which allows iterating over Mirascope's provider- agnostic chunk representation.
• Via .pretty_stream() a helper method which provides all response content as str deltas. Iterating through pretty_stream will yield text content and optionally placeholder representations for other content types, but it will still consume the full stream.
• Via .structured_stream(), a helper method which provides partial structured outputs from a response (useful when FormatT is set). Iterating through structured_stream will only yield structured partials, but it will still consume the full stream.

As chunks are consumed, they are collected in-memory on the AsyncContextStreamResponse, and they become available in .content, .messages, .tool_calls, etc. All of the stream iterators can be restarted after the stream has been consumed, in which case they will yield chunks from memory in the original sequence that came from the LLM. If the stream is only partially consumed, a fresh iterator will first iterate through in-memory content, and then will continue consuming fresh chunks from the LLM.

In the specific case of text chunks, they are included in the response content as soon as they become available, via an llm.Text part that updates as more deltas come in. This enables the behavior where resuming a partially-streamed response will include as much text as the model generated.

For other chunks, like Thinking or ToolCall, they are only added to response content once the corresponding part has fully streamed. This avoids issues like adding incomplete tool calls, or thinking blocks missing signatures, to the response.

For each iterator, fully iterating through the iterator will consume the whole LLM stream. You can pause stream execution midway by breaking out of the iterator, and you can safely resume execution from the same iterator if desired.

Bases:

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new AsyncStreamResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class AsyncTextStream

Asynchronous text stream implementation.

Bases:

Function collect

Asynchronously collect all chunks and return the final Text content.

Class AsyncThoughtStream

Asynchronous thought stream implementation.

Bases:

Function collect

Asynchronously collect all chunks and return the final Thought content.

Class AsyncTool

An async tool that can be used by LLMs.

An AsyncTool represents an async function that can be called by an LLM during a call. It includes metadata like name, description, and parameter schema.

This class is not instantiated directly but created by the @tool() decorator.

Bases: ToolSchema[AsyncToolFn[AnyP, JsonableCovariantT]], Generic[AnyP, JsonableCovariantT]

Function execute

Execute the async tool using an LLM-provided ToolCall.

Class AsyncToolCallStream

Asynchronous tool call stream implementation.

Bases:

Function collect

Asynchronously collect all chunks and return the final ToolCall content.

Class AsyncToolkit

A collection of AsyncTools, with helpers for getting and executing specific tools.

Bases:

Function execute

Execute an AsyncTool using the provided tool call.

Class Audio

Audio content for a message.

Audio can be included in messages for voice or sound-based interactions.

Function download

Download and encode an audio file from a URL.

Function download_async

Asynchronously download and encode an audio file from a URL.

Function from_file

Create an Audio from a file path.

Function from_bytes

Create an Audio from raw bytes.

Class AuthenticationError

Raised for authentication failures (401, invalid API keys).

Bases:

Class BadRequestError

Raised for malformed requests (400, 422).

Bases:

Class Base64AudioSource

Audio data represented as a base64 encoded string.

Class Base64ImageSource

Image data represented as a base64 encoded string.

Class Call

A call that directly generates LLM responses without requiring a model argument.

Created by decorating a MessageTemplate with llm.call. The decorated function becomes directly callable to generate responses, with the Model bundled in.

A Call is essentially: MessageTemplate + tools + format + Model. It can be invoked directly: call(*args, **kwargs) (no model argument needed).

The model can be overridden at runtime using with llm.model(...) context manager.

Bases: BaseCall, Generic[P, FormattableT]

Function call

Generates a response using the LLM.

Function stream

Generates a streaming response using the LLM.

Class CallDecorator

Decorator for converting a MessageTemplate into a Call.

Takes a raw prompt function that returns message content and wraps it with tools, format, and a model to create a Call that can be invoked directly without needing to pass a model argument.

The decorator automatically detects whether the function is async or context-aware and creates the appropriate Call variant (Call, AsyncCall, ContextCall, or AsyncContextCall).

Conceptually: CallDecorator = PromptDecorator + Model Result: Call = MessageTemplate + tools + format + Model

Bases:

Attribute ChunkIterator

Type: TypeAlias

Synchronous iterator yielding chunks with raw data.

Class ConnectionError

Raised when unable to connect to the API (network issues, timeouts).

Bases:

Class Context

Context for LLM calls.

This class provides a context for LLM calls, including the model, parameters, and any dependencies needed for the call.

Bases:

Class ContextCall

A context-aware call that directly generates LLM responses without requiring a model argument.

Created by decorating a ContextMessageTemplate with llm.call. The decorated function (with first parameter 'ctx' of type Context[DepsT]) becomes directly callable to generate responses with context dependencies, with the Model bundled in.

A ContextCall is essentially: ContextMessageTemplate + tools + format + Model. It can be invoked directly: call(ctx, *args, **kwargs) (no model argument needed).

The model can be overridden at runtime using with llm.model(...) context manager.

Bases: BaseCall, Generic[P, DepsT, FormattableT]

Function call

Generates a response using the LLM.

Function stream

Generates a streaming response using the LLM.

Class ContextPrompt

A context-aware prompt that can be called with a model to generate a response.

Created by decorating a ContextMessageTemplate with llm.prompt. The decorated function (with first parameter 'ctx' of type Context[DepsT]) becomes callable with a Model to generate LLM responses with context dependencies.

A ContextPrompt is essentially: ContextMessageTemplate + tools + format. It can be invoked with a model: prompt(model, ctx, *args, **kwargs).

Bases:

Function call

Generates a response using the provided model.

Function stream

Generates a streaming response using the provided model.

Class ContextResponse

The response generated by an LLM from a context call.

Bases: BaseResponse[ContextToolkit[DepsT], FormattableT], Generic[DepsT, FormattableT]

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new ContextResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class ContextStreamResponse

A ContextStreamResponse wraps response content from the LLM with a streaming interface.

This class supports iteration to process chunks as they arrive from the model.

Content can be streamed in one of three ways:

• Via .streams(), which provides an iterator of streams, where each stream contains chunks of streamed data. The chunks contain deltas (new content in that particular chunk), and the stream itself accumulates the collected state of all the chunks processed thus far.
• Via .chunk_stream() which allows iterating over Mirascope's provider- agnostic chunk representation.
• Via .pretty_stream() a helper method which provides all response content as str deltas. Iterating through pretty_stream will yield text content and optionally placeholder representations for other content types, but it will still consume the full stream.
• Via .structured_stream(), a helper method which provides partial structured outputs from a response (useful when FormatT is set). Iterating through structured_stream will only yield structured partials, but it will still consume the full stream.

As chunks are consumed, they are collected in-memory on the ContextStreamResponse, and they become available in .content, .messages, .tool_calls, etc. All of the stream iterators can be restarted after the stream has been consumed, in which case they will yield chunks from memory in the original sequence that came from the LLM. If the stream is only partially consumed, a fresh iterator will first iterate through in-memory content, and then will continue consuming fresh chunks from the LLM.

In the specific case of text chunks, they are included in the response content as soon as they become available, via an llm.Text part that updates as more deltas come in. This enables the behavior where resuming a partially-streamed response will include as much text as the model generated.

For other chunks, like Thinking or ToolCall, they are only added to response content once the corresponding part has fully streamed. This avoids issues like adding incomplete tool calls, or thinking blocks missing signatures, to the response.

For each iterator, fully iterating through the iterator will consume the whole LLM stream. You can pause stream execution midway by breaking out of the iterator, and you can safely resume execution from the same iterator if desired.

Bases: BaseSyncStreamResponse[ContextToolkit[DepsT], FormattableT], Generic[DepsT, FormattableT]

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new ContextStreamResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class ContextTool

Protocol defining a tool that can be used by LLMs.

A ContextTool represents a function that can be called by an LLM during a call. It includes metadata like name, description, and parameter schema.

This class is not instantiated directly but created by the @tool() decorator.

Bases: ToolSchema[ContextToolFn[DepsT, AnyP, JsonableCovariantT]], Generic[DepsT, JsonableCovariantT, AnyP]

Function execute

Execute the context tool using an LLM-provided ToolCall.

Class ContextToolkit

A collection of ContextTools, with helpers for getting and executing specific tools.

Bases: BaseToolkit[Tool | ContextTool[DepsT]], Generic[DepsT]

Function execute

Execute a ContextTool using the provided tool call.

Class Document

Document content for a message.

Documents (like PDFs) can be included for the model to analyze or reference.

Function from_url

Create a Document from a URL.

Function from_file

Create a Document from a file path.

Function from_bytes

Create a Document from raw bytes.

Class FeatureNotSupportedError

Raised if a Mirascope feature is unsupported by chosen provider.

If compatibility is model-specific, then model_id should be specified. If the feature is not supported by the provider at all, then it may be None.

Bases:

Class FinishReason

The reason why the LLM finished generating a response.

FinishReason is only set when the response did not have a normal finish (e.g. it ran out of tokens). When a response finishes generating normally, no finish reason is set.

Bases: str, Enum

Class Format

Class representing a structured output format for LLM responses.

A Format contains metadata needed to describe a structured output type to the LLM, including the expected schema. This class is not instantiated directly, but is created by calling llm.format, or is automatically generated by LLM providers when a Formattable is passed to a call method.

Example:

from mirascope import llm

class Book:
    title: str
    author: str

print(llm.format(Book, mode="tool"))

Bases:

Attribute FormattingMode

Type: Literal['strict', 'json', 'tool']

Available modes for response format generation.

• "strict": Use strict mode for structured outputs, asking the LLM to strictly adhere to a given JSON schema. Not all providers or models support it, and may not be compatible with tool calling. When making a call using this mode, an llm.FormattingModeNotSupportedError error may be raised (if "strict" mode is wholly unsupported), or an llm.FeatureNotSupportedError may be raised (if trying to use strict along with tools and that is unsupported).

• "json": Use JSON mode for structured outputs. In contrast to strict mode, we ask the LLM to output JSON as text, though without guarantees that the model will output the expected format schema. If the provider has explicit JSON mode, it will be used; otherwise, Mirascope will modify the system prompt to request JSON output. May raise an llm.FeatureNotSupportedError if tools are present and the model does not support tool calling when using JSON mode.

• "tool": Use forced tool calling to structure outputs. Mirascope will construct an ad-hoc tool with the required json schema as tool args. When the LLM chooses that tool, it will automatically be converted from a ToolCall into regular response content (abstracting over the tool call). If other tools are present, they will be handled as regular tool calls.

Note: When llm.format is not used, the provider will automatically choose a mode at call time.

Class FormattingModeNotSupportedError

Raised when trying to use a formatting mode that is not supported by the chosen model.

Bases:

Class Image

Image content for a message.

Images can be included in messages to provide visual context. This can be used for both input (e.g., user uploading an image) and output (e.g., model generating an image).

Function from_url

Create an Image reference from a URL, without downloading it.

Function download

Download and encode an image from a URL.

Function download_async

Asynchronously download and encode an image from a URL.

Function from_file

Create an Image from a file path.

Function from_bytes

Create an Image from raw bytes.

Attribute Message

Type: TypeAlias

A message in an LLM interaction.

Messages have a role (system, user, or assistant) and content that is a sequence of content parts. The content can include text, images, audio, documents, and tool interactions.

For most use cases, prefer the convenience functions system(), user(), and assistant() instead of directly creating Message objects.

Example:

from mirascope import llm

messages = [
    llm.messages.system("You are a helpful assistant."),
    llm.messages.user("Hello, how are you?"),
]

Class MirascopeLLMError

Base exception for all Mirascope LLM errors.

Bases:

Class Model

The unified LLM interface that delegates to provider-specific clients.

This class provides a consistent interface for interacting with language models from various providers. It handles the common operations like generating responses, streaming, and async variants by delegating to the appropriate client methods.

Usage Note: In most cases, you should use llm.use_model() instead of instantiating Model directly. This preserves the ability to override the model at runtime using the llm.model() context manager. Only instantiate Model directly if you want to hardcode a specific model and prevent it from being overridden by context.

Example (recommended - allows override):

from mirascope import llm

def recommend_book(genre: str) -> llm.Response:
    # Uses context model if available, otherwise creates default
    model = llm.use_model("openai/gpt-5-mini")
    message = llm.messages.user(f"Please recommend a book in {genre}.")
    return model.call(messages=[message])

# Uses default model
response = recommend_book("fantasy")

# Override with different model
with llm.model(provider="anthropic", model_id="anthropic/claude-sonnet-4-5"):
    response = recommend_book("fantasy")  # Uses Claude

Example (direct instantiation - prevents override):

from mirascope import llm

def recommend_book(genre: str) -> llm.Response:
    # Hardcoded model, cannot be overridden by context
    model = llm.Model("openai/gpt-5-mini")
    message = llm.messages.user(f"Please recommend a book in {genre}.")
    return model.call(messages=[message])

Function call

Generate an llm.Response by synchronously calling this model's LLM provider.

Function call_async

Generate an llm.AsyncResponse by asynchronously calling this model's LLM provider.

Function stream

Generate an llm.StreamResponse by synchronously streaming from this model's LLM provider.

Function stream_async

Generate an llm.AsyncStreamResponse by asynchronously streaming from this model's LLM provider.

Function context_call

Generate an llm.ContextResponse by synchronously calling this model's LLM provider.

Function context_call_async

Generate an llm.AsyncContextResponse by asynchronously calling this model's LLM provider.

Function context_stream

Generate an llm.ContextStreamResponse by synchronously streaming from this model's LLM provider.

Function context_stream_async

Generate an llm.AsyncContextStreamResponse by asynchronously streaming from this model's LLM provider.

Function resume

Generate a new llm.Response by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function resume_async

Generate a new llm.AsyncResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function context_resume

Generate a new llm.ContextResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function context_resume_async

Generate a new llm.AsyncContextResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function resume_stream

Generate a new llm.StreamResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function resume_stream_async

Generate a new llm.AsyncStreamResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function context_resume_stream

Generate a new llm.ContextStreamResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Function context_resume_stream_async

Generate a new llm.AsyncContextStreamResponse by extending another response's messages with additional user content.

Uses the previous response's tools and output format, and this model's params.

Depending on the client, this may be a wrapper around using client call methods with the response's messages and the new content, or it may use a provider-specific API for resuming an existing interaction.

Attribute ModelId

Type: TypeAlias

Class NoRegisteredProviderError

Raised when no provider is registered for a given model_id.

Bases:

Class NotFoundError

Raised when requested resource is not found (404).

Bases:

Class Params

Common parameters shared across LLM providers.

Note: Each provider may handle these parameters differently or not support them at all. Please check provider-specific documentation for parameter support and behavior.

Bases:

Class Partial

Generate a new class with all attributes optionals.

Bases:

Class PermissionError

Raised for permission/authorization failures (403).

Bases:

Class Prompt

A prompt that can be called with a model to generate a response.

Created by decorating a MessageTemplate with llm.prompt. The decorated function becomes callable with a Model to generate LLM responses.

A Prompt is essentially: MessageTemplate + tools + format. It can be invoked with a model: prompt(model, *args, **kwargs).

Bases:

Function call

Generates a response using the provided model.

Function stream

Generates a streaming response using the provided model.

Class PromptDecorator

Decorator for converting a MessageTemplate into a Prompt.

Takes a raw prompt function that returns message content and wraps it with tools and format support, creating a Prompt that can be called with a model.

The decorator automatically detects whether the function is async or context-aware and creates the appropriate Prompt variant (Prompt, AsyncPrompt, ContextPrompt, or AsyncContextPrompt).

Bases:

Attribute Provider

Type: TypeAlias

Type alias for BaseProvider with any client type.

Attribute ProviderId

Type: KnownProviderId | str

Class RateLimitError

Raised when rate limits are exceeded (429).

Bases:

Class RawMessageChunk

A chunk containing provider-specific raw message content that will be added to the AssistantMessage.

This chunk contains a provider-specific representation of a piece of content that will be added to the AssistantMessage reconstructed by the containing stream. This content should be a Jsonable Python object for serialization purposes.

The intention is that this content may be passed as-is back to the provider when the generated AssistantMessage is being reused in conversation.

Class Response

The response generated by an LLM.

Bases:

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new Response using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Class ServerError

Raised for server-side errors (500+).

Bases:

Attribute Stream

Type: TypeAlias

A synchronous assistant content stream.

Class StreamResponse

A StreamResponse wraps response content from the LLM with a streaming interface.

This class supports iteration to process chunks as they arrive from the model.

Content can be streamed in one of three ways:

• Via .streams(), which provides an iterator of streams, where each stream contains chunks of streamed data. The chunks contain deltas (new content in that particular chunk), and the stream itself accumulates the collected state of all the chunks processed thus far.
• Via .chunk_stream() which allows iterating over Mirascope's provider- agnostic chunk representation.
• Via .pretty_stream() a helper method which provides all response content as str deltas. Iterating through pretty_stream will yield text content and optionally placeholder representations for other content types, but it will still consume the full stream.
• Via .structured_stream(), a helper method which provides partial structured outputs from a response (useful when FormatT is set). Iterating through structured_stream will only yield structured partials, but it will still consume the full stream.

As chunks are consumed, they are collected in-memory on the StreamResponse, and they become available in .content, .messages, .tool_calls, etc. All of the stream iterators can be restarted after the stream has been consumed, in which case they will yield chunks from memory in the original sequence that came from the LLM. If the stream is only partially consumed, a fresh iterator will first iterate through in-memory content, and then will continue consuming fresh chunks from the LLM.

In the specific case of text chunks, they are included in the response content as soon as they become available, via an llm.Text part that updates as more deltas come in. This enables the behavior where resuming a partially-streamed response will include as much text as the model generated.

For other chunks, like Thinking or ToolCall, they are only added to response content once the corresponding part has fully streamed. This avoids issues like adding incomplete tool calls, or thinking blocks missing signatures, to the response.

For each iterator, fully iterating through the iterator will consume the whole LLM stream. You can pause stream execution midway by breaking out of the iterator, and you can safely resume execution from the same iterator if desired.

Bases:

Function execute_tools

Execute and return all of the tool calls in the response.

Function resume

Generate a new StreamResponse using this response's messages with additional user content.

Uses this response's tools and format type. Also uses this response's provider, model, client, and params, unless the model context manager is being used to provide a new LLM as an override.

Attribute StreamResponseChunk

Type: TypeAlias

Attribute SystemContent

Type: TypeAlias

Type alias for content that can fit into a SystemMessage.

Class SystemMessage

A system message that sets context and instructions for the conversation.

Class Text

Text content for a message.

Class TextChunk

Represents an incremental text chunk in a stream.

Class TextEndChunk

Represents the end of a text chunk stream.

Class TextStartChunk

Represents the start of a text chunk stream.

Class TextStream

Synchronous text stream implementation.

Bases:

Function collect

Collect all chunks and return the final Text content.

Class Thought

Thinking content for a message.

Represents the thinking or thought process of the assistant. These generally are summaries of the model's reasoning process, rather than the direct reasoning tokens, although this behavior is model and provider specific.

Class ThoughtChunk

Represents an incremental thought chunk in a stream.

Class ThoughtEndChunk

Represents the end of a thought chunk stream.

Class ThoughtStartChunk

Represents the start of a thought chunk stream.

Class ThoughtStream

Synchronous thought stream implementation.

Bases:

Function collect

Collect all chunks and return the final Thought content.

Class TimeoutError

Raised when requests timeout or deadline exceeded.

Bases:

Class Tool

A tool that can be used by LLMs.

A Tool represents a function that can be called by an LLM during a call. It includes metadata like name, description, and parameter schema.

This class is not instantiated directly but created by the @tool() decorator.

Bases: ToolSchema[ToolFn[AnyP, JsonableCovariantT]], Generic[AnyP, JsonableCovariantT]

Function execute

Execute the tool using an LLM-provided ToolCall.

Class ToolCall

Tool call content for a message.

Represents a request from the assistant to call a tool. This is part of an assistant message's content.

Class ToolCallChunk

Represents an incremental tool call chunk in a stream.

Class ToolCallEndChunk

Represents the end of a tool call chunk stream.

Class ToolCallStartChunk

Represents the start of a tool call chunk stream.

Class ToolCallStream

Synchronous tool call stream implementation.

Bases:

Function collect

Collect all chunks and return the final ToolCall content.

Class ToolNotFoundError

Raised if a tool_call cannot be converted to any corresponding tool.

Bases:

Class ToolOutput

Tool output content for a message.

Represents the output from a tool call. This is part of a user message's content, typically following a tool call from the assistant.

Bases:

Class Toolkit

A collection of Tools, with helpers for getting and executing specific tools.

Bases:

Function execute

Execute a Tool using the provided tool call.

Class URLImageSource

Image data referenced via external URL.

Attribute UserContent

Type: TypeAlias

Type alias for content that can fit into a UserMessage.

Attribute UserContentPart

Type: TypeAlias

Content parts that can be included in a UserMessage.

Class UserMessage

A user message containing input from the user.

Function call

Decorates a MessageTemplate to create a Call that can be invoked directly.

The llm.call decorator is the most convenient way to use Mirascope. It transforms a raw prompt function (that returns message content) into a Call object that bundles the function with tools, format, and a model. The resulting Call can be invoked directly to generate LLM responses without needing to pass a model argument.

The decorator automatically detects the function type:

• If the first parameter is named 'ctx' with type llm.Context[T] (or a subclass thereof), creates a ContextCall
• If the function is async, creates an AsyncCall or AsyncContextCall
• Otherwise, creates a regular Call

The model specified in the decorator can be overridden at runtime using the llm.model() context manager. When overridden, the context model completely replaces the decorated model, including all parameters.

Conceptual flow:

• MessageTemplate: raw function returning content
• @llm.prompt: MessageTemplate → Prompt Includes tools and format, if applicable. Can be called by providing a Model.
• @llm.call: MessageTemplate → Call. Includes a model, tools, and format. The model may be created on the fly from a model identifier and optional params, or provided outright.

Example:

Regular call:

from mirascope import llm

@llm.call("openai/gpt-4")
def recommend_book(genre: str):
    return f"Please recommend a book in {genre}."

response: llm.Response = recommend_book("fantasy")
print(response.pretty())

Example:

Context call:

from dataclasses import dataclass
from mirascope import llm

@dataclass
class User:
    name: str
    age: int

@llm.call("openai/gpt-4")
def recommend_book(ctx: llm.Context[User], genre: str):
    return f"Recommend a {genre} book for {ctx.deps.name}, age {ctx.deps.age}."

ctx = llm.Context(deps=User(name="Alice", age=15))
response = recommend_book(ctx, "fantasy")
print(response.pretty())

Module calls

The llm.calls module.

Module content

The llm.messages.content module.

Module exceptions

Mirascope llm exception hierarchy for unified error handling across providers.

Function format

Returns a Format that describes structured output for a Formattable type.

This function converts a Formattable type (e.g. Pydantic BaseModel) into a Format object that describes how the object should be formatted. Calling llm.format is optional, as all the APIs that expect a Format can also take the Formattable type directly. However, calling llm.format is necessary in order to specify the formatting mode that will be used.

The Formattable type may provide custom formatting instructions via a formatting_instructions(cls) classmethod. If that method is present, it will be called, and the resulting instructions will automatically be appended to the system prompt.

If no formatting instructions are present, then Mirascope may auto-generate instructions based on the active format mode. To disable this behavior and all prompt modification, you can add the formatting_instructions classmethod and have it return None.

Module formatting

Response formatting interfaces for structuring LLM outputs.

This module provides a way to define structured output formats for LLM responses. The @format decorator can be applied to classes to specify how LLM outputs should be structured and parsed.

Function load_provider

Create a cached provider instance for the specified provider id.

Module mcp

MCP compatibility module.

Module messages

The messages module for LLM interactions.

This module defines the message types used in LLM interactions. Messages are represented as a unified Message class with different roles (system, user, assistant) and flexible content arrays that can include text, images, audio, documents, and tool interactions.

Function model

Helper for creating a Model instance (which may be used as a context manager).

This is just an alias for the Model constructor, added for convenience.

This function returns a Model instance that implements the context manager protocol. When used with a with statement, the model will be set in context and used by both llm.use_model() and llm.call() within that context. This allows you to override the default model at runtime without modifying function definitions.

The returned Model instance can also be stored and reused:

m = llm.model("openai/gpt-4o")
# Use directly
response = m.call(messages=[...])
# Or use as context manager
with m:
    response = recommend_book("fantasy")

When a model is set in context, it completely overrides any model ID or parameters specified in llm.use_model() or llm.call(). The context model's parameters take precedence, and any unset parameters use default values.

Function model_from_context

Get the LLM currently set via context, if any.

Module models

The llm.models module for implementing the Model interface and utilities.

This module provides a unified interface for interacting with different LLM models through the Model class. The llm.model() context manager allows you to override the model at runtime, and llm.use_model() retrieves the model from context or creates a default one.

Function prompt

Decorates a MessageTemplate to create a Prompt callable with a model.

This decorator transforms a raw prompt function (that returns message content) into a Prompt object that can be invoked with a model to generate LLM responses.

The decorator automatically detects the function type:

• If the first parameter is named 'ctx' with type llm.Context[T], creates a ContextPrompt
• If the function is async, creates an AsyncPrompt or AsyncContextPrompt
• Otherwise, creates a regular Prompt

Module prompts

The prompt templates module for LLM interactions.

This module defines the prompt templates used in LLM interactions, which are written as python functions.

Module providers

Interfaces for LLM providers.

Function register_provider

Register a provider with scope(s) in the global registry.

Scopes use prefix matching on model IDs:

• "anthropic/" matches "anthropic/*"
• "anthropic/claude-4-5" matches "anthropic/claude-4-5*"
• "anthropic/claude-4-5-sonnet" matches exactly "anthropic/claude-4-5-sonnet"

When multiple scopes match a model_id, the longest match wins.