Stream

NAME

LLM::Chat::Backend::Response::OpenRouter::Stream - Streaming variant of Response::OpenRouter

SYNOPSIS


use LLM::Chat::Backend::Response::OpenRouter::Stream;

# Constructed by L<LLM::Chat::Backend::OpenRouter>'s
# C<chat-completion-stream> path. Behaves like
# C<LLM::Chat::Backend::Response::Stream> (accumulates emitted
# tokens into C<.latest>) and additionally carries
# C<.cost> / C<.generation-id> / C<.provider-name> / C<.is-byok>
# from the L<...::OpenRouter::Augment> role.

DESCRIPTION

Subclass of LLM::Chat::Backend::Response::Stream that consumes the LLM::Chat::Backend::Response::OpenRouter::Augment role to add OR-specific usage + routing fields. The streaming ._emit / ._done behaviour is inherited unchanged from the base stream class.

OR's terminal usage frame (sent after finish_reason: stop, before [DONE]) is lifted by the backend via _set-or-usage on this same instance — so cost / generation-id / provider-name become readable by the time the supply done-signal fires.

LLM::Chat v0.10.0

Simple framework for LLM inferencing

Authors

  • Matt Doughty

License

Artistic-2.0

Dependencies

Cro::HTTP::Client:ver<0.8.11+>:auth<zef:cro>Digest::SHA256::Native:ver<1.0.0+>:auth<zef:bduggan>JSON::Fast:ver<0.19+>:auth<cpan:TIMOTIMO>Template::Jinja2:ver<0.3.0+>:auth<zef:apogee>Tokenizers:ver<0.3.0+>:auth<zef:apogee>UUID::V4:ver<1.0.0+>:auth<zef:masukomi>

Provides

  • LLM::Chat
  • LLM::Chat::Backend
  • LLM::Chat::Backend::KoboldCpp
  • LLM::Chat::Backend::Mock
  • LLM::Chat::Backend::OpenAICommon
  • LLM::Chat::Backend::OpenRouter
  • LLM::Chat::Backend::Response
  • LLM::Chat::Backend::Response::OpenRouter
  • LLM::Chat::Backend::Response::OpenRouter::Augment
  • LLM::Chat::Backend::Response::OpenRouter::Stream
  • LLM::Chat::Backend::Response::Stream
  • LLM::Chat::Backend::Settings
  • LLM::Chat::Conversation
  • LLM::Chat::Conversation::Message
  • LLM::Chat::Debug
  • LLM::Chat::Retry
  • LLM::Chat::Retry::Exceptions
  • LLM::Chat::Template
  • LLM::Chat::Template::ChatML
  • LLM::Chat::Template::DeepSeekV4
  • LLM::Chat::Template::Gemma2
  • LLM::Chat::Template::Jinja2
  • LLM::Chat::Template::Llama3
  • LLM::Chat::Template::Llama4
  • LLM::Chat::Template::MistralV7
  • LLM::Chat::TokenCounter
  • LLM::Chat::ToolLoop

The Camelia image is copyright 2009 by Larry Wall. "Raku" is a trademark of the Yet Another Society. All rights reserved.

Built with Podlite — the markup and publishing tools behind this site.