Adds a message to a Conversation, that the language model will begin replying to. You can receive the reply one piece at a time by calling conversation_get_reply_piece(conversation c) in a loop
Returns a single piece of a reply (generally one word at a time) from the Conversation You can use a loop while checking Conversation Is Replying to retrieve the reply as it generates
Checks if a language model is currently generating a reply within a Conversation. If so, you can continue to receive the message with conversation_get_reply_piece(conversation c)
Checks if a language model is currently “thinking” while generating a reply within a Conversation. You can use this to filter out the “thoughts” and display them differently (or hide them entirely)
Creates a new Conversation object, that uses the default language model. The Conversation object can have messages added to it, and responses streamed back from it via the other Conversation functions and procedures
Creates a new Conversation object, that uses a chosen language model. The Conversation object can have messages added to it, and responses streamed back from it via the other Conversation functions and procedures
Generates a reply to a textual prompt by a language model The language model will respond to the textual prompt in a chat style format. It will follow instructions and answer questions. Instruct or Thinking models are recommended. Base models likely won’t output sensible results.
Generates a reply to a textual prompt by a language model The language model will respond to the textual prompt in a chat style format. It will follow instructions and answer questions. Instruct or Thinking models are recommended. Base models likely won’t output sensible results.
Generates text that continues from a prompt, with a maximum of 125 tokens. The language model will continue predicting text based on patterns in the prompt - it will not directly follow instructions or answer questions. Base models are recommended; Instruct and Thinking models may work.
Generates text that continues from a prompt, with a maximum of 125 tokens. The language model will continue predicting text based on patterns in the prompt - it will not directly follow instructions or answer questions. Base models are recommended; Instruct and Thinking models may work.
Generates text that continues from a prompt - with default of 125 tokens. The language model will continue predicting text based on patterns in the prompt - it will not directly follow instructions or answer questions. Base models are recommended; Instruct and Thinking models may work.
Parameters:
Name
Type
Description
text
String
The input text for the language model to continue.
Generates text that continues from a prompt. The language model will continue predicting text based on patterns in the prompt - it will not directly follow instructions or answer questions. Base models are recommended; Instruct and Thinking models may work.
Parameters:
Name
Type
Description
text
String
The input text for the language model to continue.
max_tokens
Integer
The maximum tokens used in response - determining the length of the output and the time taken. Keep this small for reasonable execution times.
The Conversation type is used to refer to conversations between the user
and a language model. You can use it to send messages to the language model,
and stream responses back.
created with create_conversation(), create_conversation(language_model model)
and must be released using free_conversation() (to release a specific Conversation object)
or free_all_conversation() (to release all created Conversation objects).
Language Models:
Choose between different language models to trade off speed and intelligence
Each model is scaled to fit within 1~2GB and will be automatically downloaded when needed - feel free to try them out!
Qwen3 0.6B Base model - small, extremely fast and good for text completion. Very limited world knowledge.
QWEN3_0_6B_INSTRUCT
Qwen3 0.6B Instruct model (default) - small, extremely fast and can follow simple instructions. Very limited world knowledge.
QWEN3_0_6B_THINKING
Qwen3 0.6B Thinking model - small, extremely fast and can follow more specific instructions, but has a short delay before starting to reply. Very limited world knowledge.
QWEN3_1_7B_BASE
Qwen3 1.7B Base model - decently fast and good for text completion. Limited world knowledge.
QWEN3_1_7B_INSTRUCT
Qwen3 1.7B Instruct model - decently fast and can follow instructions. Limited world knowledge.
QWEN3_1_7B_THINKING
Qwen3 1.7B Thinking model - decently fast and can follow more difficult instructions, but has a delay before starting to reply. Limited world knowledge.
QWEN3_4B_BASE
Qwen3 4B Base model - slower but excellent for text completion/pattern based completion
QWEN3_4B_INSTRUCT
Qwen3 4B Instruct model - slower but can follow complex instructions
QWEN3_4B_THINKING
Qwen3 4B Thinking model - slower but can follow complex and specific instructions, but has a potentially long delay before starting to reply
GEMMA3_270M_BASE
Gemma3 270M Base model - tiny, extremely fast, and good for text completion. Very limited world knowledge.
GEMMA3_270M_INSTRUCT
Gemma3 270M Instruct model - tiny, extremely fast, and good for very simple instructions. Very limited world knowledge.
GEMMA3_1B_BASE
Gemma3 1B Base model - fast and good for text completion. Has decent world knowledge and multilingual abilities.
GEMMA3_1B_INSTRUCT
Gemma3 1B Instruct model - fast and can follow instructions. Has decent world knowledge and multilingual abilities.
GEMMA3_4B_BASE
Gemma3 4B Base model - slower but good for text completion/pattern based completion. Has decent world knowledge and multilingual abilities.
GEMMA3_4B_INSTRUCT
Gemma3 4B Instruct model - slower but can follow complex instructions. Has decent world knowledge and multilingual abilities.
Constant
Description
LanguageModel.Qwen306BBase
Qwen3 0.6B Base model - small, extremely fast and good for text completion. Very limited world knowledge.
LanguageModel.Qwen306BInstruct
Qwen3 0.6B Instruct model (default) - small, extremely fast and can follow simple instructions. Very limited world knowledge.
LanguageModel.Qwen306BThinking
Qwen3 0.6B Thinking model - small, extremely fast and can follow more specific instructions, but has a short delay before starting to reply. Very limited world knowledge.
LanguageModel.Qwen317BBase
Qwen3 1.7B Base model - decently fast and good for text completion. Limited world knowledge.
LanguageModel.Qwen317BInstruct
Qwen3 1.7B Instruct model - decently fast and can follow instructions. Limited world knowledge.
LanguageModel.Qwen317BThinking
Qwen3 1.7B Thinking model - decently fast and can follow more difficult instructions, but has a delay before starting to reply. Limited world knowledge.
LanguageModel.Qwen34BBase
Qwen3 4B Base model - slower but excellent for text completion/pattern based completion
LanguageModel.Qwen34BInstruct
Qwen3 4B Instruct model - slower but can follow complex instructions
LanguageModel.Qwen34BThinking
Qwen3 4B Thinking model - slower but can follow complex and specific instructions, but has a potentially long delay before starting to reply
LanguageModel.Gemma3270mBase
Gemma3 270M Base model - tiny, extremely fast, and good for text completion. Very limited world knowledge.
LanguageModel.Gemma3270mInstruct
Gemma3 270M Instruct model - tiny, extremely fast, and good for very simple instructions. Very limited world knowledge.
LanguageModel.Gemma31BBase
Gemma3 1B Base model - fast and good for text completion. Has decent world knowledge and multilingual abilities.
LanguageModel.Gemma31BInstruct
Gemma3 1B Instruct model - fast and can follow instructions. Has decent world knowledge and multilingual abilities.
LanguageModel.Gemma34BBase
Gemma3 4B Base model - slower but good for text completion/pattern based completion. Has decent world knowledge and multilingual abilities.
LanguageModel.Gemma34BInstruct
Gemma3 4B Instruct model - slower but can follow complex instructions. Has decent world knowledge and multilingual abilities.
Constant
Description
LanguageModel.qwen3_0_6b_base
Qwen3 0.6B Base model - small, extremely fast and good for text completion. Very limited world knowledge.
LanguageModel.qwen3_0_6b_instruct
Qwen3 0.6B Instruct model (default) - small, extremely fast and can follow simple instructions. Very limited world knowledge.
LanguageModel.qwen3_0_6b_thinking
Qwen3 0.6B Thinking model - small, extremely fast and can follow more specific instructions, but has a short delay before starting to reply. Very limited world knowledge.
LanguageModel.qwen3_1_7b_base
Qwen3 1.7B Base model - decently fast and good for text completion. Limited world knowledge.
LanguageModel.qwen3_1_7b_instruct
Qwen3 1.7B Instruct model - decently fast and can follow instructions. Limited world knowledge.
LanguageModel.qwen3_1_7b_thinking
Qwen3 1.7B Thinking model - decently fast and can follow more difficult instructions, but has a delay before starting to reply. Limited world knowledge.
LanguageModel.qwen3_4b_base
Qwen3 4B Base model - slower but excellent for text completion/pattern based completion
LanguageModel.qwen3_4b_instruct
Qwen3 4B Instruct model - slower but can follow complex instructions
LanguageModel.qwen3_4b_thinking
Qwen3 4B Thinking model - slower but can follow complex and specific instructions, but has a potentially long delay before starting to reply
LanguageModel.gemma3_270m_base
Gemma3 270M Base model - tiny, extremely fast, and good for text completion. Very limited world knowledge.
LanguageModel.gemma3_270m_instruct
Gemma3 270M Instruct model - tiny, extremely fast, and good for very simple instructions. Very limited world knowledge.
LanguageModel.gemma3_1b_base
Gemma3 1B Base model - fast and good for text completion. Has decent world knowledge and multilingual abilities.
LanguageModel.gemma3_1b_instruct
Gemma3 1B Instruct model - fast and can follow instructions. Has decent world knowledge and multilingual abilities.
LanguageModel.gemma3_4b_base
Gemma3 4B Base model - slower but good for text completion/pattern based completion. Has decent world knowledge and multilingual abilities.
LanguageModel.gemma3_4b_instruct
Gemma3 4B Instruct model - slower but can follow complex instructions. Has decent world knowledge and multilingual abilities.
Constant
Description
LanguageModel.Qwen306BBase
Qwen3 0.6B Base model - small, extremely fast and good for text completion. Very limited world knowledge.
LanguageModel.Qwen306BInstruct
Qwen3 0.6B Instruct model (default) - small, extremely fast and can follow simple instructions. Very limited world knowledge.
LanguageModel.Qwen306BThinking
Qwen3 0.6B Thinking model - small, extremely fast and can follow more specific instructions, but has a short delay before starting to reply. Very limited world knowledge.
LanguageModel.Qwen317BBase
Qwen3 1.7B Base model - decently fast and good for text completion. Limited world knowledge.
LanguageModel.Qwen317BInstruct
Qwen3 1.7B Instruct model - decently fast and can follow instructions. Limited world knowledge.
LanguageModel.Qwen317BThinking
Qwen3 1.7B Thinking model - decently fast and can follow more difficult instructions, but has a delay before starting to reply. Limited world knowledge.
LanguageModel.Qwen34BBase
Qwen3 4B Base model - slower but excellent for text completion/pattern based completion
LanguageModel.Qwen34BInstruct
Qwen3 4B Instruct model - slower but can follow complex instructions
LanguageModel.Qwen34BThinking
Qwen3 4B Thinking model - slower but can follow complex and specific instructions, but has a potentially long delay before starting to reply
LanguageModel.Gemma3270mBase
Gemma3 270M Base model - tiny, extremely fast, and good for text completion. Very limited world knowledge.
LanguageModel.Gemma3270mInstruct
Gemma3 270M Instruct model - tiny, extremely fast, and good for very simple instructions. Very limited world knowledge.
LanguageModel.Gemma31BBase
Gemma3 1B Base model - fast and good for text completion. Has decent world knowledge and multilingual abilities.
LanguageModel.Gemma31BInstruct
Gemma3 1B Instruct model - fast and can follow instructions. Has decent world knowledge and multilingual abilities.
LanguageModel.Gemma34BBase
Gemma3 4B Base model - slower but good for text completion/pattern based completion. Has decent world knowledge and multilingual abilities.
LanguageModel.Gemma34BInstruct
Gemma3 4B Instruct model - slower but can follow complex instructions. Has decent world knowledge and multilingual abilities.