ChatNovita

Delivers an affordable, reliable, and simple inference platform for running top LLM models.

You can find all the models we support here: Novita AI Featured Models or request the Models API to get all available models.

Try the Novita AI Llama 3 API Demo today!

Overview

Model features

Tool calling	Structured output	JSON mode	Image input	Audio input	Video input	Token-level streaming	Native async	Token usage	Logprobs
❌	✅	✅	❌	❌	❌	❌	❌	✅	❌

Setup

To access Novita AI models you'll need to create a Novita account and get an API key.

Credentials

Head to this page to sign up to Novita AI and generate an API key. Once you've done this set the NOVITA_API_KEY environment variable:

import getpass
import os

if "NOVITA_API_KEY" not in os.environ:
    os.environ["NOVITA_API_KEY"] = getpass.getpass("Enter your Novita API key: ")

If you want to get automated tracing of your model calls you can also set your LangSmith API key by uncommenting below:

# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API key: ")
# os.environ["LANGSMITH_TRACING"] = "true"

Installation

The LangChain Novita integration lives in the langchain-community package:

%pip install -qU langchain-community

Instantiation

Now we can instantiate our model object and generate chat completions. Try the Novita AI Llama 3 API Demo today!

from langchain_community.chat_models.novita import ChatNovita
from langchain_core.messages import HumanMessage, SystemMessage

llm = ChatNovita(
    model="meta-llama/llama-3.1-8b-instruct",
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2,
    # other params...
)

API Reference:ChatNovita | HumanMessage | SystemMessage

Invocation

messages = [
    SystemMessage(
        content="You are a helpful assistant that translates English to French."
    ),
    HumanMessage(
        content="Translate this sentence from English to French. I love programming."
    ),
]
ai_msg = llm.invoke(messages)
ai_msg

print(ai_msg.content)

Chaining

We can chain our model with a prompt template like so:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "input_language": "English",
        "output_language": "German",
        "input": "I love programming.",
    }
)

API Reference:ChatPromptTemplate

API reference

For detailed documentation of Novita AI LLM APIs, head to Novita AI LLM API reference

Related

• Chat model conceptual guide
• Chat model how-to guides