Function trimMessages

Trim messages to be below a token count.

Throws

If two incompatible arguments are specified or an unrecognized strategy is specified.

Example

import { trimMessages, AIMessage, BaseMessage, HumanMessage, SystemMessage } from "@langchain/core/messages";

const messages = [
  new SystemMessage("This is a 4 token text. The full message is 10 tokens."),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "first",
  }),
  new AIMessage({
    content: [
      { type: "text", text: "This is the FIRST 4 token block." },
      { type: "text", text: "This is the SECOND 4 token block." },
    ],
    id: "second",
  }),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "third",
  }),
  new AIMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "fourth",
  }),
];

function dummyTokenCounter(messages: BaseMessage[]): number {
  // treat each message like it adds 3 default tokens at the beginning
  // of the message and at the end of the message. 3 + 4 + 3 = 10 tokens
  // per message.

  const defaultContentLen = 4;
  const defaultMsgPrefixLen = 3;
  const defaultMsgSuffixLen = 3;

  let count = 0;
  for (const msg of messages) {
    if (typeof msg.content === "string") {
      count += defaultMsgPrefixLen + defaultContentLen + defaultMsgSuffixLen;
    }
    if (Array.isArray(msg.content)) {
      count +=
        defaultMsgPrefixLen +
        msg.content.length * defaultContentLen +
        defaultMsgSuffixLen;
    }
  }
  return count;
}
Copy

First 30 tokens, not allowing partial messages:

await trimMessages(messages, {
  maxTokens: 30,
  tokenCounter: dummyTokenCounter,
  strategy: "first",
});
Copy

Output:

[
  new SystemMessage(
    "This is a 4 token text. The full message is 10 tokens."
  ),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "first",
  }),
]
Copy

First 30 tokens, allowing partial messages:

await trimMessages(messages, {
  maxTokens: 30,
  tokenCounter: dummyTokenCounter,
  strategy: "first",
  allowPartial: true,
});
Copy

Output:

[
  new SystemMessage(
    "This is a 4 token text. The full message is 10 tokens."
  ),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "first",
  }),
  new AIMessage({
    content: [{ type: "text", text: "This is the FIRST 4 token block." }],
    id: "second",
  }),
]
Copy

First 30 tokens, allowing partial messages, have to end on HumanMessage:

await trimMessages(messages, {
  maxTokens: 30,
  tokenCounter: dummyTokenCounter,
  strategy: "first",
  allowPartial: true,
  endOn: "human",
});
Copy

Output:

[
  new SystemMessage(
    "This is a 4 token text. The full message is 10 tokens."
  ),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "first",
  }),
]
Copy

Last 30 tokens, including system message, not allowing partial messages:

await trimMessages(messages, {
  maxTokens: 30,
  includeSystem: true,
  tokenCounter: dummyTokenCounter,
  strategy: "last",
});
Copy

Output:

[
  new SystemMessage(
    "This is a 4 token text. The full message is 10 tokens."
  ),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "third",
  }),
  new AIMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "fourth",
  }),
]
Copy

Last 40 tokens, including system message, allowing partial messages:

await trimMessages(messages, {
  maxTokens: 40,
  tokenCounter: dummyTokenCounter,
  strategy: "last",
  allowPartial: true,
  includeSystem: true,
});
Copy

Output:

[
  new SystemMessage(
    "This is a 4 token text. The full message is 10 tokens."
  ),
  new AIMessage({
    content: [{ type: "text", text: "This is the FIRST 4 token block." }],
    id: "second",
  }),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "third",
  }),
  new AIMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "fourth",
  }),
]
Copy

Last 30 tokens, including system message, allowing partial messages, end on HumanMessage:

await trimMessages(messages, {
  maxTokens: 30,
  tokenCounter: dummyTokenCounter,
  strategy: "last",
  endOn: "human",
  includeSystem: true,
  allowPartial: true,
});
Copy

Output:

[
  new SystemMessage(
    "This is a 4 token text. The full message is 10 tokens."
  ),
  new AIMessage({
    content: [{ type: "text", text: "This is the FIRST 4 token block." }],
    id: "second",
  }),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "third",
  }),
]
Copy

Last 40 tokens, including system message, allowing partial messages, start on HumanMessage:

await trimMessages(messages, {
  maxTokens: 40,
  tokenCounter: dummyTokenCounter,
  strategy: "last",
  includeSystem: true,
  allowPartial: true,
  startOn: "human",
});
Copy

Output:

[
  new SystemMessage(
    "This is a 4 token text. The full message is 10 tokens."
  ),
  new HumanMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "third",
  }),
  new AIMessage({
    content: "This is a 4 token text. The full message is 10 tokens.",
    id: "fourth",
  }),
]
Copy