VertexAI PaLM2 Chat

Generative Language PaLM API 允许开发人员使用 PaLM 模型构建生成式 AI 应用程序。大语言模型 (LLM) 是一种功能强大且用途广泛的机器学习模型类型,它使计算机能够通过一系列提示理解和生成自然语言。PaLM API 基于 Google 的下一代 LLM,PaLM。它擅长各种不同的任务,如代码生成、推理和写作。你可以使用 PaLM API 为内容生成、对话代理、摘要和分类系统等用例构建生成式 AI 应用程序。

The Generative Language PaLM API allows developers to build generative AI applications using the PaLM model. Large Language Models (LLMs) are a powerful, versatile type of machine learning model that enables computers to comprehend and generate natural language through a series of prompts. The PaLM API is based on Google’s next generation LLM, PaLM. It excels at a variety of different tasks like code generation, reasoning, and writing. You can use the PaLM API to build generative AI applications for use cases like content generation, dialogue agents, summarization and classification systems, and more.

基于 Models REST API

Based on the Models REST API.

Prerequisites

要访问 PaLM2 REST API,你需要获取 makersuite 的访问 API KEY 表单。

To access the PaLM2 REST API you need to obtain an access API KEY form makersuite.

目前 PaLM API 在美国境外不可用,但您可以使用 VPN 进行测试。

Currently the PaLM API it is not available outside US, but you can use VPN for testing.

Spring AI 项目定义了一个名为 spring.ai.vertex.ai.api-key 的配置属性,你应该将其设置为获得的 API Key 的值。导出环境变量是设置该配置属性的一种方法:

The Spring AI project defines a configuration property named spring.ai.vertex.ai.api-key that you should set to the value of the API Key obtained. Exporting an environment variable is one way to set that configuration property:

export SPRING_AI_VERTEX_AI_API_KEY=<INSERT KEY HERE>

Add Repositories and BOM

Spring AI 工件发布在 Spring Milestone 和 Snapshot 存储库中。有关将这些存储库添加到你的构建系统的说明,请参阅 Repositories 部分。

Spring AI artifacts are published in Spring Milestone and Snapshot repositories. Refer to the Repositories section to add these repositories to your build system.

为了帮助进行依赖项管理,Spring AI 提供了一个 BOM(物料清单)以确保在整个项目中使用一致版本的 Spring AI。有关将 Spring AI BOM 添加到你的构建系统的说明,请参阅 Dependency Management 部分。

To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the Dependency Management section to add the Spring AI BOM to your build system.

Auto-configuration

Spring AI 为 VertexAI Chat 客户端提供 Spring Boot 自动配置。要启用它,请将以下依赖项添加到项目的 Maven pom.xml 文件中:

Spring AI provides Spring Boot auto-configuration for the VertexAI Chat Client. To enable it add the following dependency to your project’s Maven pom.xml file:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-vertex-ai-palm2-spring-boot-starter</artifactId>
</dependency>

或添加到 Gradle build.gradle 构建文件中。

or to your Gradle build.gradle build file.

dependencies {
    implementation 'org.springframework.ai:spring-ai-vertex-ai-palm2-spring-boot-starter'
}
  1. 参见 Dependency Management 部分,将 Spring AI BOM 添加到你的构建文件中。

Refer to the Dependency Management section to add the Spring AI BOM to your build file.

Chat Properties

前缀 spring.ai.vertex.ai 用作属性前缀,它允许你连接到 VertexAI。

The prefix spring.ai.vertex.ai is used as the property prefix that lets you connect to VertexAI.

Property Description Default

spring.ai.vertex.ai.ai.base-url

The URL to connect to

[role="bare"]https://generativelanguage.googleapis.com/v1beta3

spring.ai.vertex.ai.api-key

The API Key

-

前缀 spring.ai.vertex.ai.chat 是属性前缀,它允许你配置 VertexAI Chat 的聊天客户端实现。

The prefix spring.ai.vertex.ai.chat is the property prefix that lets you configure the chat client implementation for VertexAI Chat.

Property Description Default

spring.ai.vertex.ai.chat.enabled

Enable Vertex AI PaLM API Chat client.

true

spring.ai.vertex.ai.chat.model

This is the Vertex Chat model to use

chat-bison-001

spring.ai.vertex.ai.chat.options.temperature

Controls the randomness of the output. Values can range over [0.0,1.0], inclusive. A value closer to 1.0 will produce responses that are more varied, while a value closer to 0.0 will typically result in less surprising responses from the generative. This value specifies default to be used by the backend while making the call to the generative.

0.7

spring.ai.vertex.ai.chat.options.topK

The maximum number of tokens to consider when sampling. The generative uses combined Top-k and nucleus sampling. Top-k sampling considers the set of topK most probable tokens.

20

spring.ai.vertex.ai.chat.options.topP

The maximum cumulative probability of tokens to consider when sampling. The generative uses combined Top-k and nucleus sampling. Nucleus sampling considers the smallest set of tokens whose probability sum is at least topP.

-

spring.ai.vertex.ai.chat.options.candidateCount

The number of generated response messages to return. This value must be between [1, 8], inclusive. Defaults to 1.

1

所有以 spring.ai.vertex.ai.chat.options 为前缀的属性都可以通过在 Prompt 调用中添加请求特定的 Chat Options 来在运行时进行覆盖。

All properties prefixed with spring.ai.vertex.ai.chat.options can be overridden at runtime by adding a request specific Chat Options to the Prompt call.

Chat Options

VertexAiPaLm2ChatOptions.java 提供模型配置,例如温度、topK 等。

The VertexAiPaLm2ChatOptions.java provides model configurations, such as the temperature, the topK, etc.

在启动时,可以使用 VertexAiPaLm2ChatClient(api, options) 构造函数或 spring.ai.vertex.ai.chat.options.* 属性配置默认选项。

On start-up, the default options can be configured with the VertexAiPaLm2ChatClient(api, options) constructor or the spring.ai.vertex.ai.chat.options.* properties.

在运行时,你可以通过向 Prompt 调用添加新的请求特定选项来覆盖默认选项。例如,覆盖特定请求的默认温度:

At run-time you can override the default options by adding new, request specific, options to the Prompt call. For example to override the default temperature for a specific request:

ChatResponse response = chatClient.call(
    new Prompt(
        "Generate the names of 5 famous pirates.",
        VertexAiPaLm2ChatOptions.builder()
            .withTemperature(0.4)
        .build()
    ));
  1. 除了模型特定的 VertexAiPaLm2ChatOptions 之外,你还可以使用用 ChatOptionsBuilder#builder() 创建的便携式 ChatOptions 实例。

In addition to the model specific VertexAiPaLm2ChatOptions you can use a portable ChatOptions instance, created with the ChatOptionsBuilder#builder().

Sample Controller (Auto-configuration)

Create 一个新的 Spring Boot 项目,并将 spring-ai-vertex-ai-palm2-spring-boot-starter 添加到你的 pom(或 gradle)依赖项。

Create a new Spring Boot project and add the spring-ai-vertex-ai-palm2-spring-boot-starter to your pom (or gradle) dependencies.

src/main/resources 目录下添加一个 application.properties 文件,以启用并配置 VertexAi Chat 客户端:

Add a application.properties file, under the src/main/resources directory, to enable and configure the VertexAi Chat client:

spring.ai.vertex.ai.api-key=YOUR_API_KEY
spring.ai.vertex.ai.chat.model=chat-bison-001
spring.ai.vertex.ai.chat.options.temperature=0.5

api-key 替换为你的 VertexAI 凭据。

replace the api-key with your VertexAI credentials.

这将创建一个 VertexAiPaLm2ChatClient 实现,你可以将其注入到类中。以下是一个简单的 @Controller 类的示例,它使用聊天客户端进行文本生成。

This will create a VertexAiPaLm2ChatClient implementation that you can inject into your class. Here is an example of a simple @Controller class that uses the chat client for text generations.

@RestController
public class ChatController {

    private final VertexAiPaLm2ChatClient chatClient;

    @Autowired
    public ChatController(VertexAiPaLm2ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    @GetMapping("/ai/generate")
    public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        return Map.of("generation", chatClient.call(message));
    }

    @GetMapping("/ai/generateStream")
	public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        Prompt prompt = new Prompt(new UserMessage(message));
        return chatClient.stream(prompt);
    }
}

Manual Configuration

VertexAiPaLm2ChatClient实现 `ChatClient`并使用 Low-level VertexAiPaLm2Api Client连接到 VertexAI 服务。

The VertexAiPaLm2ChatClient implements the ChatClient and uses the Low-level VertexAiPaLm2Api Client to connect to the VertexAI service.

spring-ai-vertex-ai-palm2 依赖添加到项目的 Maven pom.xml 文件中:

Add the spring-ai-vertex-ai-palm2 dependency to your project’s Maven pom.xml file:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-vertex-ai-palm2</artifactId>
</dependency>

或添加到 Gradle build.gradle 构建文件中。

or to your Gradle build.gradle build file.

dependencies {
    implementation 'org.springframework.ai:spring-ai-vertex-ai-palm'
}
  1. 参见 Dependency Management 部分,将 Spring AI BOM 添加到你的构建文件中。

Refer to the Dependency Management section to add the Spring AI BOM to your build file.

接下来,创建一个 VertexAiPaLm2ChatClient 并将其用于文本生成:

Next, create a VertexAiPaLm2ChatClient and use it for text generations:

VertexAiPaLm2Api vertexAiApi = new VertexAiPaLm2Api(< YOUR PALM_API_KEY>);

var chatClient = new VertexAiPaLm2ChatClient(vertexAiApi,
    VertexAiPaLm2ChatOptions.builder()
        .withTemperature(0.4)
    .build());

ChatResponse response = chatClient.call(
    new Prompt("Generate the names of 5 famous pirates."));

VertexAiPaLm2ChatOptions 提供聊天请求的配置信息。 VertexAiPaLm2ChatOptions.Builder 是流畅选项生成器。

The VertexAiPaLm2ChatOptions provides the configuration information for the chat requests. The VertexAiPaLm2ChatOptions.Builder is fluent options builder.

Low-level VertexAiPaLm2Api Client

VertexAiPaLm2Api 为 VertexAiPaLm2Api 聊天 API 提供轻量级 Java 客户端。

The VertexAiPaLm2Api provides is lightweight Java client for VertexAiPaLm2Api Chat API.

以下类图说明了 VertexAiPaLm2Api 聊天界面和构建模块:

Following class diagram illustrates the VertexAiPaLm2Api chat interfaces and building blocks:

vertex ai chat low level api

下面是一个简单的片段,说明如何以编程方式使用 API:

Here is a simple snippet how to use the api programmatically:

VertexAiPaLm2Api vertexAiApi = new VertexAiPaLm2Api(< YOUR PALM_API_KEY>);

// Generate
var prompt = new MessagePrompt(List.of(new Message("0", "Hello, how are you?")));

GenerateMessageRequest request = new GenerateMessageRequest(prompt);

GenerateMessageResponse response = vertexAiApi.generateMessage(request);

// Embed text
Embedding embedding = vertexAiApi.embedText("Hello, how are you?");

// Batch embedding
List<Embedding> embeddings = vertexAiApi.batchEmbedText(List.of("Hello, how are you?", "I am fine, thank you!"));