HuggingFace Chat

HuggingFace 推断端点允许你部署并将机器学习模型托管在云端，并通过 API 访问它们。

HuggingFace Inference Endpoints allow you to deploy and serve machine learning models in the cloud, making them accessible via an API.

Getting Started

有关HuggingFace推理端点的更多详细信息，请访问 here。

Further details on HuggingFace Inference Endpoints can be found here.

Prerequisites

添加 spring-ai-huggingface 依赖项：

Add the spring-ai-huggingface dependency:

<dependency>
   <groupId>org.springframework.ai</groupId>
   <artifactId>spring-ai-huggingface</artifactId>
</dependency>

你应该获取 HuggingFace API 密钥并将其设置为环境变量。

You should get your HuggingFace API key and set it as an environment variable

export HUGGINGFACE_API_KEY=your_api_key_here

  1. 参见 Dependency Management 部分，将 Spring AI BOM 添加到你的构建文件中。

Refer to the Dependency Management section to add the Spring AI BOM to your build file.

注意，此客户端实现尚无 Spring Boot Starter。

Note, there is not yet a Spring Boot Starter for this client implementation.

获取推理端点的端点 URL。您可以在推理端点的 UI here 上找到它。

Obtain the endpoint URL of the Inference Endpoint. You can find this on the Inference Endpoint’s UI here.

Making a call to the model

HuggingfaceChatClient client = new HuggingfaceChatClient(apiKey, basePath);
Prompt prompt = new Prompt("Your text here...");
ChatResponse response = client.call(prompt);
System.out.println(response.getGeneration().getText());

Example

使用 here 中的示例

Using the example found here

String mistral7bInstruct = """
        [INST] You are a helpful code assistant. Your task is to generate a valid JSON object based on the given information:
        name: John
        lastname: Smith
        address: #1 Samuel St.
        Just generate the JSON object without explanations:
        [/INST]""";
Prompt prompt = new Prompt(mistral7bInstruct);
ChatResponse aiResponse = huggingfaceChatClient.call(prompt);
System.out.println(response.getGeneration().getText());

将生成如下输出：

Will produce the output

{
    "name": "John",
    "lastname": "Smith",
    "address": "#1 Samuel St."
}