Introduction

The rise of Large Language Models (LLMs) has opened new doors for AI-powered applications, enabling dynamic interactions, natural language processing, and retrieval-augmented generation (RAG). However, integrating these powerful models into Java applications can be challenging. This is where LangChain4J comes in – a framework designed to simplify AI development in Java.

To take things a step further, in version 4.2, Helidon introduced seamless LangChain4J integration, making it easier to build AI-driven applications while leveraging Helidon’s programming model and style. In this blog post, we’ll explore how this integration simplifies AI application development and how you can use it in your projects.

What is LangChain4J?

LangChain4J is a Java framework that facilitates building AI-powered applications using LLMs from providers like OpenAI, Cohere, Hugging Face, and others. It provides:

• AI Services: A declarative and type-safe API to interact with models.
• Retrieval-Augmented Generation (RAG): Enhancing responses with external knowledge sources.
• Embeddings and Knowledge Retrieval: Working with vector-based search systems.
• Memory and Context: Managing conversational memory for intelligent interactions.

However, integrating LangChain4J manually into an application requires configuring components, managing dependencies, and handling injections manually. This is where Helidon’s integration module provides significant advantages.

How Helidon Simplifies LangChain4J Integration

Before we proceed, note that Helidon’s LangChain4J integration is a preview feature in Helidon 4.2. This means that while it’s production-ready, the Helidon team reserves the right to modify APIs in minor versions.

Helidon’s LangChain4J integration introduces:

• Helidon Inject Support: LangChain4J components are automatically created and registered in the Helidon service registry based on configuration.
• Convention Over Configuration: Reduces boilerplate by using sensible defaults.
• Declarative AI Services: Uses annotations to define AI services in a clean, structured manner.
• CDI Integration: Components work seamlessly in Helidon MP.

These features significantly reduce the complexity of incorporating AI into Helidon applications.

Setting Up LangChain4J in Helidon

To use LangChain4J with Helidon, add the following dependency to your Maven project:

<dependency>
    <groupId>io.helidon.integrations.langchain4j</groupId>
    <artifactId>helidon-integrations-langchain4j</artifactId>
</dependency>

Include the necessary annotation processors in the <build><plugins> section of your pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <configuration>
        <annotationProcessorPaths>
            <path>
                <groupId>io.helidon.codegen</groupId>
                <artifactId>helidon-codegen-apt</artifactId>
            </path>
            <path>
                <groupId>io.helidon.integrations.langchain4j</groupId>
                <artifactId>helidon-integrations-langchain4j-codegen</artifactId>
            </path>
        </annotationProcessorPaths>
    </configuration>
</plugin>

Using different LLM providers may require additional dependencies. For example, using OpenAI models requires adding a dependency to the OpenAI provider, while using Ollama requires adding the Ollama provider. This modular approach helps keep applications lightweight. For more information about providers and dependencies, refer to the documentation.

Creating AI Components in Helidon

When I refer to AI components, I mean the public API classes provided by LangChain4J. Examples include various models such as OpenAiChatModel and OllamaChatModel, embedding stores like InMemoryEmbeddingStore, content retrievers such as EmbeddingStoreContentRetriever, and ingestors like EmbeddingStoreIngestor, among others.

Some components are natively supported by the Helidon LangChain4J integration and can be automatically created when the corresponding configuration is present. The currently supported components include:

• LangChain4J Core
  • EmbeddingStoreContentRetriever
  • MessageWindowChatMemory
• Open AI
  • OpenAiChatModel
  • OpenAiStreamingChatModel
  • OpenAiEmbeddingModel
  • OpenAiImageModel
  • OpenAiLanguageModel
  • OpenAiModerationModel
• Ollama
  • OllamaChatModel
  • OllamaStreamingChatModel
  • OllamaEmbeddingModel
  • OllamaLanguageModel
• Cohere
  • CohereEmbeddingModel
  • CohereScoringModel
• Oracle
  • OracleEmbeddingStore

For example, the OpenAI chat model can be automatically created by defining the following configuration in application.yaml:

langchain4j:
  open-ai:
    chat-model:
      enabled: true
      api-key: "demo"
      model-name: "gpt-4o-mini"

With this configuration, an instance of OpenAiChatModel is automatically created and can be injected into other application components.

A key element in this setup is the enabled property. The component is only created if this property is set to true. This provides an easy way to disable component creation while retaining its configuration in the file for future use.

If you need to create a component that is not in the list above and register it in the Helidon service registry, you can use a Supplier Factory:

@Service.Singleton
@Service.Named("MyChatModel")
class ChatModelFactory implements Supplier<ChatLanguageModel> {
    @Override
    public ChatLanguageModel get() {
        return OpenAiChatModel.builder()
                .apiKey("demo")
                .build();
    }
}

This method allows you to register custom embedding models and other AI components dynamically. Service.Named("MyChatModel") annotation is optional. You can add it to define a name for your component for future reference.

Using AI Components

Helidon Inject makes it easy to use AI components within your application:

@Service.Singleton
public class MyService {
    private final ChatLanguageModel chatModel;

    @Service.Inject
    public MyService(ChatLanguageModel chatModel) {
        this.chatModel = chatModel;
    }
}

For named components, use:

@Service.Inject
public MyService(@Service.Named("MyChatModel") ChatLanguageModel chatModel) {
    this.chatModel = chatModel;
}

Alternatively, you can manually retrieve a component from the service registry as follows:

var chatModel = Services.get(OpenAiChatModel.class);

AI Services

Most often AI-powered components require a combination of different components working together. For example, a simple chat assistant requires a chat model to communicate with users, embedding store to store data, embedding model to retrieve and query the data, chat memory for keeping conversation context, etc. LangChain4J AI Services provide a way of combining different kinds of AI functionality behind a simple API which significantly reduces the boilerplate code.

Helidon’s LangChain4J integration introduces a declarative Helidon Inject-based approach for creating AI Services. It supports the following components:

• Chat Model:
  • dev.langchain4j.model.chat.ChatLanguageModel
• Streaming Chat Model:
  • dev.langchain4j.model.chat.StreamingChatLanguageModel
• Chat Memory:
  • dev.langchain4j.memory.ChatMemory
• Chat Memory Provider:
  • dev.langchain4j.memory.chat.ChatMemoryProvider
• Moderation Model:
  • dev.langchain4j.model.moderation.ModerationModel
• RAG:
  • Content Retriever: dev.langchain4j.rag.content.retriever.ContentRetriever
  • Retrieval Augmentor: dev.langchain4j.rag.RetrievalAugmentor
• Callback Functions:
  • Methods annotated with dev.langchain4j.agent.tool.Tool

Helidon makes it simple to define AI services using annotations:

@Ai.Service
public interface ChatAiService {
    String chat(String question);
}

In this scenario all LangChain4J components from the list above are taken from the service registry. User still has an ability to manually control the process by putting any of the following annotations which specify a service name which must be used for this particular function instead of discovering it automatically.

For example, in the snippet below a service with name “MyChatModel” will be used as chat model and all other components are discovered automatically.

@Ai.Service
@Ai.ChatModel("MyChatModel")
public interface ChatAiService {
    String chat(String question);
}

There is a possibility to switch off automatic discovery by using @Ai.Service(autodicovery=false). In this case the service components are not discovered automatically and users must add components manually using annotations specified above. @ChatModel or @StreamingChatModel annotations are required.

Tools: Enhancing AI with Custom Logic

LangChain4J tools enable AI models to invoke external functions during execution. This is useful when an LLM needs to perform an action during its conversation with a user, such as retrieving data, calling an external service, or executing code.

LangChain4J provides the @Tool annotation, which, when applied to a method, makes that method accessible for the LLM to call. The integration code scans the project for classes containing @Tool-annotated methods and automatically adds them to AI Services. The only requirement is that these classes must be Helidon Inject services.

@Service.Singleton
public class OrderService {
    @Tool("Get order details for specified order number")
    public Order getOrderDetails(String orderNumber) {
        // Business logic here
    }
}

If you are using Helidon MP, an additional step is required. You must annotate the service containing tools with the @Ai.Tool annotation. Additionally, the integration supports tools within CDI beans.

Samples

We have created several sample applications for you to explore. These samples demonstrate all aspects of using LangChain4J in Helidon applications.

Coffee Shop Assistant

The Coffee Shop Assistant is a demo application that showcases how to build an AI-powered assistant for a coffee shop. This assistant can answer questions about the menu, provide recommendations, and create orders. It utilizes an embedding store initialized from a JSON file.

Key features:

• Integration with OpenAI chat models
• Utilization of embedding models, an embedding store, an ingestor, and a content retriever
• Helidon Inject for dependency injection
• Embedding store initialization from a JSON file
• Support for callback functions to enhance interactions

Check it out:

• Coffee Shop Assistant for Helidon SE
• Coffee Shop Assistant for Helidon MP

Hands-on Lab

We also offer a Hands-on Lab with step-by-step instructions on how to build the Coffee Shop Assistant:

• HOL: Building AI-Powered Applications with Helidon and LangChain4J

Useful Resources

Ready to get started? Here are some useful resources:

• Helidon LangChain4J documentation
• LangChain4J documentation
• Helidon Inject documentation