developing-genkit-java

# Developing with Genkit Java You are an expert on **Genkit Java**, the open-source Java AI framework by Google. This skill covers the full framework: architecture, plugin system, AI abstractions, and contribution guidelines. ## Project Architecture Genkit Java is a Maven multi-module project requiring **Java 21+**. ### Module Hierarchy ``` genkit-java/ ├── pom.xml # Parent POM (dependency management, plugins) ├── core/ # Foundational abstractions (Action, Flow, Registry, Plugin, Middleware, Tracing) │ └── com.google.genkit.core ├── ai/ # AI-specific abstractions (Model, Tool, Embedder, Retriever, Indexer, Message, Part) │ └── com.google.genkit.ai ├── genkit/ # High-level user-facing API (Genkit class, Prompts, Sessions, Agents, Evaluators) │ └── com.google.genkit ├── plugins/ # Provider integrations (21 plugins) │ └── com.google.genkit.plugins.{name} └── samples/ # Example applications (20+ samples) └── com.google.genkit.samples ``` ### Dependency Flow ``` core ← ai ← genkit ← plugins ← samples ``` - **core** has zero Genkit internal dependencies. It depends on Jackson, SLF4J, OpenTelemetry, victools JSON Schema. - **ai** depends on core. - **genkit** depends on core + ai + Handlebars (for .prompt files). - **plugins** depend on genkit (or ai/core). - **samples** depend on genkit + chosen plugins. ### Key Dependencies (Managed in Parent POM) | Library | Version | Purpose | |---------|---------|---------| | Jackson | 2.21.2 | JSON serialization (databind, annotations, jsr310) | | SLF4J | 2.0.17 | Logging facade | | Logback | 1.5.32 | Logging implementation | | OkHttp | 5.3.2 | HTTP client + SSE streaming | | OpenTelemetry | 1.60.1 | Tracing and metrics | | Handlebars | 4.5.0 | .prompt file templating | | victools | 4.38.0 | JSON Schema generation from Java classes | | JUnit | 6.0.3 | Testing framework | | Mockito | 5.23.0 | Mocking framework | --- ## Core Abstractions ### Action — The Universal Unit Every capability in Genkit is an `Action<I, O, S>`: - `I` = Input type - `O` = Output type - `S` = Streaming chunk type (`Void` for non-streaming) ```java public interface Action<I, O, S> extends Registerable { String getName(); ActionType getType(); O run(ActionContext ctx, I input); O run(ActionContext ctx, I input, Consumer<S> streamCallback); } ``` All AI primitives (Model, Tool, Embedder, Retriever, Indexer, Flow) implement `Action`. Actions self-register with the `Registry` using keys in the format `{type}/{name}` (e.g., `model/openai/gpt-4o`, `flow/myFlow`, `tool/getWeather`). ### ActionType Enum ```java RETRIEVER, INDEXER, EMBEDDER, EVALUATOR, FLOW, MODEL, BACKGROUND_MODEL, EXECUTABLE_PROMPT, PROMPT, RESOURCE, TOOL, TOOL_V2, UTIL, CUSTOM, CHECK_OPERATION, CANCEL_OPERATION ``` ### ActionContext Passed to every action execution. Carries tracing info, registry access, session state: ```java public class ActionContext { SpanContext spanContext; String flowName; Registry registry; String sessionId; } ``` ### Registry Centralized action discovery and lookup: ```java registry.registerAction(key, action); registry.lookupAction("model/openai/gpt-4o"); registry.lookupAction(ActionType.FLOW, "myFlow"); ``` --- ## The Genkit Class — Main Entry Point The `Genkit` class is the high-level API. Always use the builder pattern: ```java Genkit genkit = Genkit.builder() .options(GenkitOptions.builder() .devMode(true) .reflectionPort(3100) .build()) .plugin(new OpenAIPlugin()) .plugin(new JettyPlugin()) .build(); ``` ### Lifecycle 1. `Genkit.builder()...build()` — creates instance, and initialize 2. `genkit.stop()` — cleanup resources --- ## Defining Flows Flows are user-defined actions exposed as HTTP endpoints: ```java // Simple (no context needed) Flow<String, String, Void> greetFlow = genkit.defineFlow( "greet", String.class, String.class, (name) -> "Hello, " + name + "!"); // With ActionContext (for nested AI calls) Flow<String, String, Void> jokeFlow = genkit.defineFlow( "tellJoke", String.class, String.class, (ctx, topic) -> { ModelResponse response = genkit.generate( GenerateOptions.builder() .model("openai/gpt-4o-mini") .prompt("Tell a joke about: " + topic) .config(GenerationConfig.builder().temperature(0.9).build()) .build()); return response.getText(); }); // With middleware Flow<String, String, Void> securedFlow = genkit.defineFlow( "secured", String.class, String.class, (ctx, input) -> processInput(input), List.of(authMiddleware, loggingMiddleware)); ``` --- ## Generation API ### Simple Generation ```java ModelResponse response = genkit.generate( GenerateOptions.builder() .model("openai/gpt-4o") .prompt("Explain quantum computing") .build()); String text = response.getText(); ``` ### Streaming Generation ```java ModelResponse response = genkit.generateStream( GenerateOptions.builder() .model("openai/gpt-4o") .prompt("Write a story") .build(), chunk -> System.out.print(chunk.getText())); ``` ### Structured Output ```java MyPojo result = genkit.generateObject( GenerateOptions.<MyPojo>builder() .model("openai/gpt-4o") .prompt("Generate a recipe for pasta") .outputClass(MyPojo.class) .build()); ``` ### Multi-turn Messages ```java ModelResponse response = genkit.generate( GenerateOptions.builder() .model("openai/gpt-4o") .messages(List.of( Message.system("You are a helpful assistant."), Message.user("What is the capital of France?"), Message.model("Paris is the capital of France."), Message.user("What about Germany?"))) .build()); ``` ### GenerationConfig ```java GenerationConfig.builder() .temperature(0.9) .maxOutputTokens(2048) .topK(40) .topP(0.95) .stopSequences(List.of("\n\n")) .build() ``` --- ## Tools — AI-Callable Functions Define tools that models can invoke: ```java // With auto-generated JSON Schema from classes Tool<WeatherInput, WeatherOutput> weatherTool = genkit.defineTool( "getWeather", "Get current weather for a location", (ctx, input) -> fetchWeather(input.getLocation()), WeatherInput.class, WeatherOutput.class); // Use tools in generation ModelResponse response = genkit.generate( GenerateOptions.builder() .model("openai/gpt-4o") .prompt("What's the weather in London?") .tools(List.of(weatherTool)) .build()); ``` --- ## RAG (Retrieval-Augmented Generation) ### Embedding ```java EmbedResponse embeddings = genkit.embed( "openai/text-embedding-3-small", List.of(Document.fromText("Hello world"))); ``` ### Indexing ```java genkit.index("devLocalVectorStore/my-index", documents); ``` ### Retrieval + Generation ```java List<Document> context = genkit.retrieve("devLocalVectorStore/my-index", query); ModelResponse response = genkit.generate( GenerateOptions.builder() .model("openai/gpt-4o") .prompt(query) .docs(context) // Inject retrieved documents .build()); ``` --- ## DotPrompt — .prompt Files Prompt files live in `resources/prompts/` with Handlebars templates and YAML frontmatter: ``` --- model: openai/gpt-4o-mini config: temperature: 0.9 maxOutputTokens: 500 input: schema: ingredient: string style?: string --- Create a recipe using {{ingredient}} in a {{style}} style. ``` ### Loading Prompts ```java ExecutablePrompt<RecipeInput> prompt = genkit.prompt("recipe", RecipeInput.class); // With variant (recipe.robot.prompt) ExecutablePrompt<RecipeInput> robotPrompt = genkit.prompt("recipe", RecipeInput.class, "robot"); ``` --- ## Sessions & Chat ```java Session<MyState> session = genkit.createSession(); Chat<MyState> chat = genkit.chat(ChatOptions.<MyState>builder() .model("openai/gpt-4o") .session(session) .build()); ``` --- ## Agents Multi-agent systems with tool delegation: ```java Agent researchAgent = genkit.defineAgent(AgentConfig.builder() .name("researcher") .model("openai/gpt-4o") .description("Research specialist") .tools(List.of(searchTool, summarizeTool)) .build()); ``` --- ## Interrupts — Human-in-the-Loop ```java Tool<ConfirmInput, ConfirmOutput> confirmTool = genkit.defineInterrupt( InterruptConfig.<ConfirmInput, ConfirmOutput>builder() .name("confirmAction") .inputClass(ConfirmInput.class) .outputClass(ConfirmOutput.class) .build()); ``` --- ## Evaluators ```java Evaluator<String> factualityEval = genkit.defineEvaluator( "factuality", "Factuality Check", "Checks factual accuracy", (datapoint) -> { // Return EvalResponse with score, rationale, detail }); EvalRunKey result = genkit.evaluate(RunEvaluationRequest.builder() .evaluators(List.of("factuality")) .dataset(dataset) .build()); ``` --- ## Plugin Development ### The Plugin Interface ```java public interface Plugin { String getName(); List<Action<?, ?, ?>> init(); default List<Action<?, ?, ?>> init(Registry registry) { return init(); } } ``` ### Creating a New Plugin 1. **Create module** under `plugins/{name}/` with its own `pom.xml`. 2. **Package**: `com.google.genkit.plugins.{name}` 3. **Implement** `Plugin` interface. 4. **Return actions** from `init()` — Models, Embedders, Tools, Retrievers, etc. ### Standard Plugin Structure ``` plugins/my-provider/ ├── pom.xml ├── README.md └── src/main/java/com/google/genkit/plugins/my_provider/ ├── MyProviderPlugin.java # Plugin entry point ├── MyProviderPluginOptions.java # Configuration POJO (builder pattern) ├── MyProviderModel.java # Model implementation ├── MyProviderEmbedder.java # Embedder (if applicable) └── ... ``` ### Plugin Implementation Pattern ```java public class MyProviderPlugin implements Plugin { public static final List<String> SUPPORTED_MODELS = List.of("model-a", "model-b"); private final MyProviderPluginOptions options; public MyProviderPlugin(MyProviderPluginOptions options) { this.options = options; } public static MyProviderPlugin create() { return new MyProviderPlugin(MyProviderPluginOptions.builder().build()); } @Override public String getName() { return "my-provider"; } @Override public List<Action<?, ?, ?>> init() { List<Action<?, ?, ?>> actions = new ArrayList<>(); for (String model : SUPPORTED_MODELS) { actions.add(new MyProviderModel( getName() + "/" + model, model, options)); } return actions; } } ```