Add error handling, logging, and an example policy (#22)

Co-authored-by: Dennis Traub <[email protected]>

Author: DennisTraub

README.md

@@ -8,19 +8,26 @@ Welcome to the Java Foundation Model (FM) Playground, a sandbox for exploring Ja
 
 This repository includes a **Spring Boot** application and a **Next.js** frontend, both executable locally. Below is a screenshot of the app in action.
 
-![Screenshot of the Java FM Playground](screenshot.png)
+![Screenshot of the Java FM Playground](resources/screenshot.png)
+
+## ⚠ Important
+
+- Running this application might result in charges to your AWS account. For more details, see [Amazon Bedrock Pricing](https://aws.amazon.com/bedrock/pricing/).
+- This app is configured to run in `us-east-1` and has not been tested in every AWS Region. For more information see [AWS Regional Services](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services).
+- We recommend that you grant your code least privilege, i.e. only the minimum permissions required by the application. You can find an IAM Policy document with the required permissions in this repository at `resources/bedrock-access-policy.json` ([display policy](./resources/bedrock-access-policy.json)).
 
 ## Prerequisites
 
 Ensure you have the following installed:
 
 - Java JDK 17+ (e.g. [Amazon Corretto](https://aws.amazon.com/corretto), a free distro of the JDK)
-- [Apache Maven](https://maven.apache.org/install.html)
-- [Node.js (v18.17+)](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) with npm (for Next.js frontend)
-- An [AWS account](https://aws.amazon.com/free/) with permissions to access Amazon Bedrock
-- To use Bedrock, you must [enable access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html#add-model-access) to at least the following models in `us-east-1`: 
-  1. Anthropic: Claude
-  2. Stability AI: Stable Diffusion XL
+- [Apache Maven](https://maven.apache.org/install.html) for the backend server
+- [Node.js (v18.17+)](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) for the frontend application
+- You must have an [AWS account](https://aws.amazon.com/free/), and have your default credentials and AWS Region configured as described in the [AWS Tools and SDKs Shared Configuration and Credentials Reference Guide](https://docs.aws.amazon.com/credref/latest/refdocs/creds-config-files.html).
+- You must request access to the models before you can use them. For more information, see [Model access](https://docs.aws.amazon.com/bedrock/latest/userguide/model-access.html). To run the app, you need access to the following models in `us-east-1`:
+  - Anthropic Claude
+  - AI21 Labs Jurassic-2 Mid
+  - Stability AI Stable Diffusion XL
 
 ## Running the Application
 
@@ -42,18 +49,41 @@ In the `java-fm-playground/backend` directory, run:
 mvn spring-boot:run
 ```
 
+Once the server is running, you will see the following output:
+
+```shell
+...
+2023-11-13T14:19:02.862+01:00  INFO 34848 --- [main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 55500 (http) with context path ''
+2023-11-13T14:19:02.868+01:00  INFO 34848 --- [main] a.c.e.b.SpringFmPlaygroundApplication    : Started SpringFmPlaygroundApplication in 1.415 seconds (process running for 1.659)
+```
+
 > 🛠 The backend runs on port 55500 by default. See below for port changes.
 
-### Frontent Setup
+### Frontend Setup
 
-In a new terminal window, navigate to `java-fm-playground/frontend` and execute:
+In a **new terminal window**, navigate to the `frontend` directory and install the packages required by running the following command:
 
 ```shell
 npm install
+```
+
+After successful installation you can start the frontend application by executing the following command:
+
+```shell
 npm run dev
 ```
 
-> 🛠 The frontend runs on port 3000 by default. See below for port changes.
+When the application is running, you will see the following output:
+
+```shell
+> [email protected] dev
+> next dev
+
+   ▲ Next.js 14.0.0
+   - Local:        http://localhost:3000
+
+ ✓ Ready in 3.3s
+```
 
 ## Accessing the Application
 

backend/src/main/java/aws/community/examples/bedrock/controller/ChatPlayground.java

@@ -1,18 +1,25 @@
 package aws.community.examples.bedrock.controller;
 
 import aws.community.examples.bedrock.aimodels.Claude;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.HttpStatus;
 import org.springframework.web.bind.annotation.PostMapping;
 import org.springframework.web.bind.annotation.RequestBody;
 import org.springframework.web.bind.annotation.RestController;
+import org.springframework.web.server.ResponseStatusException;
 import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
+import software.amazon.awssdk.services.bedrockruntime.model.AccessDeniedException;
 
 import static aws.community.examples.bedrock.aimodels.LLM.Request;
 import static aws.community.examples.bedrock.aimodels.LLM.Response;
 
 @RestController
 public class ChatPlayground {
 
+    private static final Logger logger = LoggerFactory.getLogger(ChatPlayground.class);
+
     private final BedrockRuntimeClient client;
 
     @Autowired
@@ -22,16 +29,26 @@ public ChatPlayground(final BedrockRuntimeClient client) {
 
     @PostMapping("/foundation-models/model/chat/anthropic.claude-v2/invoke")
     public Response invoke(@RequestBody Request body) {
-        String  systemPrompt =
-                """
-                Take the role of a friendly chat bot. Your responses are brief.
-                You sometimes use emojis where appropriate, but you don't overdo it.
-                You engage human in a dialog by regularly asking questions,
-                except when Human indicates that the conversation is over.
-                """;
-
-        String prompt = systemPrompt + "\n\n" + body.prompt();
-
-        return new Response(Claude.invoke(client, prompt, 0.8, 300));
+        try {
+
+            String systemPrompt =
+                    """
+                            Take the role of a friendly chat bot. Your responses are brief.
+                            You sometimes use emojis where appropriate, but you don't overdo it.
+                            You engage human in a dialog by regularly asking questions,
+                            except when Human indicates that the conversation is over.
+                            """;
+
+            String prompt = systemPrompt + "\n\n" + body.prompt();
+
+            return new Response(Claude.invoke(client, prompt, 0.8, 300));
+
+        } catch (AccessDeniedException e) {
+            logger.error("Access Denied: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.FORBIDDEN);
+        } catch (Exception e) {
+            logger.error("Exception: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR);
+        }
     }
 }

backend/src/main/java/aws/community/examples/bedrock/controller/FoundationModels.java

@@ -1,17 +1,24 @@
 package aws.community.examples.bedrock.controller;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.HttpStatus;
 import org.springframework.web.bind.annotation.GetMapping;
 import org.springframework.web.bind.annotation.PathVariable;
 import org.springframework.web.bind.annotation.RestController;
+import org.springframework.web.server.ResponseStatusException;
 import software.amazon.awssdk.services.bedrock.BedrockClient;
 import software.amazon.awssdk.services.bedrock.model.*;
+import software.amazon.awssdk.services.bedrock.model.AccessDeniedException;
 
 import java.util.List;
 
 @RestController
 public class FoundationModels {
 
+    private static final Logger logger = LoggerFactory.getLogger(FoundationModels.class);
+
     private final BedrockClient client;
 
     @Autowired
@@ -21,6 +28,7 @@ public FoundationModels(final BedrockClient client) {
 
     @GetMapping("/foundation-models")
     public List<FoundationModelListItem> listFoundationModels() {
+        try {
 
         ListFoundationModelsRequest request = ListFoundationModelsRequest.builder().build();
         ListFoundationModelsResponse response = client.listFoundationModels(request);
@@ -32,31 +40,48 @@ public List<FoundationModelListItem> listFoundationModels() {
                         model.providerName()
                 )
         ).toList();
+
+        } catch (AccessDeniedException e) {
+            logger.error("Access Denied: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.FORBIDDEN);
+        } catch (Exception e) {
+            logger.error("Exception: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR);
+        }
     }
 
     public record FoundationModelListItem(String modelId, String modelName, String providerName) { }
 
     @GetMapping("/foundation-models/model/{modelId}")
-    public getFoundationModel getFoundationModel(@PathVariable String modelId) {
-
-        GetFoundationModelRequest request = GetFoundationModelRequest.builder()
-                .modelIdentifier(modelId)
-                .build();
-        GetFoundationModelResponse response = client.getFoundationModel(request);
-
-        FoundationModelDetails model = response.modelDetails();
-
-        return new getFoundationModel(
-                model.modelArn(),
-                model.modelId(),
-                model.modelName(),
-                model.providerName(),
-                String.join(", ", model.customizationsSupportedAsStrings()),
-                String.join(", ", model.outputModalitiesAsStrings())
-        );
+    public GetFoundationModel getFoundationModel(@PathVariable String modelId) {
+        try {
+
+            GetFoundationModelRequest request = GetFoundationModelRequest.builder()
+                    .modelIdentifier(modelId)
+                    .build();
+            GetFoundationModelResponse response = client.getFoundationModel(request);
+
+            FoundationModelDetails model = response.modelDetails();
+
+            return new GetFoundationModel(
+                    model.modelArn(),
+                    model.modelId(),
+                    model.modelName(),
+                    model.providerName(),
+                    String.join(", ", model.customizationsSupportedAsStrings()),
+                    String.join(", ", model.outputModalitiesAsStrings())
+            );
+
+        } catch (AccessDeniedException e) {
+            logger.error("Access Denied: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.FORBIDDEN);
+        } catch (Exception e) {
+            logger.error("Exception: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR);
+        }
     }
 
-    public record getFoundationModel(
+    public record GetFoundationModel(
             String modelArn,
             String modelId,
             String modelName,

backend/src/main/java/aws/community/examples/bedrock/controller/ImagePlayground.java

@@ -1,13 +1,20 @@
 package aws.community.examples.bedrock.controller;
 
 import aws.community.examples.bedrock.aimodels.StableDiffusion;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.HttpStatus;
 import org.springframework.web.bind.annotation.*;
+import org.springframework.web.server.ResponseStatusException;
 import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
+import software.amazon.awssdk.services.bedrockruntime.model.AccessDeniedException;
 
 @RestController
 public class ImagePlayground {
 
+    private static final Logger logger = LoggerFactory.getLogger(ImagePlayground.class);
+
     private final BedrockRuntimeClient client;
 
     @Autowired
@@ -17,8 +24,16 @@ public ImagePlayground(final BedrockRuntimeClient client) {
 
     @PostMapping ("/foundation-models/model/image/stability.stable-diffusion-xl/invoke")
     public StableDiffusion.Response invoke(@RequestBody StableDiffusion.Request body) {
+        try {
 
-        return StableDiffusion.invoke(client, body.prompt(), body.stylePreset());
+            return StableDiffusion.invoke(client, body.prompt(), body.stylePreset());
 
+        } catch (AccessDeniedException e) {
+            logger.error("Access Denied: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.FORBIDDEN);
+        } catch (Exception e) {
+            logger.error("Exception: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR);
+        }
     }
 }

backend/src/main/java/aws/community/examples/bedrock/controller/TextPlayground.java

@@ -2,19 +2,26 @@
 
 import aws.community.examples.bedrock.aimodels.Claude;
 import aws.community.examples.bedrock.aimodels.Jurassic2;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.HttpStatus;
 import org.springframework.web.bind.annotation.PathVariable;
 import org.springframework.web.bind.annotation.PostMapping;
 import org.springframework.web.bind.annotation.RequestBody;
 import org.springframework.web.bind.annotation.RestController;
+import org.springframework.web.server.ResponseStatusException;
 import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
+import software.amazon.awssdk.services.bedrockruntime.model.AccessDeniedException;
 
 import static aws.community.examples.bedrock.aimodels.LLM.Request;
 import static aws.community.examples.bedrock.aimodels.LLM.Response;
 
 @RestController
 public class TextPlayground {
 
+    private static final Logger logger = LoggerFactory.getLogger(TextPlayground.class);
+
     private final BedrockRuntimeClient client;
 
     @Autowired
@@ -24,19 +31,29 @@ public TextPlayground(final BedrockRuntimeClient client) {
 
     @PostMapping("/foundation-models/model/text/{modelId}/invoke")
     public Response invoke(@PathVariable String modelId, @RequestBody Request request) {
-        return switch (modelId) {
-
-            case Claude.MODEL_ID -> {
-                String completion = Claude.invoke(client, request.prompt(), request.temperature(), request.maxTokens());
-                yield new Response(completion);
-            }
-
-            case Jurassic2.MODEL_ID -> {
-                String completion = Jurassic2.invoke(client, request.prompt(), request.temperature(), request.maxTokens());
-                yield new Response(completion);
-            }
-
-            default -> throw new IllegalArgumentException("Unsupported model ID");
-        };
+        try {
+
+            return switch (modelId) {
+
+                case Claude.MODEL_ID -> {
+                    String completion = Claude.invoke(client, request.prompt(), request.temperature(), request.maxTokens());
+                    yield new Response(completion);
+                }
+
+                case Jurassic2.MODEL_ID -> {
+                    String completion = Jurassic2.invoke(client, request.prompt(), request.temperature(), request.maxTokens());
+                    yield new Response(completion);
+                }
+
+                default -> throw new IllegalArgumentException("Unsupported model ID");
+            };
+
+        } catch (AccessDeniedException e) {
+            logger.error("Access Denied: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.FORBIDDEN);
+        } catch (Exception e) {
+            logger.error("Exception: %s".formatted(e.getMessage()));
+            throw new ResponseStatusException(HttpStatus.INTERNAL_SERVER_ERROR);
+        }
     }
 }

resources/bedrock-access-policy.json

@@ -0,0 +1,18 @@
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": "bedrock:ListFoundationModels",
+      "Resource": "*"
+    },
+    {
+      "Effect": "Allow",
+      "Action": [
+        "bedrock:GetFoundationModel",
+        "bedrock:InvokeModel"
+      ],
+      "Resource": "*"
+    }
+  ]
+}