feat(tests): node examples use offline testing (#35)

* tests: update cors-node example to use serverless local testing

* tests: update typescript-with-node example to use serverless local testing

* tests: update node-upload-file-s3-multipart example to use serverless local testing

* tests: update postgre-sql-node example to use serverless local testing

* tests: update image-labelling-node example to use serverless local testing

* tests: update image-transform-node example to use serverless local testing

* style: always use sh to stay consistent for code in README (node examples only)

Author: Bemilie

Diff for: functions/cors-node/README.md

@@ -4,24 +4,66 @@ This function shows how to handle preflight CORS requests that will be sent by a
 
 The example uses totally permissive, open CORS, you may want to modify this to make it more secure.
 
-## Setup
+## Requirements
+
+This example assumes you are familiar with how serverless functions work. If needed, you can check [Scaleway's official documentation](https://www.scaleway.com/en/docs/serverless/functions/quickstart/)
+
+This example uses the Scaleway Serverless Framework Plugin. Please set up your environment with the requirements stated in the [Scaleway Serverless Framework Plugin](https://github.com/scaleway/serverless-scaleway-functions) before trying out the example.
 
-This example uses the [Scaleway Serverless Framework Plugin](https://github.com/scaleway/serverless-scaleway-functions).
+Additionnaly it uses the [serverless-functions-node](https://github.com/scaleway/serverless-functions-node) library for local testing.
 
-Once this is set up, you can run:
+## Context
 
-```console
+The function of the example allows most CORS requests. For more documentation on how to control access for CORS requests, see [HTTP response headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#the_http_response_headers)
+
+## Setup
+
+Once your environment is set up, you can run:
+
+```sh
 npm install
 
+NODE_ENV=test node handler.js
+```
+
+This will launch a local server, allowing you to test the function. For that, you can run in another terminal:
+
+The result should be similar to:
+
+```sh
+HTTP/1.1 200 OK
+access-control-allow-origin: *
+access-control-allow-headers: *
+access-control-allow-methods: *
+access-control-expose-headers: *
+content-type: text/plain
+content-length: 35
+Date: Mon, 17 Apr 2023 07:03:06 GMT
+Connection: keep-alive
+Keep-Alive: timeout=72
+server: envoy
+
+This is allowing most CORS requests
+```
+
+You can also check the result of your function in a browser. It should be "This is allowing most CORS requests".
+
+## Deploy and run
+
+Finally, if the test succeeded, you can deploy your function with:
+
+```sh
 serverless deploy
 ```
 
 Then, from the given URL, you can run:
 
-```console
+```sh
 # Options request
 curl -i -X OPTIONS <function URL>
 
 # Get request
 curl -i -X GET <function URL>
 ```
+
+When invoking this function, the output should be similar to the one obtained when testing locally.

Diff for: functions/cors-node/handler.js

@@ -35,3 +35,10 @@ const handleCorsVeryPermissive = (event, context, cb) => {
 };
 
 export { handleCorsPermissive, handleCorsVeryPermissive };
+
+/* This is used to test locally and will not be executed on Scaleway Functions */
+if (process.env.NODE_ENV === 'test') {
+  import("@scaleway/serverless-functions").then(scw_fnc_node => {
+    scw_fnc_node.serveHandler(handleCorsPermissive, 8080);
+  });
+}

Diff for: functions/cors-node/package.json

@@ -8,6 +8,7 @@
   "license": "ISC",
   "dependencies": {},
   "devDependencies": {
+    "@scaleway/serverless-functions": "^1.0.1",
     "serverless-scaleway-functions": "^0.4.4"
   },
   "description": ""

Diff for: functions/image-labelling-node/.gitignore

@@ -0,0 +1,3 @@
+node_modules/
+package-lock.json
+.serverless/

Diff for: functions/image-labelling-node/README.md

@@ -1,20 +1,23 @@
 # Image labeling example with TensorFlow.js
 
+This example shows how to label an RGB image in an S3 bucket using serverless functions.
+
 ## Requirements
 
 This example assumes that you are familiar with:
- * how serverless functions work. If needed, you can check [Scaleway official documentation](https://www.scaleway.com/en/docs/serverless/functions/quickstart/).
- * how S3 object storage works, and especially how to create a bucket and upload files within a bucket. Please refer to scaleway's documentation [here](https://www.scaleway.com/en/docs/storage/object/quickstart/).
- * how to create and get user API access and secret keys using IAM. Please refer to IAM documentation [here](https://www.scaleway.com/en/docs/identity-and-access-management/iam/concepts/).
+
+* how serverless functions work. If needed, you can check [Scaleway official documentation](https://www.scaleway.com/en/docs/serverless/functions/quickstart/).
+* how S3 object storage works, and especially how to create a bucket and upload files within a bucket. Please refer to scaleway's documentation [here](https://www.scaleway.com/en/docs/storage/object/quickstart/).
+* how to create and get user API access and secret keys using IAM. Please refer to IAM documentation [here](https://www.scaleway.com/en/docs/identity-and-access-management/iam/concepts/).
 
 This example uses the Scaleway Serverless Framework Plugin. Please set up your environment with the requirements stated in the [Scaleway Serverless Framework Plugin](https://github.com/scaleway/serverless-scaleway-functions) before trying out the example.
 
+Additionnaly it uses the [serverless-functions-node](https://github.com/scaleway/serverless-functions-node) library for local testing.
 
 ## Context
 
 This example shows how to label an RGB image in an S3 bucket using serverless functions. The example uses a pre-trained ready-to-use model from [TensorFlow.js](https://www.tensorflow.org/js/models). The model is called `mobilenet` and can be used on server or client side. It returns three labels of an image with their respective prediction probabilities (namely, logits). Check `mobilenet` Github repository [here](https://github.com/tensorflow/tfjs-models/tree/master/mobilenet).
 
-
 ## Description
 
 The function gets an RGB image (jpg, jpeg, or png formats) from an S3 bucket, transforms it into a TensorFlow-compatible object (namely, a tensor), and then applys a pretrained `mobilenet` model to return the labels and their respective logits in a json format.
@@ -29,40 +32,40 @@ Create an S3 bucket and upload an RGB image (jpg, jpeg or png format) within thi
 
 ### Fill environment variables
 
-Fill your environment variables within `serverless.yml` file:
+Ensure to create a bucket and have the following secrets variables available in your environment (to be able to test locally) and within `serverless.yml` file (to be able to deploy):
 
-```
+```yml
 secret:
     USER_ACCESS_KEY: <bucket scw access key>
     USER_SECRET_KEY: <bucket scw access key id>
-    S3_ENDPOINT_URL: http://s3.fr-par.scw.cloud
+    S3_ENDPOINT_URL: s3.fr-par.scw.cloud
 ```
 
 ### Install npm modules
 
 Once your environment is set up, you can install `npm` dependencies from `package.json` file using:
 
-```
+```sh
 npm install
 ```
 
-### Deploy and call the function
+### Test locally
 
-Then deploy your function and get its URL using:
+Once your environment is set up, you can test your function locally with:
 
-```
-serverless deploy
+```sh
+NODE_ENV=test node handler.js
 ```
 
-From the given function URL, you can get image labels in a `json` format:
+This will launch a local server, allowing you to test the function. For that, you can run in another terminal:
 
-```
-curl -X GET "<function URL>/?sourceBucket=<source bucket name>&sourceKey=<file name within bucket>" | jq
+```sh
+curl -X GET "http://localhost:8080?sourceBucket=<source bucket name>&sourceKey=<filename within bucket>"
 ```
 
-The result should be similar to:
+The output should be similar to:
 
-```
+```json
 {
   "labels": [
     {
@@ -79,7 +82,22 @@ The result should be similar to:
     }
   ]
 }
-
 ```
 
 You can also check the result of your function in a browser. It should return the same result.
+
+### Deploy and call the function
+
+Finally, if the test succeeded, you can deploy your function with:
+
+```sh
+serverless deploy
+```
+
+Then, from the given URL, you can run:
+
+```sh
+curl -X GET "<function url>?sourceBucket=<source bucket name>&sourceKey=<filename within bucket>"
+```
+
+When invoking this function, the output should be similar to the one obtained when testing locally.

Diff for: functions/image-labelling-node/handler.js

@@ -9,16 +9,12 @@ const userSecretKey = process.env.USER_SECRET_KEY;
 
 async function handle(event, context, callback) {
 
-  // useful for function logging
-  console.log(event)
-  console.log(context)
-
   const sourceBucket = event.queryStringParameters.sourceBucket;
   const sourceKey = event.queryStringParameters.sourceKey;
 
   const response = {
     statusCode: 200,
-    headers: { "Content-Type": ["application/json"] },
+    headers: { "Content-Type": "application/json" },
     body: {
       labels: await classifyImage(imageToTensor(await getImageFromS3(sourceBucket, sourceKey, s3EndpointUrl, userAccessKey, userSecretKey)))
     }
@@ -28,7 +24,6 @@ async function handle(event, context, callback) {
 
 };
 
-
 async function getImageFromS3(sourceBucket, sourceKey, s3EndpointUrl, userAccessKey, userSecretKey) {
 
   const s3 = new S3({
@@ -90,3 +85,10 @@ async function classifyImage(imageTensor) {
 }
 
 export { handle }
+
+/* This is used to test locally and will not be executed on Scaleway Functions */
+if (process.env.NODE_ENV === 'test') {
+  import("@scaleway/serverless-functions").then(scw_fnc_node => {
+    scw_fnc_node.serveHandler(handle, 8080);
+  });
+}

Diff for: functions/image-labelling-node/package.json

@@ -15,6 +15,7 @@
         "aws-sdk": "^2.1314.0"
     },
     "devDependencies": {
+        "@scaleway/serverless-functions": "^1.0.1",
         "serverless-scaleway-functions": "^0.4.5"
     },
     "description": "",

Diff for: functions/image-transform-node/.gitignore

@@ -0,0 +1,3 @@
+node_modules/
+package-lock.json
+.serverless/

Diff for: functions/image-transform-node/BucketScan.js

@@ -3,7 +3,6 @@ const https = require("https");
 const { isNull } = require("util");
 
 // Get info from env variables
-
 const SOURCE_BUCKET = process.env.SOURCE_BUCKET;
 const S3_ENDPOINT_URL = process.env.S3_ENDPOINT_URL;
 const ID = process.env.ACCESS_KEY_ID;
@@ -78,3 +77,10 @@ exports.handle = async (event, context, callback) => {
     }
   });
 };
+
+/* This is used to test locally and will not be executed on Scaleway Functions */
+if (process.env.NODE_ENV === 'test') {
+  import("@scaleway/serverless-functions").then(scw_fnc_node => {
+    scw_fnc_node.serveHandler(exports.handle, 8080);
+  });
+}

Diff for: functions/image-transform-node/ImageTransform.js

@@ -3,13 +3,14 @@ const AWS = require("aws-sdk");
 const sharp = require("sharp");
 
 // Get connexion information from secret environment variables
-const srcBucket = process.env.SOURCE_BUCKET;
+const SOURCE_BUCKET = process.env.SOURCE_BUCKET;
 const S3_ENDPOINT_URL = process.env.S3_ENDPOINT_URL;
 const ID = process.env.ACCESS_KEY_ID;
 const SECRET = process.env.ACCESS_KEY;
-const dstBucket = process.env.DESTINATION_BUCKET;
+const DEST_BUCKET = process.env.DESTINATION_BUCKET;
+const RESIZED_WIDTH = process.env.RESIZED_WIDTH;
 
-let width = parseInt(process.env.RESIZED_WIDTH, 10);
+let width = parseInt(RESIZED_WIDTH, 10);
 if (width < 1 || width > 1000) {
   width = 200;
 }
@@ -49,7 +50,7 @@ exports.handle = async (event, context, callback) => {
   // Download the image from the S3 source bucket.
   try {
     const params = {
-      Bucket: srcBucket,
+      Bucket: SOURCE_BUCKET,
       Key: srcKey,
     };
     var origimage = await s3.getObject(params).promise();
@@ -71,7 +72,7 @@ exports.handle = async (event, context, callback) => {
   // Upload the image to the destination bucket
   try {
     const destparams = {
-      Bucket: dstBucket,
+      Bucket: DEST_BUCKET,
       Key: dstKey,
       Body: buffer,
       ContentType: "image",
@@ -83,11 +84,11 @@ exports.handle = async (event, context, callback) => {
   }
   console.log(
     "Successfully resized " +
-    srcBucket +
+    SOURCE_BUCKET +
     "/" +
     srcKey +
     " and uploaded to " +
-    dstBucket +
+    DEST_BUCKET +
     "/" +
     dstKey
   );
@@ -99,10 +100,17 @@ exports.handle = async (event, context, callback) => {
         "Image : " +
         srcKey +
         " has successfully been resized and pushed to the bucket " +
-        dstBucket,
+        DEST_BUCKET,
     }),
     headers: {
       "Content-Type": "application/json",
     },
   };
 };
+
+/* This is used to test locally and will not be executed on Scaleway Functions */
+if (process.env.NODE_ENV === 'test') {
+  import("@scaleway/serverless-functions").then(scw_fnc_node => {
+    scw_fnc_node.serveHandler(exports.handle, 8081);
+  });
+}