Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for inbound webhooks that use form data #1998

Merged
merged 4 commits into from
Jul 19, 2024

Conversation

BolajiOlajide
Copy link
Contributor

Closes #1987

This PR adds support for inbound webhooks that send data via form-data. It defaults to JSON when the content type isn't multipart/form-data to avoid introducing a breaking change. As time goes one, we'll want to add support for www-url-encoded.

CleanShot 2024-05-07 at 01 52 44@2x
CleanShot 2024-05-07 at 01 52 38@2x

@CLAassistant
Copy link

CLAassistant commented May 7, 2024

CLA assistant check
All committers have signed the CLA.

@jirevwe
Copy link
Collaborator

jirevwe commented May 15, 2024

Shouldn't the JOSN payload received at the endpoint be this?

{
    "name": "legbegbe",
    "error": "no error",
    "status": "success"
}

@BolajiOlajide
Copy link
Contributor Author

BolajiOlajide commented May 15, 2024

Shouldn't the JOSN payload received at the endpoint be this?

{
    "name": "legbegbe",
    "error": "no error",
    "status": "success"
}

@jirevwe You're right. However, I did it that way because the return type of FormData is actually a map of the structure type Values map[string][]string. I left it that way to preserve the parsing incase something changes, WDYT?

@BolajiOlajide
Copy link
Contributor Author

@jirevwe Updated.

CleanShot 2024-05-17 at 01 51 30@2x
CleanShot 2024-05-17 at 01 51 21@2x

@jirevwe
Copy link
Collaborator

jirevwe commented May 22, 2024

It works but struggles with nested values.

This

curl -L -X POST 'http://localhost:5005/ingest/bRR0NEtU6sw0ZeD3' \
-H 'Authorization: Bearer CO.Ci4jlWTc4gJeWnXZ.NatfMfQM3Ks78Y3Ibm6XKqwCLS1eG35kNwcRe6jd42mOc0EdzBu5JCl9qZHJhSBE' \
-F 'endpoint_id="01HS16E631RTHZ2EN4K54ZFC9H"' \
-F 'data[event_type]="created"' \
-F 'data[name]="digital"' \
-F 'data[email]="[email protected]"' \
-F 'data[age]="10"' \
-F 'custom_headers[id]="2aff13ec-a373-4218-9fde-0c1328801291"' \
-F 'custom_headers[timestamp]="2024-03-20T06:56:06.991Z"' \
-F 'custom_headers[x-tenant-id]="xpi"' \
-F 'idempotency_key="test123"' \
-F 'event_type="created"'

comes out as

{
    "data": {
        "data[age]": "10",
        "data[name]": "auxiliary",
        "event_type": "created",
        "data[email]": "[email protected]",
        "endpoint_id": "01HS16E631RTHZ2EN4K54ZFC9H",
        "idempotency_key": "test123",
        "data[event_type]": "created",
        "custom_headers[id]": "2aff13ec-a373-4218-9fde-0c1328801291",
        "custom_headers[timestamp]": "2024-03-20T06:56:06.991Z",
        "custom_headers[x-tenant-id]": "xpi"
    },
    "event_type": "created",
    "endpoint_id": "01HS6EX2KC8FMN8NDTMNKQQN13",
    "custom_headers": {
        "header": "valve"
    }
}

@BolajiOlajide
Copy link
Contributor Author

@jirevwe

Are you sure you're on the correct branch, I tried this using the same payload and I get this:
CleanShot 2024-05-24 at 10 28 13@2x
CleanShot 2024-05-24 at 10 29 11@2x

@BolajiOlajide
Copy link
Contributor Author

@jirevwe Gentle nudge on this. :)

@jirevwe jirevwe merged commit 53dd447 into frain-dev:main Jul 19, 2024
4 checks passed
jirevwe added a commit that referenced this pull request Jul 22, 2024
* fix: fixed api migration bug (#2087)

Co-authored-by: Raymond Tukpe <[email protected]>

* chore: update changelog (#2093)

* chore: update ui dependencies (#2097)

* feat: Add support for inbound webhooks that use form data (#1998)

* accept multipart/form-data content type during ingestion

* change logic for handling form data

* fix failing test

* added e2e integration test suite (#2083)

* added e2e broadcast integration test

* updated e2e broadcast integration test

* removed unneeded files

* WIP: integrated convoy sdk

* added direct event tests

* added fan-out event tests

* updated e2e tests

* updated go version to 1.21

* revert retention policy

* switched to agent cmd

* updated tests

* updated tests with negative scenarios

* added integration test flag

* removed pointer from chan

* reverted typescript version change

* use correct time for delivery attempt

---------

Co-authored-by: Subomi Oluwalana <[email protected]>
Co-authored-by: Raymond Tukpe <[email protected]>
Co-authored-by: Bolaji Olajide <[email protected]>
Co-authored-by: Smart Mekiliuwa <[email protected]>
@BolajiOlajide BolajiOlajide deleted the bo/accept-form-data-ingestions branch August 12, 2024 00:22
Oluwadaminiola pushed a commit that referenced this pull request Oct 1, 2024
* accept multipart/form-data content type during ingestion

* change logic for handling form data

* fix failing test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature request]: Add support for inbound webhooks that use form data
4 participants