Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bluesky Social (Independent Publisher) #3706

Open
wants to merge 88 commits into
base: dev
Choose a base branch
from

Conversation

krautrocker
Copy link
Contributor

@krautrocker krautrocker commented Nov 10, 2024


When submitting a connector, please make sure that you follow the requirements below, otherwise your PR might be rejected. We want to make you have a well-built connector, a smooth certification experience, and your users are happy :)

If this is your first time submitting to GitHub and you need some help, please sign up for this session.

  • I attest that the connector doesn't exist on the Power Platform today. I've verified by checking the pull requests in GitHub and by searching for the connector on the platform or in the documentation.
  • I attest that the connector works and I verified by deploying and testing all the operations.
  • I attest that I have added detailed descriptions for all operations and parameters in the swagger file.
  • I attest that I have added response schemas to my actions, unless the response schema is dynamic.
  • I validated the swagger file, apiDefinition.swagger.json, by running paconn validate command.
  • If this is a certified connector, I confirm that apiProperties.json has a valid brand color and doesn't use an invalid brand color, #007ee5 or #ffffff. If this is an independent publisher connector, I confirm that I am not submitting a connector icon.

If you are an Independent Publisher, you must also attest to the following to ensure a smooth publishing process:

  • I have named this PR after the pattern of "Connector Name (Independent Publisher)" ex: HubSpot Marketing (Independent Publisher)
  • Within this PR markdown file, I have pasted screenshots that show: 3 unique operations (actions/triggers) working within a Flow. This can be in one flow or part of multiple flows. For each one of those flows, I have pasted in screenshots of the Flow succeeding.
  • Within this PR markdown file, I have pasted in a screenshot from the Test operations section within the Custom Connector UI.
  • If the connector uses OAuth, I have provided detailed steps on how to create an app in the readme.md.

@krautrocker krautrocker requested a review from a team as a code owner November 10, 2024 19:39
@krautrocker
Copy link
Contributor Author

@microsoft-github-policy-service agree company="Dan Romano aka IDR, Inc. dba SWOLCAT"

@troystaylor
Copy link
Contributor

Hi Dan, I have a connector I’m working on too. Are you building more out or is this what you are going to submit (without the Proposal)?

@krautrocker
Copy link
Contributor Author

krautrocker commented Nov 10, 2024

Hi Dan, I have a connector I’m working on too. Are you building more out or is this what you are going to submit (without the Proposal)?

Hi Troy, I will be building out more. These three actions are just the start and will be expanding in the next few hours. To get this up and running, we only need three actions but there's a lot more than we can and will add, naturally. What do you think?

@troystaylor
Copy link
Contributor

I tend to build them all. Are you having any problems with the OAuth?

@krautrocker
Copy link
Contributor Author

Right now, I've got basic auth going with the app passwords and rooting around for OAuth...but not seeing where we can create a client and secret. Posted in the dev channel on Bluesky but no response yet (recent post).

@fordosa90
Copy link
Contributor

@krautrocker Dan, please remove changes to the BkkFutar connector, I am not yet planning to update its authentication ;)

@krautrocker
Copy link
Contributor Author

@krautrocker Dan, please remove changes to the BkkFutar connector, I am not yet planning to update its authentication ;)

Damn, my mistake - will remove at once this evening.

@krautrocker
Copy link
Contributor Author

krautrocker commented Nov 12, 2024

Test operation validation below. Proof of life:

1.) Create session

Bluesky_Custom_Connector_CreateSession

2.) Get preferences

Bluesky_Custom_Connector_GetPreferences

3.) List notifications

Bluesky_Custom_Connector_ListNotifications

/**/

Flow will be ready later tonight or tomorrow once warnings and schemas are otherwise validate. All errors have been cleared via paconn validate.

@krautrocker
Copy link
Contributor Author

krautrocker commented Nov 12, 2024

Working flow:

Actions list (non-exhaustive):

image

1.) Create session

image

/**/

More tomorrow!

@krautrocker
Copy link
Contributor Author

krautrocker commented Dec 18, 2024

Custom connector UI returns a "cannot validate schema" warning for about 19 endpoints and 14 definitions. Digging into this, this can be attributed to nested references ($ref). I can add the schema definition inline so the connector UI can parse and validate. This affects a small number of endpoints.

It's a handful of endpoints referencing definitions so if we put them inline, this may settle this...though I'm not crazy about adding definitions inline. $ref is cleaner and more organized but I would like validation to be as clean as possible.

The connector does indeed save in the UI, so this isn't a showstopper (at least not at the moment).

@krautrocker
Copy link
Contributor Author

krautrocker commented Dec 22, 2024

Work remaining:

  • Update policy parameters in apiProperties to acquire and use an access token (been chasing this one)
  • Replace nested schemas in endpoints where schema cannot be validated.

- Modified the ImportRepo endpoints based on paconn validation

- paconn validated, no errors.
@krautrocker
Copy link
Contributor Author

Setting up a PDS this weekend. Stay tuned @Laskewitz

@krautrocker
Copy link
Contributor Author

PDS setup complete. It is online and accessible via web front end. DNS is resolving, allowing at least a half hour.

Next:

1.) We will connect to the instance via Power Automate

Then:

2.) Create a flow to test session token creation and refresh, confirming joy on the authentication mechanism.

From there:

3.) We will be in a position to test the rest of the endpoints with the connector.

The testing schedule shall commence per namespace:

  • App - Typical social features such as liking a post, fetching a profile, creating a starter Pack, etc.
  • Chat - DMs
  • ATProto - Actions around server and account management
  • Ozone - Moderation service

Human users have been engaged to help create lists, starter packs, send DMs, upload videos and memes, etc.

Success criteria:

  • All endpoints operations are successful.

  • Any problematic endpoints will be quarantined and set aside but going for 100% inclusion.

  • Evidence of success will be provided via image.

Target UAT completion date: January 19th

- Validated with paconn, no errors.

- Changed security definition to "api_key" from "bearerAuth"
@krautrocker
Copy link
Contributor Author

krautrocker commented Jan 20, 2025

Made quite a dent but a few things have tripped up the effort.

1.) TikTok still in play.

We've been monitoring the situation around social media platforms closely over the past few weeks. TikTok has gone dark officially but we expect it to return based on comments from the incoming administration. Palace intrigue indeed, and the situation remains fluid.

2.) Testing is 25% complete.

I would like to post figures to show testing progress, which is currently at 25%. Figures below the following comments:

  • Despite setting up a PDS (Personal Data Server), DNS has still not resolved since it was set up last week. This means handles will display "@Invalid-Handle" until resolution. Good news is the email service is working and email and handle creations and updates are processed successfully via tokens in messages. A bit of a head scratcher after we've placed a TXT record at the registrar level...

A few discussions in forums says this goes away on its own but something else is going on here.

  • It may be tricky to independently test ATProto and Ozone endpoints until the PDS issues are resolved. The next option is to use Postman to simulate PDS behavior but doesn't beat the real thing, naturally.

  • Dynamic hosts are required. This became evident when we started testing the chat endpoints; errors were returned saying "XRPC not supported," which was anticipated. See here: https://docs.bsky.app/docs/advanced-guides/api-directory

apiProperties will be updated to handle dynamic hosts:

Host: {{host}}

Hosts will change dynamically at runtime; a particular flow construction must be used. When calling endpoints that don't use the bsky.social default endpoint, a flow needs to change the host to, say, the chat service, api.bsky.chat.

/**/

  • app.bsky endpoints are close to 90% done.

  • Chat and other services will be completed once dynamic hosts are implemented.

  • ATProto and Ozone will be completed once the PDS DNS issue is resolved or a mock server is used in Postman.

More tomorrow and the rest of this week.

@vmanoharas
Copy link
Contributor

Hello @krautrocker,

Is it ready for review? if not, we would request you to close this PR and submit when the connector is ready

@krautrocker
Copy link
Contributor Author

@vmanoharas I'm actively working on it. Parts of it are ready and I'm updating the PR as I go along. What's the policy here?

@troystaylor
Copy link
Contributor

@vmanoharas Wouldn't it be better for Dan just to restore the [Proposal] tag to the PR title? There's no reason to disconnect his existing commits

@krautrocker
Copy link
Contributor Author

Again, what's the policy?

@krautrocker
Copy link
Contributor Author

@vmanoharas All set for now. Please proceed.

- Validated with paconn, no errors.
vmanoharas
vmanoharas previously approved these changes Jan 29, 2025
Copy link
Contributor

@vmanoharas vmanoharas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dear Partner

I hope you are doing well.
Congratulations, your pull request is approved. We will move forward with the certification process. We are preparing your connector for production deployment and will queue up for the next deployment schedule. Once your connector onboards the next deployment schedule, it starts to deploy your connector in our production environments that typically takes 4-5 weeks.
Please let us know if you have any questions.
Thank you very much for working with us.

@vmanoharas
Copy link
Contributor

[[certify-connector]]

@Laskewitz
Copy link
Member

Laskewitz commented Jan 29, 2025 via email

@troystaylor
Copy link
Contributor

Hi Dan,
Just now getting a chance to test your apiProperties and I'm seeing a couple issues with your policies:

  1. Set Host URL - must use templateId called dynamichosturl. The parameter must be changed to x-ms-apimTemplateParameter.urlTemplate. The urlTemplate value you've provided is an if-else statement which isn't valid.
  2. Acquire Access Token - I can find no reference that this is a valid template, either in the paconn Schema or by online search. The tokenBody uses a key called username whereas the API requires the key to be identifier. The key runPolicyOn is also invalid and must be included in the parameters as x-ms-apimTemplate-policySection set to Request.
  3. The {{accessJwt}} would have to be set as a connection parameter. And runPolicyOn is also invalid as above.

Please correct me if I'm wrong with reference sources. I've also reached out to Lane Swenka to see if these policies are in development. @vmanoharas did the apiProperties fail your validator?

…tandard). Updated x-ms-apimTemplateParameter.host to x-ms-apimTemplateParameter.urlTemplate.

- Removed acquiretoken (not a valid template).

- identifier now used instead of username in authentication.

- runPolicyOn removed; replaced with x-ms-apimTemplate-policySection: Request.

- accessJwt moved to connection parameters.

- accessJwt now stored as a secure connection parameter.

- Updated setheader policy: Authorization: Bearer {{accessJwt}}
@troystaylor
Copy link
Contributor

Hi Dan,
Please continue to test the actions using a connection made with the current connection parameters - I believe you will find that using the accessJwt as a connection parameter would lead to the action only being successful for the 60 seconds after the connection is made. My previous statement that {{accessJwt}} be set as a connection parameter was more about the syntax you were using, not that it was a suggestion for authentication to be implemented that way. Again, I have previously provided to you custom code to handle authentication (that works) and I don't think your method will ever be successful given the current limitations with what Microsoft allows for authentication methods - which I'm happy to be wrong about if proven otherwise.

@krautrocker
Copy link
Contributor Author

I see, I misunderstood you there. Let me take another look. Thanks, @troystaylor

…ntication instead of headers.

- Added C# script to integrate with apiProperties authentication.

- paconn validated, no errors.
@krautrocker
Copy link
Contributor Author

Alrighty, took a sec but let's see how this tests out. Stay tuned and thank you!

_logger.LogError("404 Not Found: The requested resource does not exist.");
return BuildErrorResponse(HttpStatusCode.NotFound, "The requested resource was not found.");

case (HttpStatusCode)529:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@krautrocker, maybe this should be HTTP status code 429, to align with Rate Limits
| Bluesky
/ 429 Too Many Requests - HTTP | MDN?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants