Single endpoint for all prometheus metrics

[kennsippell]

The fastify-metrics library doesn't play particularly nicely for extensibility. I attempted a few vectors to merge these two endpoints:

1. Interupt the response from fastify-metrics and append additional metrics from BullMQ
2. Obtain the promClient object and add the bullmq metrics as custom counters within fastify-metrics
3. Registry the fastify-metrics as its own endpoint and have /metrics invoke it

In the end, I liked the code for 1 the best. Let me know if you'd like to see some of the other approaches!

#273

[mrjones-plip]

While the code looks good as far as I can tell, I cannot get the /metrics endpiont to work - I get a 302 to login. I actually went back to a branch before you made your bullq change and the /metrics endpoint didn't work then either. I had to go all the way back to 4dafb28 - and then it finally worked! Something changed between now and then - I can't see what off hand 🤔

However, researching what we did back then, I found the watchdog-config directory! This has all the files we just recreated in watchdog. We should pick one winner (watchdog or UMT) and remove the watchdog config files from the loser.

I was involved with this earlier PR - sorry I didn't remember about these files already existing!

[kennsippell]

Oh that's weird it isn't working... The first guess I have is that have an outdated build artifact in your dist folder. Can we follow these steps to test?

1. Delete dist folder
2. Run ./docker-local-setup.sh build
3. Navigate to http://localhost:3500/metrics

I'm seeing a prometheus endpoint with both fastify and bullmq metrics when I run this test. Hope that works.

We should pick one winner (watchdog or UMT) and remove the watchdog config files from the loser.

Oh wow! I didn't see that until now. That's too bad they also aren't on watchdog. I'd personally prefer to move them to the watchdog repro probably so we can track changes and deploy them for https://watchdog.app.medicmobile.org. Does that make sense to you?

[mrjones-plip]

Ah - sweet - thanks Kenn! Nuking the dist folder did the trick. Faster for me to run npm run dev, but I'm sure docker-local-setup.sh will work too. /metrics works now - yay!

and yeah - go ahead and nuke the watchdog files here in favor of watchdog repo. Maybe replace the readme with a big "see watchdog repo" and a link?

[mrjones-plip]

I finished testing in the other watchdog ticket and this new set up works great - nicely done! OK to merge.