Tip for effective technique to determine if two HTTP responses are the same #11
Description
Hello,
A long time ago I wrote a similar tool https://github.com/vincentcox/bypass-firewalls-by-DNS-history. I see you have the same problem I had back in the day, you mentioned it in the ToDo: Finding an effective technique to determine if two HTTP responses are the same.
Times have changed and AI provides new opportunities.
I would suggest solving that issue with Gemini AI. Google provides free API keys and the big context windows are great to just dump big chunks of HTML inside of it (maybe only take the first X number of characters to avoid issues with big pages). Then you can ask if it's very likely the same or not. I'm sure there is a Go Library for easy access to Gemini.