Skip to content

A Python-based web scraping tool that detects technologies used on a website by analyzing its scripts, meta tags, and HTML content.

Notifications You must be signed in to change notification settings

VIPULbunny/Web-Tech-Scanner

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

8 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ” Technology Detector

A powerful Python-based tool that scrapes a given website and detects the technologies used on it! This tool utilizes web scraping and pattern matching to identify various technologies, frameworks, and libraries implemented in the site's HTML, scripts, and metadata.

πŸš€ Features

  • Scrapes a website and analyzes its HTML, meta tags, and scripts
  • Matches technologies against a predefined dataset
  • Provides a clean and accurate list of detected technologies
  • Fast and efficient, using BeautifulSoup for parsing and requests for fetching data

πŸ“Œ Tags

Python Web Scraping Technology Detector BeautifulSoup Requests Automation


πŸ“₯ Installation

Prerequisites

Ensure you have Python 3.x installed along with the required libraries.

pip install pandas requests beautifulsoup4

Clone the Repository

git clone https://github.com/yourusername/technology-detector.git
cd technology-detector

⚑ Usage

Run the script and enter a website URL to analyze.

python technolog.py

Example Output:

Enter the website URL: https://example.com
Formatted URL: example.com
Technologies used in this website: jQuery, Bootstrap, Google Analytics

πŸ›  How It Works

  1. Loads Technology Data πŸ“‚

    • Fetches a dataset of web technologies from a JSON file.
    • Converts the dataset into a structured pandas DataFrame.
  2. Scrapes the Website 🌐

    • Uses requests to fetch the page source.
    • Parses the HTML using BeautifulSoup.
  3. Matches Technologies πŸ”

    • Extracts scripts, meta tags, and headers from the website.
    • Checks for predefined technology patterns.
    • Returns a list of matched technologies.


🌟 Future Enhancements

βœ… Add support for more technology datasets πŸ”§ βœ… Improve accuracy with machine learning-based detection πŸ€– βœ… Build a GUI or Web Interface for ease of use πŸ–₯️


🀝 Contributing

Pull requests are welcome! For major changes, please open an issue first to discuss what you'd like to modify.


πŸ“œ License

This project is open-source and available under the MIT License.


πŸ“§ Contact

Have questions or suggestions? Feel free to reach out!

πŸ“© Email: [email protected]
πŸ”— GitHub: VIPULbunny

About

A Python-based web scraping tool that detects technologies used on a website by analyzing its scripts, meta tags, and HTML content.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published