Skip to content

Latest commit

 

History

History
28 lines (14 loc) · 4.03 KB

TRANSPARENCY_FAQ.md

File metadata and controls

28 lines (14 loc) · 4.03 KB

Conversation Knowledge Mining Solution Accelerator: Responsible AI FAQ

  • What is Conversation Knowledge Mining Solution Accelerator?

    Conversation Knowledge Mining is an open-source GitHub Repository to help ingest, extract, and classify content from a high volume of assets to gain deeper insights and generate relevant suggestions for quick and easy reasoning. It enables call centers to process call transcripts, audio files and call transcripts meta data to improve decision making at scale. The accelerator is built using RAG pattern to process unstructured data. RAG enhances the ability by combining retrieval from conversation data with generative AI models. Topic modeling identifies themes within large data sets. The accelerator is built on top of Azure OpenAI, Microsoft Fabric, Azure AI services, Azure Functions and Cosmos DB.

  • What can Conversation Knowledge Mining Solution Accelerator do?

    Conversation Knowledge Mining can perform a variety of functions related to analyzing conversation data from contact centers. The sample solution focuses on a contact center analyst looking to extract actionable insights from complex data sets. The sample data is synthetically generated utilizing Phi-3-medium-4k-instruct and GPT-4. Names are pulled from the CELA approved names list database.

  • What is/are Conversation Knowledge Mining Solution Accelerator’s intended use(s)?

    This repository is to be used only as a solution accelerator following the open-source license terms listed in the GitHub repository. The example scenario’s intended purpose is to demonstrate how users can analyze and process audio files and call transcripts to help them work more efficiently and streamline their human made decisions.

  • How was Knowledge Mining Solution Accelerator evaluated? What metrics are used to measure performance?

    As an AI-powered solution accelerator, Conversation Knowledge Mining was evaluated through human review of the LLM output that summarizes conversations, utilizing synthetically generated fictional data, and the identification of keywords from the conversations.

    It's worth noting that the system is designed to be customizable and can be tailored to fit specific business needs and use cases. As such, the metrics used to evaluate the system's performance may vary depending on the specific use case and business requirements.

  • What are the limitations of Conversation Knowledge Mining Solution Accelerator? How can users minimize the impact of Conversation Knowledge Mining Solution Accelerator’s limitations when using the system?

    This solution accelerator can only be used as a sample to accelerate the creation of knowledge mining solutions. Users of the accelerator should review the system prompts provided and update as per their organizational guidance. AI generated content in the solution may be inaccurate and should be manually reviewed by the user. Right now, the sample repository is available in English only.

  • What operational factors and settings allow for effective and responsible use of Conversation Knowledge Mining Solution Accelerator?

    Effective and responsible use of Conversation Knowledge Mining depends on several operational factors and settings. The system is expected to perform reliably and safely within a range of conversation types and languages that it was designed and evaluated for. Users can customize certain settings, such as the language model (eg. GPT3.5 vs GPT4) used by the system, meta prompt, and the types of conversations that are analyzed. However, it's important to note that these choices may impact the system's behavior in the real world. For example, choosing a language model that is not well-suited to the conversation data being analyzed may result in lower accuracy and performance. Similarly, analyzing conversations that are outside of the system's intended scope may result in errors or inaccurate results. To ensure effective and responsible use of Conversation Knowledge Mining, users should carefully consider their choices and use the system within its intended scope.