DOCSEARCHbyAlgolia
Submit Docs

AI-102: Designing and Implementing a Microsoft Azure AI Solution

repository·master·Indexed 21 days ago

https://github.com/microsoftlearning/ai-102-aiengineer

Training materials and code samples for the AI-102 certification, focusing on Cognitive Security implementations using C#. This deprecated repository includes guides for provisioning Azure AI Services, using REST interfaces and SDKs (such as Azure.AI.TextAnalytics v5.3.0), and implementing Azure AI Search with Python. It provides setup instructions for .NET 7.0 SDK, Miniconda, and Azure CLI, as well as guidance on managing authentication keys and registering Azure resource providers.

Tokens
40.1K
Snippets
98
Records
173
Agent score
75%

What's inside ai-102-aiengineer

  1. Locate supported lab files for AI-102 and other AI courses

    master

    This repository is deprecated and no longer supported. Lab files for AI-102 and other Microsoft AI-related courses have been migrated to specialized repositories. Depending on the specific AI service you are studying, use the corresponding repository below:

  3. Authenticate with the Video Analyzer REST API

    master

    All interactions with the Video Analyzer REST API follow a two-step authentication pattern:

    1. Obtain Access Token: Call the AccessToken method using your API Key in the request header.
    2. Call API Methods: Use the obtained access token to authenticate subsequent requests to work with videos.

    Required Credentials:

    • Account ID: Found in the Account settings page of the Video Analyzer portal.
    • API Key: Found in the Video Analyzer developer portal under Subscriptions (either primary or secondary key).

  4. How bot activities and turns work

    master

    A conversation with a bot is composed of several key concepts:

    • Activities: The units of exchange in a conversation. These can include text, graphics, or UI cards. Common activities include Member Added (a conversation update) and Message (user input).
    • Turns: An interaction where the bot receives, processes, and responds to an activity.
    • Turn Context: An object used to track information about the specific activity being processed during the current turn.
    • Activity Handlers: Functions within your bot code (like OnMessageActivityAsync in C# or on_message_activity in Python) that respond to specific types of activities.

  5. Configure Index and Indexer field mappings

    master

    To incorporate enriched data from a skillset into your search index, you must update both the index definition and the indexer definition.

    1. Update the Index (index.json)

    Add new fields to hold the skill outputs. For example, to store sentiment:

    {
    "name": "sentiment",
    "type": "Edm.String",
    "facetable": false,
    "filterable": true,
    "retrievable": true,
    "sortable": true
}

    2. Update the Indexer (indexer.json)

    Use fieldMappings for source metadata and outputFieldMappings for skillset outputs.

    • fieldMappings: Maps source data (e.g., metadata_storage_path) to index fields (e.g., url).
    • outputFieldMappings: Maps skillset output paths (e.g., /document/sentimentLabel) to index fields (e.g., sentiment).
    // Example outputFieldMapping in indexer.json
{
    "sourceFieldName": "/document/sentimentLabel",
    "targetFieldName": "sentiment"
}

// Example fieldMapping in indexer.json
{
    "sourceFieldName" : "metadata_storage_path",
    "targetFieldName" : "url"
}

  6. How Azure AI Services containers work

    master

    Azure AI services can be deployed as containers in your own infrastructure (e.g., local Docker servers, Azure Container Instances, or Azure Kubernetes Services).

    Key characteristics:

    • Data Privacy: Application data is processed locally within the container and is not passed to the Azure back-end service.
    • Billing: Containers must communicate with an Azure-based Azure AI services account to support billing. This communication is used for usage reporting, not for processing request data.
    • Control: Provides greater control over deployment configuration, authentication, and scalability.

  7. Process intents and entities from Language Service predictions

    master

    Once you receive a prediction, your application logic should branch based on the topIntent and extract specific data from the entities list to perform actions.

    Common Pattern

    1. Check Intent: Evaluate the topIntent (e.g., GetTime, GetDay, GetDate).
    2. Extract Entities: Iterate through the entities collection. Match the category (e.g., Location, Date, Weekday) to find the relevant text value.
    3. Execute Action: Use the extracted entity value to call your application's specific logic (e.g., GetTime(location)).

    Entity Categories in Example

    • Location: Used with GetTime intent.
    • Date: Used with GetDay intent.
    • Weekday: Used with GetDate intent.
    // C# Logic Pattern
switch (topIntent)
{
    case "GetTime":
        var location = "local";           
        foreach (dynamic entity in conversationPrediction.Entities)
        {
            if (entity.Category == "Location")
            {
                location = entity.Text;
            }
        }
        string timeResponse = GetTime(location);
        Console.WriteLine(timeResponse);
        break;
    // ... other cases
}
    # Python Logic Pattern
if top_intent == 'GetTime':
    location = 'local'
    if len(entities) > 0:
        for entity in entities:
            if 'Location' == entity["category"]:
                location = entity["text"]
    print(GetTime(location))

elif top_intent == 'GetDay':
    date_string = date.today().strftime("%m/%d/%Y")
    if len(entities) > 0:
        for entity in entities:
            if 'Date' == entity["category"]:
                date_string = entity["text"]
    print(GetDay(date_string))

  8. Search an index using the SDK

    master

    The Azure AI Search SDK allows you to create a SearchClient using your endpoint and query key. You can then submit search queries with various parameters to refine results:

    • Search Mode: Set to require all individual words in the search text (e.g., all mode).
    • Count: Include the total number of documents found.
    • Filter: Apply a filter expression to include only specific documents.
    • Sort: Specify a sort order for the results.
    • Facets: Request discrete values for a field (e.g., metadata_author) to enable UI filtering.
    • Highlights: Request extracts from specific fields (e.g., merged_content or imageCaption) with search terms highlighted.
    • Select: Specify only the fields you want returned in the results.

  9. Automate training with the Custom Vision Training API

    master

    You can use the Custom Vision Training SDK to programmatically manage your project. The workflow typically involves:

    1. Authentication: Create an authenticated client (e.g., CustomVisionTrainingClient in C#) using the training resource's endpoint and key.
    2. Project Reference: Use the Project Id to create a reference to your specific project.
    3. Image Upload: Use functions to upload images and assign them the correct tag ID based on your classification categories.
    4. Model Training: Trigger a new training iteration (e.g., using a Train_Model function) and wait for completion.

    Configuration Requirements:

    • C#: Update appsettings.json with the training endpoint, training key, and Project ID.
    • Python: Update .env with the training endpoint, training key, and Project ID.

  10. Core concepts of Conversational Language Understanding (CLU)

    master

    Conversational Language Understanding (CLU) in the Azure AI Language service is used to interpret natural language input. It focuses on two primary tasks:

    1. Intent Prediction: Determining what the user wants to achieve (e.g., GetTime).
    2. Entity Identification: Identifying specific pieces of information related to the intent (e.g., a location like London).

    Key Terminology:

    • Utterance: The raw text input provided by a user (e.g., "What is the time in London?").
    • Intent: The goal or action the user intends to perform.
    • Entity: The specific data points within an utterance that provide context to the intent.

    Important Distinction: A CLU model only predicts intent and identifies entities; it does not perform the actual logic (like fetching the time). The client application must implement the logic to act on the model's predictions.