In this episode of the Azure podcast, Sujit and the team, including Cale, Russell, and Cynthia, are joined by special guest Matteo Pagani, a Cloud Solutions Architect in the Tech Strategy team at Microsoft. Matteo provides insights into the agentic world of Co-pilot, explaining how agents can enhance business processes and improve efficiency. Tune in to learn about the practical applications of these technologies and how they can be integrated into existing workflows.

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode515.mp3

YouTube: https://youtu.be/qMJ88BLbTVo

Resources:

• Overview of Microsoft 365 Copilot extensibility: https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility/
• Building declarative agents with Visual Studio Code, Copilot Studio and Agent Builder: https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility/overview-declarative-agent
• Building custom engine agents with Visual Studio Code and Copilot Studio: https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility/overview-custom-engine-agent
• My blog with some fun experiments with multi-agents scenarios: https://www.developerscantina.com/

Other updates:

Announcing GA for Azure Container Apps Serverless GPUs | Microsoft Community Hub

https://www.linkedin.com/pulse/introducing-deep-reasoning-agent-flows-copilot-studio-charles-lamanna-n1zxc/

Let's try GitHub Copilot Agent mode in VS Code to build a FULL app!

Cale and Sujit discuss their current projects in Azure as 2024 comes to a close. They also cover a ton of AKS updates.

Semantic Kernel makes it easier for developers to build Azure Open AI applications that can also include SLMs like Phi-4. 

Azure has many options to use File Shares and Volumes, and we walk through the process of figuring out which one is right for your needs.

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode511.mp3

YouTube: https://youtu.be/dLfCJ6btKng

Resources:

Semantic Kernel - https://github.com/microsoft/semantic-kernel

• Journey with SK on OpenAI and AzureOpenAI
• Ollama (running SLM local) - https://github.com/ollama/ollama
• Ollamazure (running SLM that looks like Azure OpenAI) - https://github.com/sinedied/ollamazure
• PhiSilica - https://learn.microsoft.com/en-us/windows/ai/apis/phi-silica

File Shares: https://learn.microsoft.com/en-us/azure/storage/common/storage-introduction 

Other updates:

Lots of AKS updates!

https://learn.microsoft.com/en-us/azure/aks/concepts-network-isolated

https://learn.microsoft.com/en-us/troubleshoot/azure/azure-kubernetes/availability-performance/container-image-pull-performance

https://learn.microsoft.com/en-us/azure/aks/imds-restriction

https://learn.microsoft.com/en-us/azure/aks/use-windows-gpu

https://azure.microsoft.com/en-us/updates/?id=471295

https://learn.microsoft.com/en-us/azure/backup/tutorial-restore-aks-backups-across-regions

https://learn.microsoft.com/en-us/azure/aks/app-routing-nginx-configuration?tabs=azurecli#control-the-default-nginx-ingress-controller-configuration-preview

https://learn.microsoft.com/en-us/azure/aks/automated-deployments

https://learn.microsoft.com/en-us/azure/aks/aks-extension-ghcopilot-plugins

https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-logs-schema#kubernetes-metadata-and-logs-filtering

Azure Open AI is widely used in industry but there are number of security aspects that must be taken into account when using the technology. Luckily for us, Audrey Long, a Software Engineer at Microsoft, security expert and renowned conference speaker, gives us insights into securing LLMs and provides various tips, tricks and tools to help developers use these models safely in their applications.

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode502.mp3

YouTube: https://youtu.be/64Achcz97PI

Resources:

AI Tooling:

1. Azure AI Tooling Announcing new tools in Azure AI to help you build more secure and trustworthy generative AI applications | Microsoft Azure Blog
   • Prompt Shields to detect and block prompt injection attacks, including a new model for identifying indirect prompt attacks before they impact your model, coming soon and now available in preview in Azure AI Content Safety. 
   • Groundedness detection to detect “hallucinations” in model outputs, coming soon. 
   • Safety system messagesto steer your model’s behavior toward safe, responsible outputs, coming soon.
   • Safety evaluations to assess an application’s vulnerability to jailbreak attacks and to generating content risks, now available in preview.  
   • Risk and safety monitoring to understand what model inputs, outputs, and end users are triggering content filters to inform mitigations, coming soon, and now available in preview in Azure OpenAI Service.
2. AI Defender for Cloud
   • AI Security Posture Management AI security posture management (Preview) - Microsoft Defender for Cloud | Microsoft Learn
   • AI Workloads Enable threat protection for AI workloads (preview) - Microsoft Defender for Cloud | Microsoft Learn      
3.  AI Red Teaming Tool
   • Announcing Microsoft’s open automation framework to red team generative AI Systems | Microsoft Security Blog

AI Development Considerations:

1. AI Assessment from Microsoft
   • Conduct an AI assessment using Microsoft’s Responsible AI Impact Assessment Template
   • Responsible AI Impact Assessment Guide for detailed instructions
2. Microsoft Responsible AI Processes
   • Follow Microsoft’s Responsible AI principles: fairness, reliability, safety, privacy, security, inclusiveness, transparency, and accountability
   • Utilize tools like the Responsible AI Dashboard for continuous monitoring and improvement
3. Define Use Case and Model Architecture
   • Determine the specific use case for your LLM
   • Design the model architecture, focusing on the Transformer architecture
   •  
4. Content Filtering System
   • How to use content filters (preview) with Azure OpenAI Service - Azure OpenAI | Microsoft Learn
   • Azure OpenAI Service includes a content filtering system that works alongside core models, including DALL-E image generation models. This system uses an ensemble of classification models to detect and prevent harmful content in both input prompts and output completions
   • The filtering system covers four main categories: hate, sexual, violence, and self-harm
   • Each category is assessed at four severity levels: safe, low, medium, and high
   • Additional classifiers are available for detecting jailbreak risks and known content for text and code. JailBreaking Content Filters
5. Red Teaming the LLM
   • Plan and conduct red teaming exercises to identify potential vulnerabilities
   • Use diverse red teamers to simulate adversarial attacks and test the model’s robustness
   • Microsoft AI Red Team building future of safer AI | Microsoft Security Blog
6. Create a Threat Model with OWASP Top 10
• owasp.org/www-project-top-10-for-large-language-model-applications/assets/PDF/OWASP-Top-10-for-LLMs-2023-slides-v1_1.pdf
• Develop a threat model and implement mitigations based on identified risks

Other updates:

• Los Angeles Azure Extended Zones
• Carbon Optimization
• App Config Ref GA
• OS SKU In-Place Migration for AKS
• Operator CRD Support with Azure Monitor Managed Service
• Azure API Center Visual Studio Code Extension Pre-release
• Azure API Management WordPress Plugin
• Announcing a New OpenAI Feature for Developers on Azure

Santosh Somayajula, a Principal Product Manager in Azure, breaks down the new Copilot for Azure service which is currently in Private Preview in the Azure Portal. Using this feature, administrators and management can perform operations in the Portal using simple verbal commands instead of navigating the UI.

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode485.mp3

YouTube: https://youtu.be/Gqv-qOOoK28

Resources: https://learn.microsoft.com/en-us/azure/copilot/overview

 https://aka.ms/MSCopilotforAzurePreviewRequest

Other updates:

General Availability: Azure Virtual Network encryption | Azure updates | Microsoft Azure

Public preview: ExpressRoute guided portal experience for multi-site resiliency | Azure updates | Microsoft Azure

General Availability: Azure MySQL Import feature for Azure Database for MySQL Single to Flexible Server migrations | Azure updates | Microsoft Azure 

Public Preview: Circuit Breaker in Azure API Management | Azure updates | Microsoft Azure  

https://azure.microsoft.com/en-us/updates/general-availability-azure-topology/

https://azure.microsoft.com/en-us/updates/mount-azure-storage-as-a-local-share-in-app-service-linux-now-supports-nfs/

https://azure.microsoft.com/en-us/updates/generally-available-continuous-model-monitoring-in-azure-machine-learning/

https://azure.microsoft.com/en-us/updates/public-preview-azure-api-center-features/

https://azure.microsoft.com/en-us/updates/public-preview-new-major-version-of-durable-functions-extension/

https://azure.microsoft.com/en-us/updates/generally-available-outbound-type-migration-in-aks/

We talk to Steve Selleny, a Global Partner Development Manager at Microsoft, about the AI and ML trends he has noticed in Financial Services ISVs. 

Steve has been working in the technology sector for over 21 years in various roles including software development, systems architecture, consulting, sales, and partner management. For the past two years, he has been working with Financial Services Independent Software Vendors (ISVs) to build and market solutions on the Microsoft Cloud that drive digital transformation for customers.

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode471.mp3

YouTube: https://youtu.be/6WDlqR34M48

Resources:

https://news.microsoft.com/2023/06/29/moodys-and-microsoft-develop-enhanced-risk-data-analytics-research-and-collaboration-solutions-powered-by-generative-ai/ 

Other updates:

Strengthening Your Defenses: Simulation Testing for Azure DDoS Protection - Microsoft Community Hub

https://azure.microsoft.com/updates/public-preview-ratelimit-rules-for-application-gateway-web

https://azure.microsoft.com/updates/general-availability-terraform-support-for-azure-cosmos-db

https://azure.microsoft.com/updates/generally-available-durable-functions-for-python-v2

https://azure.microsoft.com/updates/general-availability-99th-percentile-latency-metric-in-azure

Upgrade to App Service Environment v3 today and unlock new capabilities | Azure Blog | Microsoft Azure

https://azure.microsoft.com/en-us/updates/cloud-services-retirement-announcement/

Cloud Solution Architect, Dr. Linda Sheard, joins Russell and Sujit for a primer on Azure Open AI. Linda breaks down the service and provides context on how it can be used for the typical scenario where a customer wants to reason over their own data. 

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode466.mp3

YouTube: https://youtu.be/nDcIWvHa43s

Resources: https://learn.microsoft.com/en-us/azure/cognitive-services/openai/overview

Other updates:

https://learn.microsoft.com/en-us/azure/container-registry/tutorial-registry-cache

The team speaks with Nikhil Vimal from Chat Mode about different use cases where combinations of Azure services can be used to infuse AI or leverage machine learning into automating processes.

Media File: https://azpodcast.blob.core.windows.net/episodes/Episode391.mp3

YouTube: https://www.youtube.com/watch?v=OYOmYhLbdyw

Resources:

https://www.microsoft.com/en-us/ai

Power Automate Desktop

Power Automate RPA

Power Automate UI Flow

Power Virtual Agents (AI Chatbots)

Microsoft Teams Bots

Azure Bot Services

Microsoft AI Research

Updates:

Genomics testing on the ISS with HPE Spaceborne Computer-2 and Azure

Developer Evangelist David Gristwood and Cloud Solution Architect Robin Lester, from Microsoft UK, are enthusiastic about the AI hacks they do with partners to help them quickly realize an application that uses Azure's AI capabilities. They discuss common patterns and tips and tricks from their various hacks.

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode259.mp3

Transcript: https://www.videoindexer.ai/accounts/aca83d23-620b-46d6-beec-e920bff88847/videos/d4f658c21b/

Merry Christmas and Happy Holidays!

Cale and Sujit talk about the new style of applications that leverage AI, Blockchain and Big Data services in Azure.

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode235.mp3

Other updates:

https://azure.microsoft.com/en-us/updates/log-analytics-solutions-network-security-goup-analytics-and-azure-web-apps-analytics-now-available-in-azure-quickstart-template-gallery/

https://azure.microsoft.com/en-us/blog/azure-storage-static-web-hosting-public-preview/

Microsoft from GeekWire Cloud Tech Summit: New Azure innovations will advance the intelligent cloud and intelligent edge
https://azure.microsoft.com/en-us/blog/microsoft-from-geekwire-cloud-tech-summit-new-azure-innovations-will-advance-the-intelligent-cloud-and-intelligent-edge/

The emerging big data architectural pattern
https://azure.microsoft.com/en-us/blog/the-emerging-big-data-architectural-pattern/

Reasons why retail and consumer goods brands should address Artificial Intelligence today
https://azure.microsoft.com/en-us/blog/reasons-why-retail-and-consumer-goods-brands-should-address-artificial-intelligence-today/

Azure Data Factory new capabilities are now generally available
https://azure.microsoft.com/en-us/blog/azure-data-factory-new-capabilities-are-now-generally-available/

Senior Product Marketing Manager Sonya Koptyev talks to us about the Azure AI Platform and give us some great pointers on how to get started with AI.

Media file: https://azpodcast.blob.core.windows.net/episodes/Episode224.mp3

Other updates:

Following last month's announcement of zone-redundant database support, Azure SQL Database is further expanding the power of databases and elastic pools by including read scale-out support in the Premium and Business Critical service tiers. 
This capability redirects the read-only client connections to one of the automatically provisioned HA replicas and effectively doubles the compute capacity of the database or elastic pool at no additional charge. This is ideal for load balancing of complex analytical workloads without affecting the primary OLTP workload.

https://azure.microsoft.com/en-us/updates/public-preview-of-read-scale-out-support/

Azure DevTest Labs is now available in Azure Government.
With this availability, US government agencies and their partners can modernize their legacy infrastructure to build fast, easy, and lean environments.

https://azure.microsoft.com/en-us/blog/azure-source-volume-26/

Offline media import for Azure
https://azure.microsoft.com/en-us/blog/offline-media-import-for-azure/

Achieving GDPR compliance in the cloud with Microsoft Azure
https://azure.microsoft.com/en-us/blog/achieving-gdpr-compliance-in-the-cloud-with-microsoft-azure/

Three common analytics use cases with Microsoft Azure Databricks
https://azure.microsoft.com/en-us/blog/three-critical-analytics-use-cases-with-microsoft-azure-databricks/