CRM Telemetry

Why Telemetry?

Leadership

• How is my business doing?
• Are my customers happy with the service?
• What features should I invest to improve my RoI?

PM

• Am I building the right feature?
• Am I building it the right way?
• Who is using my feature and how are they using
• How is my feature functioning?

Engineering

• What are my top 3 errors and how to fix them?
• Do I need to add more nodes to my application?
• Does my P95 load time is meeting SLA?

What is Telemetry?

• tele = remote
• metron = measure
• telemetry = pipeline + data store + insights

Telemetry Components

Telemetry pipelines

• CRMOL CMA Pipeline
• Cloud Services Pipeline
• Insights Endpoint
• GCC(US Government Cloud)
• BlackForest(Deusche Cloud)

CRMOL CMA Pipeline

• Server side data collection
• Built on Geneva Infra

Geneva Architecture

• Server side data collection

CMA Architecture

What data is captured?

• Windows Performance Counters
• ETW events
• IFx events
• Windows System events
• IIS Logs - Geo, User, Browser info
• SQL Data Collector Component - Querying Geo, Sitewide or Org DBs
• Extended Events - SQL Server Advanced Metrics
• Extensions
• Derived Events

Insights REST Endpoint

• Client Side data collection
• MoCA/ISH/UCI
• 30 million hits/day
• Scalable app on Azure

Cloud Services

Private Cloud Pipeline

Telemetry Components

Cosmos

• MS Internal big data store with compute
• Cold path - for complex data processing
• ML and DS usecases
• Needs another data store for connecting to applications
• Scope language
• CAJobScheduler

Kusto

• Azure based Log Analytics, Log Search system
• Extremely fast and expensive(!!)
• High ingestion rate and low latency
• Suitable for real time debugging
• Integrated charting
• Can be connected to other applications
• Avodaco Jobs

Kusto

• 12 TB/day ingestion
• 1 day hot path/30 day warm path

Azure DataMart

• Azure SQL for Storing aggregated data from Kusto/Cosmos
• Can be connected to applications
• Scaled for large requests

Telemetry Components

CRM Insights

• Azure based web application
• Angular JS/High charts
• Suitable for complex charting
• Used by leadership and non-engineering teams for status updates, all hands

Power BI

• Desktop or Web
• Quick charting
• Internal team consumption

Org Insights

• CRM first party solution on top of Telemetry data
• Best option to present data to customers

Telemetry Components

MDM Metrics

• Hot path alerts
• Jarvis - Manage Hot Path

AIMS Alerts

• Warm path alerts
• Complex queries
• IcM, Email integration
• Keep as a silent alert until fine tuned

Kusto Alerts

• Built on top of Kusto
• Deploy in Test before Prod

Planning Telemetry

Stage 1

Telemetry Requirements

• What I am going to measure?
• How frequently I need to report?
• Short term or long term requirement
• Who is going to consume the data
• Is it going to be a static data or the user will slice and dice the data

Telemetry Requirements

• Do I need alerting and monitoring?
• Do I need ML or Predictive Analytics

Stage 2

CRM Code Instrumentation

• Code Instrumentation needs to align with CRM delivery cycle
• Plan requirements for future
• Have columns for future

Stage 3

Telemetry Repo Changes

• MA/MDS changes
• OneBox testing
• TiE testing
• PROD deployment
• Validation in Jarvis, Kusto, Cosmos

Stage 4

Dashboards and Chart

• Kusto Charts
• Cosmos Scope scripts for complex data processing - Azure Data Mart
• Power BI or CRM Insights
• Excel
• Kusto Explorer

Stage 5

Untitled Slide

Alerting and Monitoring

• AIMS alert/MDM
• Kusto Alert - Validate in Test before Prod

General Telemetry Usecases

Untitled Slide

• Feature Usage
• Performance
• QoS
• Diagnostics

General Telemetry Usecases

• Feature Usage
• Performance
• QoS
• Diagnostics

Data Privacy

• Validate data for compliance and PII with Compliance team
• PII Data Reporting is a Sev1 issue

Contribute

• Ideas
• Code
• Features
• Support

Contact Us - StackOverFlow

Untitled Slide

What

Contribute

Telemetry Team