Copilot Interaction Logging

Overview

What you build

No package is published, so you build every component yourself, inside one Dataverse solution in a development environment, then promote it as a managed solution (see Build from scratch).

Security

Read this first: high-privilege, tenant-wide access

The app registration needs only AuditLogsQuery.Read.All. Microsoft Graph offers workload-scoped variants such as AuditLogsQuery-Exchange.Read.All and AuditLogsQuery-SharePoint.Read.All, but none is documented as covering Copilot interaction records. The reference build hasn't been tested with them. AuditLog.Read.All is not enough for the audit log query API. See the API permissions and the permissions reference.

Before you start

Do you need this? Built-in options first

Microsoft already provides several supported ways to see Copilot activity. Check whether one of them meets the need before you build and maintain a custom collector.

Prompts and responses aren't held in the audit log. They're stored in users' mailboxes and are subject to your retention and eDiscovery controls (Copilot audit logs).

Design

How it works

Two flows share the same logic: a scheduled flow that collects one day at a time, and a manual flow for backfills and re-runs.

1. Set the window

   The scheduled flow runs once a day (time zone GMT Standard Time) and collects one UTC day, from startOfDay(addDays(utcNow(), -4)) to startOfDay(addDays(utcNow(), -3)), so a run on 10 March collects 6 March. The lag allows for audit records that arrive late. The manual flow collects from StartDate to EndDate, and EndDate is exclusive. See Phase A.

2. Resolve the secret and open the run log

   If poc_Audit_UsingAKVtruefalse is true, the flow reads the client secret from Azure Key Vault with the Dataverse unbound action RetrieveEnvironmentVariableSecretValue. Otherwise it uses the plain-text variable poc_Audit_Secret. The Key Vault action has secure inputs and outputs, and the Compose actions that pass the secret on have secure inputs, so the secret is hidden in run history. The flow then adds a run log row with Flow State set to Running. See Phase B and Phase C.

3. Create the query

   An HTTP action sends POST /beta/security/auditLog/queries with the window, and with recordTypeFilters and operationFilters both set to ["CopilotInteraction"] (create auditLogQuery, auditLogQuery). A retry loop makes up to 5 attempts within 5 minutes, 20 seconds apart. See D1.

4. Wait for the query

   Purview runs the search in the background. The flow checks the query's status every 60 seconds until it's succeeded. See D2.

5. Read the records

   The flow reads the results up to 500 at a time (list auditLogRecords) and follows @odata.nextLink to the next page. Each page read gets up to 5 attempts within 10 minutes, 30 seconds apart. All three HTTP actions have secure inputs, which hide the secret, but their outputs aren't secured in the reference build, so run history shows the records (Securing run history). See D3.

6. Write each record

   For every audit log record, Get_a_row_by_ID checks whether it's already stored (only so that new and existing records can be counted), then Upsert_a_row_2 writes it with the row ID set to the audit record id, so re-running a window never creates duplicates. If an upsert fails, a run-after branch adds a "Failed - Upsert a row 2" error row and adds 1 to Records Error; that record isn't saved. The flow moves to the next page only after every record on the page succeeds, so a failed page is read again, and a failure that persists ends in CollectionIncomplete. See D4.

7. Finish, or fail loudly

   If the query never started or pages were left unread, the flow writes an error row and ends the run as Failed with CollectionIncomplete. An unhandled failure inside Get_Logs ends the run as Failed with GetLogsFailed. Otherwise the run ends Succeeded. See Phases E and F.

Loop limits

• A loop that reaches its limit ends Succeeded, and the flow carries on (tested with a mock flow; see Until loop considerations). The completeness check catches a paging loop that stopped early, but not a wait loop that gave up before the query finished.
• Time usually runs out before pages do. Records are written one at a time, at about 25 to 60 a minute, so the 10-hour limit is reached at about 15,000 to 36,000 records, well before 100 pages.
• A run can be long. At the scheduled limits a run can take about 18 hours. A cloud flow run can last up to 30 days.
• The endpoint is beta. Graph beta APIs can change without notice.

Details, and how to size the limits for your volumes: Loop limits and Planning for high volumes.

Reference build

Inside the flow

Screenshots of the scheduled flow in the reference build, from top to bottom. Select an image to enlarge it. Each caption links to the part of the action reference that explains every action in it and why it exists. Names and values are from a demo environment.

Reading the top level

1. A Trigger and variables (collapsed) set the collection window, counters and flags. 23 actions.
2. B EnvVarTextCheck, Check_AKV_Secret and Resolve_Graph_Secret read the app secret from Key Vault or from a plain-text environment variable. 8 actions.
3. C Capture_Flow_Run_Details writes a Running row to Copilot Interaction Flow Runs. 2 actions.
4. D1–D4 Get_Logs creates the Graph query, waits for it, reads each page and upserts every record. 46 actions.
5. E Error_Handling ends the run as Failed if Get_Logs failed. 3 actions.
6. F Check_Collection_Complete ends the run as Failed if data is missing or partial. 3 actions.

85 actions in total. The phase map lists them, and the control-flow and sequence diagrams show how they connect.

Every part, in order

Close

Under the hood

The HTTP calls

Three HTTP actions do all the talking to Microsoft Graph. AuditLogQuery creates an audit log query, AuditLogQueryStatus waits for it to finish, and AuditLogRecords reads its records a page at a time. This section shows what each one sends and what the Parse JSON action after it expects back. The flows use no Graph connector: the HTTP actions sign in on their own, as in the CoE Starter Kit pattern. For your own build, consider a custom connector.

The three calls

Shown with … for https://graph.microsoft.com/beta. Placeholders in braces are filled in by the flow at run time.

{
  "filterStartDateTime": "@{variables('startTime')}",
  "filterEndDateTime": "@{variables('endTime')}",
  "recordTypeFilters": ["CopilotInteraction"],
  "operationFilters": ["CopilotInteraction"]
}

{
  "type": "object",
  "properties": {
    "@odata.nextLink": { "type": "string" },
    "value": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": { "type": "string" },
          "userPrincipalName": {},
          "auditData": {}
        },
        "required": ["id"]
      }
    }
  }
}

Every action, with its settings, purpose and the change a clean build should make: HTTP authentication · D1 · D2 · D3 · D4 · Securing run history · Limits and throttling.

Before you build

Prerequisites and permissions

Decide first

Secret-handling decision: how the client secret is stored

Scripts are off: all three options are shown below.

Client secrets last 24 months at most, and Microsoft recommends less than 12. The value is shown only once, when you create it. When a secret expires, the Graph calls fail and the run log shows the query never started; see Troubleshooting and Rotating the secret.

Configuration

Environment variables

The flows read their settings from Dataverse environment variables, so the same flows work in every environment. poc is the reference build's publisher prefix; yours will be your own publisher's prefix. In code view, a flow refers to a variable by display name and schema name, for example parameters('Audit_Tenant (poc_Audit_Tenant)'). Create these:

The last column and the highlighted rows follow the option chosen in the secret-handling decision.

Don't create the three legacy variables in the reference build: poc_Audit_ReviewerEmail (unused), poc_AuditMinutestoLookBack (Decimal number, 65) and poc_AuditEndTimeMinutesAgo (Decimal number, 2820). Only legacy Compose actions read the last two.

From an empty environment

Build from scratch: step by step

No solution package is published, so you create every component yourself, in this order, in a development environment. Building it yourself means you choose the secret option, publisher prefix, retention and security that fit your organisation, rather than inheriting someone else's. Get the approvals described in Read this first before step 1. Each step links the detail you need.

1. Register the app

   1. In the Entra admin center, go to App registrations > New registration. Choose a single-tenant registration.
   2. Go to API permissions > Add a permission > Microsoft Graph > Application permissions, and add AuditLogsQuery.Read.All. AuditLog.Read.All isn't enough.
   3. Select Grant admin consent. This needs a Privileged Role Administrator or Global Administrator.
   4. Go to Certificates & secrets > New client secret. Choose an expiry of less than 12 months (24 months at most), and copy the Value straight away: it's shown only once. Record the expiry date and an owner for rotation.

   A certificate is preferred in production. See Graph permissions.

2. Store the secret

   Store it using the option you chose in the secret-handling decision.

   Option A: add the secret value to Key Vault, grant Key Vault Secrets User to yourself and the Dataverse service principal, and keep the vault details ready for when you create poc_KeyVaultSecret in step 5 (see option A).

   Option B: keep the secret value ready to enter as the current value of poc_Audit_Secret in step 5. Not recommended (see option B).

   Option C: upload the certificate's public key to the app registration, and store the PFX and its password, ideally in Key Vault. You set the HTTP actions' credential type to Certificate in step 7 (see option C).

3. Create a publisher and a solution

   1. In the development environment, go to Solutions > New solution and create a new publisher with your own prefix. This guide uses poc, the reference build's prefix; read it as yours throughout.
   2. Create the solution with that publisher. Create everything in steps 4 to 7 inside the solution, so the tables, variables, connection reference and flows move together when you promote it.

   See solution concepts and create a solution.

4. Create the three tables

   1. Copilot Interactions: one row per audit record. Rename the primary name column to Audit Log ID, then add the 16 columns in the data model.
   2. Copilot Interaction Flow Runs: the run log, with the two choice columns Flow State (Running, Finished) and Run Type (Scheduled, Manual) and the four whole-number count columns.
   3. Copilot Interaction Flow Run Errors: one row per failure. In a clean build, add a lookup from each error row to its run.

   Use the exact column types and lengths in Dataverse tables (create tables). Choice values come from your publisher, so when a flow sets a choice column, pick the option by its label.

5. Create the environment variables

   Create only the variables your secret option needs, from the seven in Environment variables:

   • Always: Audit_Tenant, Audit_AppRegID, Audit_Authority and Audit_Audience.
   • Option A or B: the switch Audit_UsingAKVtruefalse.
   • Option A: the Secret-type variable KeyVaultSecret, pointing at your vault.
   • Option B (not recommended): the Text variable Audit_Secret.
   • Option C: your own variables for the certificate's PFX and password, ideally Key Vault-backed.

   Leave every default value empty or non-sensitive, because defaults are always exported with the solution (environment variables in flows).

6. Create a connection reference

   In the solution, select New > More > Connection reference, choose Microsoft Dataverse, and map it to a connection owned by the account that will own the flows. Both flows use this one reference for every Dataverse action. The HTTP actions need no connection: they sign in with the app registration (connection references). If you use a custom connector for the Graph calls, create it in the solution first, and add a connection reference for it too.

7. Build the flows

   Create a new scheduled cloud flow inside the solution (daily recurrence), then add the actions in this order. The screenshots show each part.

   1. Phase A: initialise the variables and the collection window (one UTC day, from midnight four days ago to midnight three days ago).
   2. Phase B: resolve the secret for your option, with secure inputs and outputs.
   3. Phase C: add the run log row with Flow State set to Running.
   4. D1 to D3: create the audit log query, wait for it to finish, then read the records page by page.
   5. D4: upsert each record into Copilot Interactions and update the run counts.
   6. Phases E and F: write error rows and fail the run if collection is incomplete.

   Then build the manual flow the same way, with a manual trigger that takes two dates and the few differences listed there. Turn on the recommended secure inputs and outputs, size the loop limits for your volume, and add a note to each action saying why it exists (solution-aware flows).

8. Validate

   Run the manual flow over one day that's at least four days old. For example, if today is the 10th, use StartDate …-06 and EndDate …-07. Then confirm all of these:

   • the run Succeeded in run history;
   • the new Copilot Interaction Flow Runs row shows Records Retrieved above 0, if anyone used Copilot that day;
   • the Copilot Interactions row count for that day is close to a Purview Audit search over the same day;
   • Copilot Interaction Flow Run Errors has no new rows;
   • run history doesn't show the secret.

   Then work through the clean-build checklist.

9. Lock it down

   Give report readers a read-only security role, schedule a bulk-delete job for your retention period, confirm the data policy and set up monitoring (see Failure states and monitoring). Secure run history as described in Securing run history.

10. Promote it

    Before you export, remove the current value of every environment variable from the solution, so no secret, tenant ID or client ID leaves the environment. Then export the solution as managed and import it into test, then production (export solutions), and set the environment variable values in each target environment.

Dataverse

Data model

The flows write to three custom tables. All three are user or team owned, and none uses auditing, change tracking or alternate keys. This guide includes no security role, so create your own. See Dataverse tables written by the flows and Upsert mapping.

The tables aren't related in the reference build. A clean build adds a lookup from each error row to its run.

For reporting ideas, see Reporting.

Build your own view

Reporting

The build is a collector only: you build the report. Connect Power BI to the three tables with the Dataverse connector, or use any Dataverse client.

Ideas for a first report

• Interactions over time from CreationDate, once you store UTC (see Data model).
• Sessions as distinct Thread Id values. Thread Id is populated on every row in the reference data, so it's a better session key than Conversation Id.
• Where people use Copilot, by App Host and AppIdentity.
• Agents, by Agent Id and Agent Platform, joined to your own agent inventory.
• Collection health from the run log and error rows, including days with no rows (see Failure states).

For a fuller example that combines this data with agent inventory from several platforms, see Custom Agent Reporting – Architecture. There, these tables sit in one reporting environment alongside the other collectors' tables, so its Power BI model reads them all through a single Dataverse connection. If you build both, put them in the same environment.

Run

Failure states and monitoring

Monitoring options

The reference build sends no alerts. Choose at least one:

• Power Automate's built-in failure notification emails to the flow owner.
• An alert flow on the Dataverse cloud flow run metadata.
• The CoE Starter Kit.
• A Power BI tile showing days with no rows.

Run history is kept for 28 days (limits, missing run history), so record failures somewhere that lasts longer. See also the error-handling guidance, and Failure paths and Known issues in the action reference.

Day two

Operations

Fix

Troubleshooting

No matching rows. Clear the filter to see all ten.

Know before you rely on it

Limitations

Where the pattern comes from

Origins and credit

This build is adapted from the audit log collection pattern in the Microsoft Power Platform CoE Starter Kit, described in Collect audit logs using an HTTP action with Microsoft Graph. The kit is open source under the MIT licence at github.com/microsoft/coe-starter-kit. Microsoft describes it as no longer actively maintained, with its core capabilities now in the Power Platform admin center. Credit for the original pattern belongs to the CoE Starter Kit team.

This build changes the pattern: it collects only CopilotInteraction records into their own table, logs every run and every failure, lets you choose Key Vault or a plain-text variable (not recommended) for the secret, fails the run loudly when collection is incomplete, and adds a manual back-fill flow. Some legacy actions and variables from earlier versions remain in the reference build and are labelled Legacy in the reference. It keeps the kit's HTTP actions for the Graph calls; for your own build, consider a custom connector instead. See Origins and credit in the action reference.

See also

Related

• Audit logs for Copilot and AI applications
• Get started with auditing
• Auditing solutions in Microsoft Purview