Three months with Azure Data Manager for Energy: what we wish we had known on day one

Over the past quarter year our Forward Deployed Engineering team built and operated a data platform on Azure Data Manager for Energy (ADME), Microsoft's managed OSDU service. The workload was ordinary: ingest records from source systems with Azure Data Factory, expose them through a .NET API that queries OSDU Search, and put Power BI and a small React app on top. Synthetic data first, then real data, then a handover.

Nothing about that plan is exotic. Still, ADME tripped us up in ways the documentation does not spell out. This post is the list we wanted when we started. It covers provisioning, networking, identity, entitlements, schemas, Search behaviour, ingestion and operations. Everything here was verified against a live instance.

1. Provisioning: treat the instance as a one-shot decision

ADME is not a resource you tear down and recreate on a whim.

• Instance name (1 to 15 lowercase alphanumerics), region, SKU, data partition names and the networking model are effectively permanent. There is no relocate. A region change is a full redeploy plus data migration.
• Provisioning takes up to two hours. We measured 1h 43m once. Do not cancel it.
• Regional capacity is finite. We needed five attempts across three regions before one had backend capacity. A failed ARM deployment still leaves a live resource in Failed state. Delete it (type Microsoft.OpenEnergyPlatform/energyServices) and confirm no ME_* managed resource groups linger before you retry.
• Each instance needs its own Entra app registration. Its application id becomes the instance id for the data plane and cannot be changed later. The app id plus a client secret is admin access to everything, so plan rotation and vaulting before you create it.
• The Developer SKU is a single partition with no redundancy. It is fine for a vertical slice and wrong for production. Extra partitions and multi-region disaster recovery are billed separately.

Terraform

There is no native azurerm resource for ADME. We tried two approaches.

1. Wrap the ARM template in a template deployment resource. It works for creation, but you get no drift detection and no in-place updates of ADME properties, and an accidental replace would destroy the instance.
2. Create the instance by hand and consume it from Terraform with an azapi_resource data source (Microsoft.OpenEnergyPlatform/energyServices@2025-12-15). The data source gives you the DNS name, the auth app id and the partition ids at plan time. If the instance is missing the plan fails early instead of mid-apply.

We ended on option 2 and moved ADME into its own state so the day-to-day root never plans against it. We considered lifecycle { ignore_changes = [all] } and rejected it. It hides drift instead of preventing it.

Schemas, entitlements and CORS live outside Terraform. More on that below.

Cost

In our resource group the ADME instance was roughly 99 percent of monthly spend. A single budget on the resource group therefore only tells you about ADME. Put a second budget on a tag that excludes it so you can see when the rest of the estate moves. Also note that the Cost Management API rate limits aggressively with 429s if you poll it from a pipeline.

2. Networking: private endpoints are supported but fiddly

ADME supports a private endpoint. Wiring it correctly took several iterations.

Things that were not obvious:

• One ADME private endpoint exposes the API hostname plus six service-managed Blob members. Only the API name needs privatelink.energy.azure.com. The Blob members need privatelink.blob.core.windows.net, and you only need them if you use File Service, Dataset Service, DDMS bulk transfers or signed Blob URLs. Search, Storage and Schema work with the API zone alone.
• Azure allows one DNS zone group per private endpoint. If someone attaches a zone group out of band under a name other than default, a later Terraform create fails instead of replacing it.
• Replacing the private endpoint can change its private IP and needs re-approval on the ADME side. Only do that while public access is still on.
• The Data Factory managed private endpoint uses the literal group id Azure Data Manager for Energy, spaces included. Older versions of the typed azurerm provider rejected it, and azurerm_private_endpoint.subresource_names rejected it too. We used azapi resources until the provider accepted the string. The ADF-side connection still needs manual approval, and we chose not to give delivery pipelines that permission.
• There are three independent approvals on one estate: the subnet private endpoint, the ADF managed endpoint to ADME, and the ADF managed endpoint to landing storage. Approving one does not approve the others.
• Setting publicNetworkAccess to disabled is a one-way write. Gate it behind an explicit "private resolution verified" flag rather than a plain boolean, because flipping the variable back does not reopen anything.
• Microsoft-hosted pipeline agents cannot reach a private-only instance. Every onboarding job that touches the data plane (schemas, legal tags, entitlements, smoke tests) has to run on VNet-injected agents before you can close public access. This was our longest blocker.
• Developer workstations cannot reach a private-only instance either. Plan for a jumpbox or Bastion, and expect tokens pasted through a Bastion clipboard to arrive mangled (IDX14102). Trim and check for exactly three dot-separated segments before you use one.

3. Identity and tokens: three patterns, only one of which preserves per-user access

ADME's data plane is OAuth 2.0 bearer only. Every request carries Authorization: Bearer and data-partition-id. The scope is always {adme-auth-app-id}/.default (bare GUID or api://GUID, both work). Our auth design went through three stages.

Stage 1: managed identity client credentials

A managed identity requesting {auth_app_id}/.default works for background workloads. Data Factory uses a REST linked service with authenticationType: ManagedServiceIdentity and aadResourceId set to the ADME app id. Web activities use authentication.type: MSI with the same resource.

It does not work for a user-facing API. Two problems:

• The ADME auth app registration is delegated-only. It exposes user_impersonation and has no app roles. Our first Search call from the API returned 401.
• Even if it worked, one identity for everyone flattens per-user OSDU entitlements. Every caller sees whatever the identity sees.

Stage 2: forward the caller's token

Forwarding only works if the caller already holds a token whose audience is the ADME app. Power BI and normal OAuth clients request the API's audience. That token is rejected by ADME with a 401.

Stage 3: on-behalf-of exchange

The pattern that worked:

1. The API validates the inbound JWT against its own audience.
2. If the token's aud is already the ADME app id, forward it unchanged.
3. Otherwise call MSAL AcquireTokenOnBehalfOf with the ADME scope and the caller's token as the user assertion.
4. Sign the OBO request with a managed identity token for api://AzureADTokenExchange/.default instead of a client secret. This requires a federated identity credential on the API app registration.

Requirements to make step 3 and 4 succeed:

• The API app registration needs the ADME user_impersonation delegated permission with tenant-wide admin consent. requiredResourceAccess on the app only records the request. The proof of consent is an oauth2PermissionGrants entry with consentType: AllPrincipals and a null principalId. A Principal grant means one user consented and is not enough.
• The federated credential has subject set to the managed identity's object id, issuer https://login.microsoftonline.com/{tenant}/v2.0 and audiences ["api://AzureADTokenExchange"].
• The scope passed to OBO must end in /.default. We guard this at startup.

Entra performs OBO only for user principals and only against delegated scopes. An app-only token from a pipeline service connection passes JWT validation but cannot be exchanged. Our post-deploy smoke test marks those checks SKIP instead of failing. Proving the data path end to end always needs a signed-in user.

Error codes we hit on the way

Use user-assigned managed identities

OSDU entitles by application (client) id. The federated credential binds to an object id. A system-assigned identity mints new ids whenever the host resource is recreated, for example on a plan tier swap or a region move. That silently breaks both OBO and every OSDU grant, and if a separate team owns Entra you wait days to repair it.

We switched the API and Data Factory to user-assigned identities, added prevent_destroy, and put a CanNotDelete lock on them.

4. Entitlements: the source of most of our 401s and 403s

Over a two-week window, 33 of about 50 API server errors were OSDU 401 or 403 responses. Nearly all traced back to entitlements, not token plumbing. Three lessons.

Members must be bare ids

The Entitlements API is PUT /api/entitlements/v2/groups/{group}@{partition}.dataservices.energy/members with body {"email": "...", "role": "MEMBER"}. The email field must be the bare Entra object id for users and groups, and the bare application id for managed identities. The {oid}@{tenant-id} form is accepted, stored and never matches any token. You end up with dead entries that look correct in the group listing.

Diagnose by comparing members/{plain-oid}/groups against members/{oid}@{tenant}/groups. The symptom of a missing membership is 403 Forbidden with "Access denied to data partition".

Entra groups are not expanded

Adding an Entra security group's object id to an OSDU group succeeds and grants nothing to the group's members. Their tokens still get 401 from Search. We confirmed this empirically after a day of debugging token audiences. The documented pattern is to map one Entra group to one OSDU group and manage membership in Entra, but effective access still required direct user object id grants in our tenant. Do not debug token plumbing first when a new user hits 401. Check their direct membership.

Grants are additive and idempotency needs care

We manage grants from a per-environment JSON manifest applied by a pipeline job. Rules that held up:

• Validate every input before mutating anything.
• Do an authorized preflight GET on each group.
• POST the membership. Treat 200 and 201 as granted. Treat 409 as "already a member", then re-read and verify the role matches. A MEMBER entry must not satisfy an OWNER request.
• Collect per-grant failures and report at the end instead of aborting on the first one.
• Keep the bearer token out of process arguments. Use curl config files for headers, and --retry 3 --retry-all-errors for transport.
• Include a negative probe with a wrong-audience token. A 2xx there is a failure.

The script adds and never revokes. Removing an entry from the manifest changes nothing in ADME. We never automated revocation and flagged it as a privilege creep risk at handover. Also note that OSDU has no default privileges beyond the pre-provisioned service.*, data.* and users.* groups, and service.entitlements.admin holders can change anyone's access. Standing developer admin membership belongs in your threat model.

A recreated managed identity means removing and re-adding the entitlement by hand and redoing the federated credential. See the previous section for why we use user-assigned identities.

5. Schemas and records

Reuse well-known schemas, add custom kinds only where OSDU has nothing

We used standard osdu:wks master-data kinds (Well, Wellbore, TubularAssembly, Rig, ActivityPlan and friends) and created custom kinds only for entities OSDU has no counterpart for. Two rules we adopted after getting it wrong once:

• Do not force-fit an entity into the nearest well-known kind. One custom kind per entity type.
• For links between records with no real foreign key, land the link as its own custom kind carrying both source values, the matching score, method and run id. A standalone link record is independently searchable and auditable. Search filters nested arrays poorly, so embedded arrays of links are a dead end.

Kinds are four colon-separated segments, authority:source:entity:major.minor.patch. We used {authority}:{source}:{entity}:1.0.0 for custom kinds and kept the version on additive property changes.

Register custom schemas as DEVELOPMENT

schemaInfo.schemaIdentity{authority,source,entityType,major,minor,patch} with status: "DEVELOPMENT" allows PUT updates. A finalized schema rejects PUT and needs a version bump. There is no delete endpoint exposed, so throwaway schemas persist forever. Name them carefully.

Standard schemas have dependencies. Register abstract schemas first, because master-data schemas $ref them. We vendored the OSDU data-definitions repository at a pinned tag (37 abstract plus 9 master-data schemas for our slice) and register with GET first, skip on 200, POST otherwise.

The indexer does not accept nullable unions

This one cost us a week. Schema Service happily accepts "type": ["string", "null"]. The Indexer does not. A forced reindex (POST /api/indexer/v2/reindex?force_clean=true) returns HTTP 400 Failed to parse the schema and Search reports totalCount = 0 for the kind while Storage GET returns every record.

Declare the scalar type, omit the field from required, and both omitted and explicit null values index fine. After the fix, records became searchable about 55 seconds after reindex.

Record envelope

Every record needs id, kind, acl and legal. Storage rejects anything missing acl or legal.

{
  "id": "{partition}:{kind-stem}:{businessKey}",
  "kind": "osdu:wks:master-data--Well:1.4.0",
  "acl": {
    "owners": ["data.default.owners@{partition}.dataservices.energy"],
    "viewers": ["data.default.viewers@{partition}.dataservices.energy"]
  },
  "legal": {
    "legaltags": ["{partition}-{tagname}"],
    "otherRelevantDataCountries": ["US"]
  },
  "data": { }
}

Deterministic ids are the single most useful decision we made. PUT /api/storage/v2/records is upsert by id, so a stable id derived from the business key makes every re-run idempotent. We compute the same expression in C# and in the ADF data flow. Escape the colons when you put an id in a URL path.

One caveat: records ingested by an ETL path may carry ids you cannot recompute from the data. When patching, always read id from the Search hit and echo it back. Never regenerate it.

Storage PUT returns 2xx on partial failure

The response carries recordCount, recordIds and skippedRecordIds. A 201 with half the batch in skippedRecordIds is a normal outcome. Diff the skipped list against what you sent. The HTTP status will not tell you.

Legal tags are mostly immutable

countryOfOrigin cannot be changed on an existing legal tag. Changing it means a new tag, new defaults in every dataflow and a full re-seed of the partition. Decide the country before you load anything. 409 on create means the tag exists and is safe to ignore. We gave the dev tag a far-future expiry so it never expired mid-demo.

Patching

Our patch tool refuses to run without a snapshot directory. It GETs each record, writes the raw JSON to disk, deep-merges into data only, and PUTs the full envelope with id, kind, acl and legal untouched. There is no PATCH endpoint. 404s are recorded as missing, never created.

Bulk delete by kind is a loop: search ids with limit: 1000, delete each, repeat until Search returns zero. Treat 404 on delete as success.

6. Search: indexing lag, paging limits and the one rate limit

All reads go through POST /api/search/v2/query with kind, query, limit, offset and returnedFields.

Indexing lag

Storage PUT is not immediately searchable. We consistently saw 30 to 60 seconds of lag. Every place that writes then reads has an explicit wait:

• Seeder verification polls totalCount every 5 seconds, up to 24 attempts, hard timeout 120 seconds, pass when observed equals expected.
• The ADF orchestrator has a fixed 90-second Wait activity between the load stage and the linkage stage, because linkage resolves references through Search.
• The e2e harness polls for 90 seconds and counts only ids carrying its run prefix so parallel runs do not confuse each other.
• The patch tool sleeps 120 seconds between dependent edit sets.

A cheap count query is { "kind": "...", "query": "*", "returnedFields": ["id"], "limit": 1000 } and read totalCount. returnedFields with ["id","kind","data","acl","legal"] gets you full envelopes in one call, much cheaper than per-record GET.

Paging window

limit is capped at 1,000 and offset + limit must stay at or below 10,000. Beyond that you need a cursor-based drain or a narrower query. Our API wraps (offset, limit) in an opaque Base64Url cursor and throws if a full drain reaches the ceiling with fewer records collected than totalCount. We also cap cursor length at 2,048 characters as a cheap DoS guard.

Rate limits

The only documented Search rate limit we found (January 2026 release notes) applies to fully unbounded *:*:*:* kind queries. Bounded kind queries are exempt. We ran about 20 concurrent bounded Search and Storage calls from Data Factory without seeing a 429. We still added retry: 2 on each activity and a 429 branch in the API's error mapping.

Resilience settings in the API

The API fans out four to six parallel Search drains per dashboard request, 1,000 records per page. The HttpClient uses the standard resilience handler with 10 seconds per attempt and 30 seconds total. A TimeoutRejectedException after 10 seconds means Search did not answer, which we map to 503. Every request carries a Correlation-Id GUID that we join on in logs. Response bodies from ADME are logged at Trace only and never placed into exceptions, both to avoid over-disclosure and because a naive truncation once split a UTF-16 surrogate pair.

This design holds while each kind stays below a few thousand records. If that changes, the options are server-side aggregation, a cache or materialized projections. We documented the trigger (429s or a kind above a few thousand records) rather than build ahead.

7. Ingestion with Data Factory

• The REST sink to Storage works with writeBatchSize: 50, requestInterval: 100 ms and a 2-minute request timeout. Our CLI seeder uses batches of 100. Both are well within what Storage accepts.
• Parallel ForEach with isSequential: false and batchCount: 10 cut a linkage run from about 4 hours to 35 to 40 minutes for a few hundred link records with zero duplicates. batchCount allows 1 to 50.
• Bodies inside a parallel ForEach must be variable-free. Pipeline variables are global and race across iterations.
• Make the linkage step idempotent by re-PUTing only when the freshly searched reference array differs from what is on the record.
• Lucene syntax for field filters is data.wellId:"X". Escape backslashes and quotes before embedding a value in the JSON expression.
• The Storage route runs no schema validation. Bad records land and only fail later in the indexer or in a consumer.
• ADF has no triggers in our setup. Reloads start from the studio with explicit partition, ACL owner and viewer, legal tag and origin country parameters. Submission is not success. Wait for Succeeded.
• Keep the ADME REST linked service and the managed private endpoint in the ADF source artifacts and export them with the ADF utilities package. privateLinkResourceId is an ARM parameter supplied at deploy time from the ADME resource.

8. Operations

Diagnostics and alerting

Send OEPDataplaneLogs, OEPAuditLogs and AllMetrics to Log Analytics. Dataplane logs carry no HTTP status code, so alert on the TotalHttpRequests metric split by response_code. List the 5xx codes explicitly. A prefix match on 5 also matches 415. Our rule fires on more than five responses in the 500 to 504 range within 15 minutes.

CORS and the Admin UI

CORS for a browser client is set on the control plane under dataPartitions[].cors.allowedOrigins, not through the data plane. The SPA redirect URI goes on the ADME auth app registration, which needs Application.ReadWrite.OwnedBy. In a governed tenant that means an Entra owner does it out of band.

The OSDU Admin UI container image is not on the Microsoft Container Registry under the documented tag (MANIFEST_UNKNOWN). It lives on the OSDU community registry behind per-user token auth. Mirror it into your own registry and pull with a managed identity.

Backups

We did not back up the partition. Data was regenerated deterministically from seed data plus a scenario, and Git was the backup. That works for synthetic and staged data. It does not answer recovery time or data loss targets for real data, and those remained open at handover along with support plan ownership.

9. Testing against a live instance

• Every seeder command defaults to dry run. --live is required to touch ADME. This saved us more than once.
• A read-path harness that seeds committed data straight into Storage with the CLI, waits for indexing, counts via Search, exercises the API, and tears down again is worth building early. It bypasses the ETL and isolates ADME or API failures from ingestion failures. We ran it before every demo and whenever counts looked wrong.
• A smoke test with a transient legal tag: create tag, PUT record, GET, PUT update, DELETE, GET expecting 404, delete tag. It proves the service and the identity in under a minute.
• Post-deploy checks split into three tiers: anonymous (health, OpenAPI contract, 401 with a WWW-Authenticate challenge), authenticated, and ADME-dependent. ADME-dependent checks report BLOCKED rather than fail when the pipeline identity cannot do OBO.
• For manual user-context validation, mint a delegated token through a consented SPA client using auth code plus PKCE. The Azure CLI often cannot mint a token for your own API audience in a locked-down tenant (see AADSTS650057).
• Offline development mode serves the same seed JSON through a fake Search client so the API and dashboard can be developed without ADME at all.
• The onboarding pipeline needs secretless tokens. Use a workload-identity-federated service connection requesting https://energy.azure.com/.default for pipeline jobs, and keep the instance-specific {auth_app_id}/.default scope for runtime consumers. Validate the token's tenant, caller, version and audience before any data-plane call, and never log an Authorization header.

10. The order of operations that worked

For a fresh environment:

1. Deploy the instance by hand. Wait the full two hours.
2. Consume it from Terraform via the azapi data source. Provision private endpoint, DNS zones, user-assigned identities, Data Factory, Function App.
3. Approve all three private endpoint connections.
4. Mint a data-plane token from a federated service connection.
5. Create legal tags. Decide countryOfOrigin now.
6. Register abstract schemas, then master-data schemas, then custom schemas as DEVELOPMENT.
7. Add managed identity application ids to the OSDU data groups.
8. Grant users by bare object id from a manifest.
9. Set CORS on the control plane. Ask an Entra owner for the SPA redirect URI and admin consent on the OBO permission.
10. Seed records in batches. Poll Search until counts match.
11. Run the smoke test and the read-path harness.
12. Only then flip publicNetworkAccess off, and only once every agent that touches the data plane is inside the VNet.

Things we would still change

Entra group expansion in entitlements is the gap that hurt the most. Direct object id grants work, but they do not scale to an organization and they need a revocation story we never built. If you have the option, build a group sync step between Entra and OSDU groups on day one instead of maintaining a manifest of individuals.

The second regret is closing public network access late. Every week of "we will do it once the agents are in the VNet" was a week where OSDU entitlements plus Entra were the only boundary around the data plane. Put VNet-injected agents in the very first sprint.

Everything else on this list is a gotcha with a clear fix. ADME does what it says once you know where the sharp edges are.