feat(app): AppClient for native (v3) app workflow APIs (BLDX-1472) by Aryamanz29 · Pull Request #959 · atlanhq/atlan-python

Aryamanz29 · 2026-06-23T11:09:08Z

What

Adds a first-class AppClient (client.app, sync + async) for Atlan's native app workflow APIs, plus typed, auto-generated builder classes for connector apps (crawlers + miners) under pyatlan.model.apps.

AppClient (`client.app`)

Full lifecycle over the /v1/app* APIs — create, get, get_all, update, submit, get_run, cancel_run, add_schedule, remove_schedule, delete, describe, get_input_contract — with sync + async parity.

App builders (`pyatlan.model.apps`)

A fluent builder per connector app that mirrors the "new app" wizard (Credential → Connection → Metadata): it vaults the credential, mints the connection qualified name, and assembles the full payload — so a caller never hand-builds a connection object or guesses an input key. For example:

from pyatlan.model.apps import BigqueryCrawler

response = (
    BigqueryCrawler(client)
    .service_account(email=..., service_account_json=sa_json, project_id="my-project")
    .connection(name="production", admin_roles=[client.role_cache.get_id_for_name("$admin")])
    .include({"my-project": ["analytics"]})
    .run(name="bigquery-prod")
)

Classes are generated from each app's UI configmaps (union'd with a small manifest for apps not currently running on the tenant), via a single command — no hand-maintained payloads:
```
uv run generate-apps
```
~33 connectors covered (crawlers + miners). A few are intentionally hand-maintained (_HAND_WRITTEN: bigquery, databricks, kafka) where the configmap can't express the UI (e.g. databricks' multi-mode asset selection; kafka vaults under a different connector-config name than its configmap).
For apps without a builder, client.app.create(app_id=..., inputs={...}) accepts a raw inputs dict.

Deprecations

WorkflowClient (client.workflow) and model.packages.* now emit a DeprecationWarning — they target the legacy workflow surface, superseded by client.app.

Docs

Customer-facing docs (new app references + deprecation of the old packages docs) ship separately: atlanhq/atlan-docs#1302.

Status — experimental

Released as an experimental minor: it supersedes the legacy package classes, and the new auto-generated method/param names differ from the old ones and may evolve as customers exercise them.

Adds `client.app` (sync) / `AsyncAtlanClient.app` (async) — an Argo-free client for the Automation-Engine (Temporal-native) `/v1/app*` surface that superseded the old Argo WorkflowClient after the AE migration. Generic + contract-driven: workflows are created from an `app_id` plus a plain `inputs` dict validated server-side against the app's live input contract, so a connector never needs a hand-maintained `model/packages/*` class. Discovery (`get_app`, `get_input_contract`) lets callers introspect fields at runtime. Surface: get_app, get_input_contract, create, get_all, get, update, delete, submit, get_run, cancel_run, add_schedule, remove_schedule (sync + async parity). - constants.py: 12 v3 endpoints under the api/service prefix - model/app.py: request/response models (snake_case; camelCase aliasing disabled) + AppInputContract with runtime discovery helpers - client/common/app.py: shared prepare_request/process_response (sync+async reuse) - 19 unit tests; all 12 endpoints verified live on developer-experience.atlan.com Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…es (BLDX-1472) - tests/integration/test_app_client.py: live coverage of the v3 surface (describe, input-contract, create→get→list→update→schedule add/remove, delete cleanup); run=False + agent mode so no crawl/credential is needed. 7 tests pass against developer-experience.atlan.com. - Deprecate the Argo-era paths with runtime DeprecationWarnings + docstrings pointing to AtlanClient.app (AppClient): * AbstractPackage.__init__ (covers all model/packages/* builders) * WorkflowClient.__init__ (DeprecationWarning is ignored by the test config, so no suite breakage.) - docs: add App Client + App models pages; mark Workflow Client and Packages deprecated, pointing to the native v3 client. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

linear · 2026-06-23T11:09:11Z

BLDX-1472

APPPLAT-271

… (BLDX-1472) Adds an ergonomic alternative to raw inputs dicts that needs zero per-connector code. `client.app.inputs(app_id, entrypoint)` fetches the live contract and returns a builder with: - universal helpers shared by every app: .connection(), .direct_credential() xor .agent() (SDR/agent_json), .filters() - generic .set()/.update() validated against the contract (unknown field → error with difflib suggestions; .build() enforces required fields) create()/update() now accept a dict OR a builder (both sync + async). Both input styles remain first-class: raw dict for FE payloads/power users, builder for guided construction. Tests: builder mechanics + per-app config assertions (bigquery crawler/miner, oracle SDR, snowflake, postgres, powerbi, mssql, trino SDR) covering direct and agent modes — mirrors the old test_packages payload assertions. Builder→create verified live on kill-argo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…472) Adds pyatlan/model/app_inputs/: an AppInput base + a generator (pyatlan.generator.generate_app_inputs) that emits ONE module per app/entrypoint (bigquery_crawler.py, bigquery_miner.py, snowflake_crawler.py, ...) from each app's live /v1/apps/{app}/inputs JSON Schema — typed fields, contract defaults, titles as field docstrings. Mirrors the old packages/* one-file-per-connector layout but generated, not hand-maintained. So customers no longer guess property names: from pyatlan.model.app_inputs import BigqueryCrawlerInputs client.app.create(app_id="bigquery-crawler", name="prod", inputs=BigqueryCrawlerInputs(connection=..., enable_nested_columns=True)) create()/update() (sync + async) now accept a dict, an AppInputsBuilder, or a generated AppInput. Generated from the live contract (titles/defaults/types); richer PKL descriptions can be layered later. Verified live on kill-argo (typed class -> create -> delete). 38 app unit tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…rnals (BLDX-1472) Generator: - discover apps from the tenant's deployed workflows + resolve entrypoints; fetch contracts concurrently (ThreadPoolExecutor) — was serial/slow. - denylist infra/internal fields the UI never surfaces (output_dir, checkpoint_dir, load_to_atlan, publish_dry_run, atlas_auth_type, max_*_activities, handshake ids) so generated classes mirror the user-facing form, not the full data contract. - emit one module per app under pyatlan/model/app_inputs/ (regenerated: 18). Tests (tests/unit/test_app_generated_inputs.py): auto-discover every generated class and assert the contract they all honour — AppInput subclass, _APP_ID classvar, no-arg instantiation + dict serialization, NO internal fields leak, marked AUTO-GENERATED, extra-tolerant, and create() accepts each. Source-agnostic so they survive a future source change (e.g. uiConfig). 166 app unit tests pass. Note: true UI-parity curation (labels/groups from each app's uiConfig PKL) is a follow-up — uiConfig is not served by any tenant endpoint, only the app repos. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…) API - pyatlan.model.apps: 33 per-connector builders + typed *Inputs, generated from each app's UI configmaps (full legacy-package parity + new connectors); BigqueryCrawler is the hand-written flagship the generator never overwrites - AppBuilder mirrors the UI wizard (Credential -> Connection -> Metadata): vaults the raw credential server-side, mints the connection QN, assembles the payload; split create() (run=False) / run() (run=True) - rename AppClient.get_app -> describe (vs get for a workflow) - generator: `uv run generate-apps` console script; snake_case params, Literal enums, example docstrings, per-app config tests; all-or-nothing write - remove the AppInputsBuilder experiment - robust integration suite covering every AppClient method (schedule add/remove round-trip); fix add_schedule null-timezone (default UTC) - scrub internal platform jargon + ticket refs from the public surface - bump 9.8.0 + changelog Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…r names - AppBuilder.connection(): `name` is now optional; pass only `qualified_name` to reference an existing connection (miners). The server resolves the connection and its default credential from the QN. - derive connectorName from the connection QN (default/{connector}/{epoch}) so referenced connections report the right connector, not the app-id fallback - send credential_guid="" when no credential is supplied (e.g. miners) so the contract's non-null string requirement is satisfied - generator: derive _CONNECTOR_NAME for credential-less apps by stripping the atlan- prefix and -crawler/-miner suffix; miner docstring examples now select an existing connection by qualified_name (no credential_guid, no name) - tests: existing-connection-by-QN + connector-from-QN coverage Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…enerated imports - existing-connection (miner) flow now needs ONLY the connection QN: .connection(qualified_name=...) — on create()/run() the builder looks up the connection via FluentSearch and reuses its defaultCredentialGuid, so the caller never supplies a credential. connection name is optional. - generator emits exact per-module imports (no blanket `# noqa: F401`) and now self-formats its output (ruff check --fix + format) for deterministic, clean files - miner docstring examples show QN-only connection (no credential step) - tests: miner credential auto-resolution + connector-from-QN coverage Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…placeholder defaults - credential extraction now descends into the jdbcUrl/AdvancedJDBCUrlGroup group when there's no top-level auth, so JDBC-URL connectors (e.g. mssql) get real auth methods (.basic/.azure_ad_sp/.ntlm with host/port/database) - credential method names use the concise auth-type key (basic, ntlm, jwt, ...) - fields with no explicit default fall back to a CONCRETE ui placeholder (numeric like port 443, or a URL) — hint placeholders are ignored - host/port exposed (required when the form has no default) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ve, postgres) JDBC-URL connectors split credential fields inconsistently: mssql nests everything under jdbcUrl, while hive/postgres keep auth-type/auth objects at the top level but host/port/extra under jdbcUrl. The earlier descend-or-top-level logic missed host/port for the latter. Now merge the jdbcUrl group with the top level (top wins on conflict), so auth methods always include host/port/database regardless of where each field lives. .basic/.kerberos/etc. for hive & postgres now expose host/port. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

… + core) Builders now support N credential widgets, not one — generated from every credential widget in the inputs form. Each auth-type method vaults its raw credential into that widget's specific input field: - dbt: .api(...) -> api_credential_guid (Cloud); .aws()/.gcp()/.azure(...) -> object_store_credential_guid (Core) - single-credential connectors unchanged (target the standard credential_guid) AppBuilder tracks staged credentials by target field (_raw_creds: field->Credential); _assemble vaults each into its field. Verified live: dbt Cloud create succeeds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…al` key Only the shape-recognized `credential` key is vaulted by the create endpoint; placing a raw credential directly in a named field (e.g. dbt's api_credential_guid) passed validation but did NOT vault, so the run failed credential preflight ("Host URL and API token are required"). Now every staged credential is sent via `credential` (with credential_guid=""), and the server routes the issued guid to the right field by connectorConfigName. Verified live: dbt Cloud resolves its credential. Multi-credential builders still expose a method per auth-type; one is staged per run (selected via .source()). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

dbt reads its credential from a named field (api_credential_guid), not the standard credential_guid. The create endpoint only vaults the shape-recognized `credential` key (which fills credential_guid) — so a named-field credential was never resolved and the run failed preflight ("Host URL and API token are required"). _create() now vaults each named-field credential via credentials.creator(test=True) and places the issued guid in that field; the standard credential_guid still uses the `credential` shape-key. Verified: dbt Cloud resolves with a real guid in api_credential_guid. Also: cleaner generated docstring example samples ({}/0/[] instead of "..."). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

A connector whose credential lands in a *named* field (dbt Cloud's api_credential_guid, dbt Core's object_store_credential_guid) failed at preflight with "Host URL and API token are required": the create endpoint only vaults the single magic `credential` key (which fills the standard credential_guid), so a raw credential placed in a named field was never vaulted, and the standard field isn't the one dbt reads. The builder now vaults each named-field credential up front via the credential store (the same endpoint the UI uses) and places the issued guid in the field; the standard credential_guid path is unchanged. Vaulting uses test=false — these connectors have no credential-test helper, and the workflow's own preflight validates the credential when it runs. Also fix dbt's include/exclude filters: object-typed filter fields are submitted as a JSON string (the contract accepts a string, not a nested object, and the worker json.loads it back), so the generator now types object fields as Union[Dict, str] and their setters json.dumps a dict argument. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

A builder with no entrypoint (_ENTRYPOINT = "") sent entrypoint="" to the create endpoint, which the server rejects as an unknown entrypoint for some apps. Send None when the entrypoint is empty so it is omitted and the app's default is used. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The databricks crawler's "Asset selection" is a multi-mode widget (include/exclude × hierarchy/regex) that the configmap can't express, and its filters are objects in the input contract — not the JSON strings the generic generator emitted. Fixes: - include_filter / exclude_filter are now objects (default {}); the create endpoint rejected the prior string "{}". - a single asset_selection(...) method covers all four widget modes, mapping to the native contract fields (include_filter / exclude_filter objects; {asset_type}_include_regex / _exclude_regex strings for schema/table). - workspace_credential_overrides stays a JSON string (contract type). - the module is hand-maintained now (added to _HAND_WRITTEN) so regen won't clobber the widget-faithful asset_selection. Verified end-to-end: create succeeds with hierarchy + regex selection. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ig name KafkaConfluent.basic() vaulted the credential under the configmap's credentialType (atlan-connectors-confluent-kafka), which the vault doesn't recognize — every create failed with 500 "failed to store credential". The vault registers this connector as atlan-connectors-kafka-confluent-cloud (per the legacy package); use that name, and send port 9092 as the contract expects. The module is hand-maintained now (added to _HAND_WRITTEN) so regen won't restore the configmap-derived name. Verified: create succeeds on a live tenant. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bigquery-miner is a deployed native app (has an input contract) but wasn't in the generator MANIFEST or discovered live, so no builder was emitted. Add it to the MANIFEST and generate the module + per-app test. It's a connection-selector miner: reference an existing bigquery connection by qualified name and the builder reuses that connection's credential. Verified: create succeeds on a live tenant. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…iner) The builder always sent extraction_method="direct", but some apps require another value — bigquery-miner's configmap pins extraction-method to "query_history". Add an _EXTRACTION_METHOD class var (default "direct") that the base honors, have the generator emit it from the configmap's extraction-method default when it differs, and set it on bigquery-miner. Verified: create succeeds with extraction_method=query_history; crawlers still send "direct". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Add a customer-facing docs section for the apps SDK, mirroring the old packages docs format (portal-style: frontmatter, language tabs, annotated snippets, admonitions): - index: Supported apps landing + "no builder? use a raw inputs dict" guidance - manage-apps: every client.app method (create, run, list, get, get_run, cancel_run, update, schedule add/remove, delete, describe, get_input_contract) plus a Raw REST API section (endpoint table + cURL) - one page per connector (33): each auth method, configuration options, and a runnable snippet — crawlers and miners Staged under docs/apps/ for porting to the developer portal; not wired into the pyatlan mkdocs nav. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

… method name - Replace the templated per-connector pages with hand-written, verbose docs (required/optional params marked, every config explained, real example values, connector-specific notes) for all 33 connectors + the manage-apps + index pages. - Rename the MongoDB builder MongodbatlasAtlas -> MongodbAtlas (with a generator class-name override so regen keeps it). - Fix a generated method name: fetch_excluded_project_s_query_history -> fetch_excluded_projects_query_history (the generator's _snake now drops possessive apostrophes / parens, so "project's" -> "projects"). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The app docs now live in the developer docs (atlanhq/atlan-docs#1302). Remove docs/apps/ here so the pyatlan PR carries only code (the AppClient, generated app classes, docstring fixes, and renames). Also reformat powerbi_crawler. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

test.py is a local manual smoke-test scratch file; it was tracked and failing ruff in CI. Untrack it and gitignore the manual test artifacts (test.py, test_apps/, native-v3-workflow-api-guide.md) so they aren't committed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ring include_metadata/exclude_metadata use _anchor_filter (returns a JSON string), but include_filter/exclude_filter were typed Dict[str, Any] — so passing a dict raised 'value is not a valid dict'. Type them Union[Dict[str, Any], str] like the other string-filter connectors. (Stale fields predating the object->Union generator fix.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Match admin_groups / admin_roles (and the adminUsers/adminGroups/adminRoles attributes they map to). 'admins' was inconsistent and surprising. Updated the base method, generator example output, generated-module docstrings, and tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…r SQL connectors glue + snowflake passed the filter dict through raw, so a plain {db: [schema]} wasn't anchored and {db: {}} was rejected by the contract. Use _anchor_filter (as athena/presto/postgres/etc. already do): {db: [schema]} -> {"^db$": ["^schema$"]}. Users no longer type ^...$ themselves. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…vert snowflake guess Glue's include/exclude_filter is nested {catalog: {db: {}}} (no regex anchoring), not the {^db$: [^schema$]} form. Add a _selective_filter helper and use it for glue. Revert the speculative _anchor_filter change to snowflake (its shape is unverified) back to the shipped pass-through. Filter shapes are per-connector. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…potency - Builders: serialize BI/object filters (Tableau, Power BI, QuickSight, Sigma) to the JSON-string `{id: {}}` form the input contract expects; send Power BI `odbc_dsn_config_mapping` as an object. Fixes 1000 contract-validation errors. - Miners: Oracle/Postgres/Teradata send extraction_method=query_history (per their configmaps); Oracle start_date documented as an epoch timestamp. - create(): retry without the entrypoint when an app registers its input contract at the default slot (1003 self-heal). - AppClient: get_all(name=) filter + idempotent create (resolve+reuse the existing workflow on a 409 duplicate name) — sync + async, with unit and integration tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Aryamanz29 and others added 2 commits June 23, 2026 16:15

Aryamanz29 commented Jun 23, 2026

View reviewed changes

Comment thread pyatlan/client/workflow.py

Aryamanz29 commented Jun 23, 2026

View reviewed changes

Comment thread pyatlan/model/packages/base/package.py

Aryamanz29 self-assigned this Jun 23, 2026

Aryamanz29 added the feature New feature or request label Jun 23, 2026

Aryamanz29 and others added 23 commits June 23, 2026 17:23

fix(app): reformat powerbi_crawler (ruff format)

04dd6c6

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Aryamanz29 and others added 4 commits June 26, 2026 15:12

OnkarVO7 approved these changes Jun 26, 2026

View reviewed changes

style: ruff format generate_apps.py (qa-checks)

3368070

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Aryamanz29 merged commit dc19d1e into main Jun 26, 2026
27 of 28 checks passed

Aryamanz29 mentioned this pull request Jun 26, 2026

fix(app): handle 409 duplicate-name raised as a bare AtlanError #960

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(app): AppClient for native (v3) app workflow APIs (BLDX-1472)#959

feat(app): AppClient for native (v3) app workflow APIs (BLDX-1472)#959
Aryamanz29 merged 30 commits into
mainfrom
BLDX-1472

Aryamanz29 commented Jun 23, 2026 •

edited

Loading

Uh oh!

linear Bot commented Jun 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Aryamanz29 commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

AppClient (client.app)

App builders (pyatlan.model.apps)

Deprecations

Docs

Status — experimental

Uh oh!

linear Bot commented Jun 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Aryamanz29 commented Jun 23, 2026 •

edited

Loading

AppClient (`client.app`)

App builders (`pyatlan.model.apps`)