Skip to content

Commit 2179358

Browse files
dwolfsonclaude
andauthored
feat(metadata_expert): add/update/delete open metadata type definitions (#428)
* feat(metadata_expert): add/update/delete open metadata type definitions Adds add_enum_def, add_type_def, update_type_def, delete_type_def and delete_enum_def (Egeria-api-metadata-expert.http, open-metadata-types and open-metadata-attribute-types/enum-defs). The scripts/omvs_audit.py check for metadata-expert now reports 0 missing. The OpenMetadataEnumDef / OpenMetadataTypeDef / OpenMetadataTypeDefPatch models declare only the documented fields and use extra='allow', so the rest of these deep Egeria beans (relationship ends, enum elements, ...) is passed through instead of being silently dropped by PyegeriaModel's extra='ignore'. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com> * docs(metadata_expert): record the live findings on update_type_def and delete_enum_def Live verification (2026-10-05): add_enum_def and add_type_def return the new GUID and the read-back is intact; delete_type_def works. update_type_def is rejected by the server (updatedBy is never set on the patch -- Egeria bug, ISSUE-124) and delete_enum_def answers 500 'unknown TypeDef' (ISSUE-125), leaving the throwaway enum behind. Docstrings now say so. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com> * docs(issues): ISSUE-125 still fails after the platform redeploy Re-tested live on the 2026-10-05 rebuilt platform: the stray enum survives the restart and delete_enum_def fails with the identical OMRS-CONTENT-MANAGER-500-001. Closes the 'retry after a restart' idea. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com> * docs(issues): ISSUE-124 still fails after the platform redeploy Re-tested live on the 2026-10-05 rebuilt platform: a patch to a throwaway entity type is rejected with the identical 'updatedBy set to null' error. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com> * docs(metadata_expert): warn on add_enum_def that the enum cannot be removed (ISSUE-125) The warning belongs on the method a caller reaches first, not only on delete_enum_def. Also notes on add_type_def that only entity types have been verified live. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com> --------- Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
1 parent b0570b4 commit 2179358

5 files changed

Lines changed: 457 additions & 1 deletion

File tree

‎PYEGERIA_ISSUES.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1257,6 +1257,67 @@ queries, rendering "Not measured" instead of `0` (odpi/egeria-workspaces
12571257

12581258
---
12591259

1260+
### ISSUE-124: `updateTypeDef` (Metadata Expert `open-metadata-types/update`) can never succeed -- the server builds the patch without `updatedBy`, which the repository services require
1261+
1262+
**Layer:** Egeria Server (not pyegeria) · **Status:** open, not yet reported upstream · **Found:**
1263+
2026-10-05, live verification of `MetadataExpert.update_type_def` (PR #428).
1264+
1265+
**What:** posting a valid `OpenMetadataTypeDefPatch` (type added through the API moments earlier,
1266+
`applyToVersion` 1) answers a 500 wrapping
1267+
`OMRS-REPOSITORY-400-069 Method updateTypeDef has detected that a TypeDef patch from qs-metadata-store has the
1268+
mandatory field updatedBy set to null ... TypeDefPatch{typeDefGUID=..., typeDefName=`PyegeriaTmpRecipe`,
1269+
applyToVersion=1, updateToVersion=2, newVersionName=`2.0`, updatedBy=`null`, ...}`. The request itself arrived
1270+
intact (the attribute definition is visible in the message).
1271+
1272+
**Cause, from Egeria's source (`origin/main`, 2026-10-01):** `OpenMetadataStoreRESTServices.updateTypeDef`
1273+
calls `converter.getTypeDefPatch(requestBody, methodName)`; `OMRSTypeDefConverter.getTypeDefPatch` takes no user
1274+
id and has no `setUpdatedBy` call, while `OMRSMetadataCollection.updateTypeDef` rejects a patch whose
1275+
`updatedBy` is null. `OpenMetadataTypeDefPatch` has no `updatedBy` field either, so a client cannot supply it.
1276+
Not a pyegeria defect: the SDK sends the documented body. Fix belongs upstream (pass `userId` to the converter and
1277+
set it).
1278+
1279+
**Re-tested 2026-10-05 after the platform was rebuilt and restarted** (image built 14:19Z from egeria main
1280+
`449ad06894`): a throwaway entity type (primitive attribute only) was added and patched with one new attribute.
1281+
The server rejected the patch with the identical `OMRS-REPOSITORY-400-069 ... updatedBy set to null`
1282+
(`TypeDefPatch{... applyToVersion=1, updateToVersion=2, newVersionName=2.0, updatedBy=null ...}`). The type was
1283+
then deleted and verified gone. **Confirmed still present on the rebuilt platform.**
1284+
1285+
**Effect here:** `update_type_def` is correct but unusable against current Egeria; its docstring says so.
1286+
1287+
---
1288+
1289+
### ISSUE-125: `deleteEnumDef` answers 500 "unknown TypeDef" for an enum the server lists -- an enum added through the API cannot be removed
1290+
1291+
**Layer:** Egeria Server (not pyegeria) · **Status:** open, not yet reported upstream · **Found:**
1292+
2026-10-05, live verification of `MetadataExpert.delete_enum_def` (PR #428). **Left a stray type behind.**
1293+
1294+
**What:** after `add_enum_def` (returned GUID `118441be-...`), `add_type_def` of an entity type using it, and a
1295+
successful `delete_type_def` of that entity type, `delete_enum_def(guid, name)` answered
1296+
`OMRS-CONTENT-MANAGER-500-001 The repository content manager method getAttributeTypeDef has detected an unknown
1297+
TypeDef 118441be-... from qs-metadata-store on behalf of method deleteAttributeTypeDef` ("Open up a Github issue").
1298+
Yet `ValidMetadataManager.get_attribute_types()` still lists the enum, so the server knows it by one route and
1299+
not the other.
1300+
1301+
**From Egeria's source:** `OMRSRepositoryContentManager.getAttributeTypeDef(sourceName, guid, methodName)`
1302+
throws `BAD_TYPEDEF` when the GUID is not in `knownAttributeTypeDefGUIDs`. Hypothesis, not traced: an enum added
1303+
at runtime is registered in the name-keyed map but not that GUID-keyed one, or deleting the type that uses it
1304+
drops the GUID entry.
1305+
1306+
**Left on the shared dev platform:** enum `PyegeriaTmpCuisineType` (`118441be-6e03-4442-96c6-e431f75fcb3f`),
1307+
unused, announced to the cohort. Harmless but visible in type listings. A retry after the platform's next restart
1308+
is worthwhile (the .http says API-defined types survive a restart when the repository is persistent, so the content
1309+
manager may then know it by GUID).
1310+
1311+
**Re-tested 2026-10-05 after the platform was rebuilt and restarted** (image built 14:19Z from egeria main
1312+
`449ad06894`, `egeria-main` restarted 14:39Z): the enum survived the restart and is still listed, and
1313+
`delete_enum_def` fails with the identical `OMRS-CONTENT-MANAGER-500-001`. So a restart does not repopulate the
1314+
GUID map; the persisted type is loaded without being registered under its GUID (or the delete looks in the wrong
1315+
map). The retry-after-restart idea above is closed.
1316+
1317+
**Workaround:** none through the API. Avoid creating throwaway enums on a shared platform.
1318+
1319+
---
1320+
12601321
### ISSUE-122: `AssetMaker.get_catalog_targets` / `get_catalog_target` send `metadataElementTypeName="CatalogTarget"` (a relationship type) — server rejects with OMAG-COMMON-400-019, surfaced as SERVER_ERROR_500
12611322

12621323
**Status: fixed on branch `fix/issue-122-catalog-target-type` (2026-10-04), pending PR/merge**

‎pyegeria/models/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,9 @@
1818
DeleteElementRequestBody,
1919
DeleteRelationshipRequestBody,
2020
NewSubscriptionTypeRequestBody,
21+
OpenMetadataEnumDef,
22+
OpenMetadataTypeDef,
23+
OpenMetadataTypeDefPatch,
2124
DeleteClassificationRequestBody,
2225
InitialClassifications,
2326
NewElementRequestBody,

‎pyegeria/models/models.py‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -940,3 +940,46 @@ class NewSubscriptionTypeRequestBody(RequestBody):
940940
service_level_objective_guid: str | None = None
941941
notification_interval: int | None = None # minutes
942942
monitored_resource_guids: list[str] | None = Field(None, alias="monitoredResourceGUIDs")
943+
944+
945+
# ---------------------------------------------------------------------------
946+
# Open metadata type definitions (MetadataExpert add/update/delete type defs).
947+
#
948+
# These mirror Egeria's OpenMetadataEnumDef / OpenMetadataTypeDef / OpenMetadataTypeDefPatch
949+
# beans, which are deep graphs (attribute definitions, relationship ends, enum elements,
950+
# links to other types by name). Only the fields the .http ground truth documents are
951+
# declared; extra="allow" lets everything else in the bean (relationshipAttributes, endDefs,
952+
# propagationRule, validEntityDefs, ...) pass through unchanged rather than being silently
953+
# dropped by PyegeriaModel's extra="ignore" -- see the CLAUDE.md gotcha on request models.
954+
# ---------------------------------------------------------------------------
955+
class OpenMetadataEnumDef(PyegeriaModel):
956+
model_config = ConfigDict(extra='allow')
957+
class_: Annotated[Literal["OpenMetadataEnumDef"], Field(alias="class")]
958+
name: str
959+
description: str | None = None
960+
element_defs: list[dict[str, Any]] | None = None
961+
default_value: dict[str, Any] | None = None
962+
963+
964+
class OpenMetadataTypeDef(PyegeriaModel):
965+
"""An entity, relationship or classification type definition (the bean's concrete subclass is
966+
chosen by `class`). Links to other types - supertype, attribute types, relationship ends -
967+
only need the name of the type."""
968+
model_config = ConfigDict(extra='allow')
969+
class_: Annotated[Literal["OpenMetadataEntityDef", "OpenMetadataRelationshipDef",
970+
"OpenMetadataClassificationDef"], Field(alias="class")]
971+
name: str
972+
description: str | None = None
973+
super_type: dict[str, Any] | None = None
974+
attribute_definitions: list[dict[str, Any]] | None = None
975+
976+
977+
class OpenMetadataTypeDefPatch(PyegeriaModel):
978+
"""Carries only what changes in a type definition that was added through the API."""
979+
model_config = ConfigDict(extra='allow')
980+
class_: Annotated[Literal["OpenMetadataTypeDefPatch"], Field(alias="class")]
981+
type_def_guid: str
982+
type_def_name: str
983+
apply_to_version: int
984+
description: str | None = None
985+
attribute_definitions: list[dict[str, Any]] | None = None

‎pyegeria/omvs/metadata_expert.py‎

Lines changed: 208 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,8 @@
1717
SearchStringRequestBody,
1818
FilterRequestBody, GetRequestBody, ResultsRequestBody,
1919
SearchStringRequestBody as SearchStringBody, DeleteElementRequestBody,
20-
DeleteRelationshipRequestBody)
20+
DeleteRelationshipRequestBody,
21+
OpenMetadataEnumDef, OpenMetadataTypeDef, OpenMetadataTypeDefPatch)
2122
from pyegeria.core.utils import body_slimmer, dynamic_catch
2223
from pyegeria.core._server_client import ServerClient, max_paging_size
2324
from pyegeria.core._globals import default_timeout, NO_ELEMENTS_FOUND
@@ -4375,3 +4376,209 @@ def get_property_comparison_operator_list(self) -> list:
43754376
"""Retrieve the list of valid property comparison operators."""
43764377
loop = asyncio.get_event_loop()
43774378
return loop.run_until_complete(self._async_get_property_comparison_operator_list())
4379+
4380+
#
4381+
# Maintain the open metadata types that are defined through the API. These are validated against the types
4382+
# already defined, stored in the metadata access store's local repository, and announced to the rest of the
4383+
# cohort. Types from the open metadata archives cannot be changed this way. (Egeria-api-metadata-expert.http)
4384+
#
4385+
4386+
async def _async_type_def_request(self, url: str, body: dict | Any, model: type,
4387+
params: dict | None = None) -> Any:
4388+
"""POST a type-definition body (validated through `model`, with undeclared bean fields passed
4389+
through) and return the response's "guid" if it has one, else the whole response payload."""
4390+
if isinstance(body, model):
4391+
validated = body
4392+
elif isinstance(body, dict):
4393+
validated = self._validate_body(model.model_validate, body)
4394+
else:
4395+
raise ValueError(f"body must be a dict or {model.__name__}")
4396+
response = await self._async_make_request(
4397+
"POST", url, validated.model_dump_json(indent=2, exclude_none=True), params=params)
4398+
payload = response.json()
4399+
return payload.get("guid", payload) if isinstance(payload, dict) else payload
4400+
4401+
@dynamic_catch
4402+
async def _async_add_enum_def(self, body: dict | OpenMetadataEnumDef) -> str | dict:
4403+
"""
4404+
Add a new enum definition, which can then be used as the type of attributes in new type definitions.
4405+
Async version.
4406+
4407+
.. warning::
4408+
An enum added through the API cannot be removed again: ``delete_enum_def`` currently fails against
4409+
Egeria with a 500 "unknown TypeDef" (an Egeria server bug, ISSUE-125, reproduced on Egeria main
4410+
449ad06894, and the enum survives a restart). Do not create throwaway enums on a shared platform.
4411+
4412+
Parameters
4413+
----------
4414+
body : dict | OpenMetadataEnumDef
4415+
The enum definition.
4416+
4417+
Returns
4418+
-------
4419+
str | dict
4420+
The unique identifier (GUID) of the new enum definition, or the raw response if it carries none.
4421+
4422+
Notes
4423+
-----
4424+
Sample JSON body:
4425+
{
4426+
"class" : "OpenMetadataEnumDef",
4427+
"name" : "CuisineType",
4428+
"description" : "The style of cooking that a recipe belongs to.",
4429+
"elementDefs" : [
4430+
{ "ordinal" : 0, "value" : "Unclassified", "description" : "The cuisine has not been recorded." },
4431+
{ "ordinal" : 99, "value" : "Other", "description" : "Another cuisine." }
4432+
],
4433+
"defaultValue" : { "ordinal" : 0, "value" : "Unclassified", "description" : "..." }
4434+
}
4435+
"""
4436+
url = f"{self.command_root}/open-metadata-attribute-types/enum-defs"
4437+
return await self._async_type_def_request(url, body, OpenMetadataEnumDef)
4438+
4439+
@dynamic_catch
4440+
def add_enum_def(self, body: dict | OpenMetadataEnumDef) -> str | dict:
4441+
"""Add a new enum definition, which can then be used as the type of attributes in new type definitions.
4442+
Note: it cannot be removed again through the API (ISSUE-125); see the async version's warning."""
4443+
loop = asyncio.get_event_loop()
4444+
return loop.run_until_complete(self._async_add_enum_def(body))
4445+
4446+
@dynamic_catch
4447+
async def _async_add_type_def(self, body: dict | OpenMetadataTypeDef) -> str | dict:
4448+
"""
4449+
Add a new type definition for an entity, relationship or classification. Links to other types - the
4450+
supertype, attribute types, relationship ends - only need the name of the type. Async version.
4451+
4452+
.. note::
4453+
Verified against a live server for entity types (with primitive and enum attributes). Relationship and
4454+
classification definitions go through the same code and models but have not been exercised live.
4455+
4456+
Parameters
4457+
----------
4458+
body : dict | OpenMetadataTypeDef
4459+
The type definition; `class` is OpenMetadataEntityDef, OpenMetadataRelationshipDef or
4460+
OpenMetadataClassificationDef.
4461+
4462+
Returns
4463+
-------
4464+
str | dict
4465+
The unique identifier (GUID) of the new type definition, or the raw response if it carries none.
4466+
4467+
Notes
4468+
-----
4469+
Sample JSON body:
4470+
{
4471+
"class" : "OpenMetadataEntityDef",
4472+
"name" : "Recipe",
4473+
"description" : "A description of how to prepare a dish.",
4474+
"superType" : { "name" : "Referenceable" },
4475+
"attributeDefinitions" : [
4476+
{ "attributeName" : "cuisine",
4477+
"attributeType" : { "class" : "OpenMetadataEnumDef", "name" : "CuisineType" },
4478+
"attributeDescription" : "The style of cooking." },
4479+
{ "attributeName" : "servings",
4480+
"attributeType" : { "class" : "OpenMetadataPrimitiveDef", "name" : "int" },
4481+
"attributeDescription" : "The number of people the recipe serves." }
4482+
]
4483+
}
4484+
"""
4485+
url = f"{self.command_root}/open-metadata-types"
4486+
return await self._async_type_def_request(url, body, OpenMetadataTypeDef)
4487+
4488+
@dynamic_catch
4489+
def add_type_def(self, body: dict | OpenMetadataTypeDef) -> str | dict:
4490+
"""Add a new type definition for an entity, relationship or classification."""
4491+
loop = asyncio.get_event_loop()
4492+
return loop.run_until_complete(self._async_add_type_def(body))
4493+
4494+
@dynamic_catch
4495+
async def _async_update_type_def(self, body: dict | OpenMetadataTypeDefPatch) -> None:
4496+
"""
4497+
Update a type definition that was added through the API. The patch names the version it applies to
4498+
and carries only what changes. Async version.
4499+
4500+
.. warning::
4501+
Against Egeria as of 2026-10-05 (including its main branch) every patch is rejected by the server
4502+
with OMRS-REPOSITORY-400-069, "mandatory field updatedBy set to null": the server builds the patch
4503+
with a converter that is never given the calling user. Nothing a client sends can supply it. See
4504+
PYEGERIA_ISSUES.md, ISSUE-124. This method matches the documented API and will work once that
4505+
is fixed upstream.
4506+
4507+
Parameters
4508+
----------
4509+
body : dict | OpenMetadataTypeDefPatch
4510+
The patch.
4511+
4512+
Notes
4513+
-----
4514+
Sample JSON body:
4515+
{
4516+
"class" : "OpenMetadataTypeDefPatch",
4517+
"typeDefGUID" : "add guid here",
4518+
"typeDefName" : "Recipe",
4519+
"applyToVersion" : 1,
4520+
"attributeDefinitions" : [
4521+
{ "attributeName" : "preparationTimeMinutes",
4522+
"attributeType" : { "class" : "OpenMetadataPrimitiveDef", "name" : "int" },
4523+
"attributeDescription" : "How long the recipe takes to prepare." }
4524+
]
4525+
}
4526+
"""
4527+
url = f"{self.command_root}/open-metadata-types/update"
4528+
await self._async_type_def_request(url, body, OpenMetadataTypeDefPatch)
4529+
4530+
@dynamic_catch
4531+
def update_type_def(self, body: dict | OpenMetadataTypeDefPatch) -> None:
4532+
"""Update a type definition that was added through the API."""
4533+
loop = asyncio.get_event_loop()
4534+
return loop.run_until_complete(self._async_update_type_def(body))
4535+
4536+
@dynamic_catch
4537+
async def _async_delete_type_def(self, type_def_guid: str, type_def_name: str) -> None:
4538+
"""
4539+
Delete a type definition that was added through the API. This fails while there are any instances of
4540+
the type, including soft-deleted ones, or while another type definition refers to it. Async version.
4541+
4542+
Parameters
4543+
----------
4544+
type_def_guid : str
4545+
Unique identifier of the type definition.
4546+
type_def_name : str
4547+
Name of the type definition (the server checks it against the GUID).
4548+
"""
4549+
url = f"{self.command_root}/open-metadata-types/guid/{type_def_guid}/delete"
4550+
await self._async_make_request("POST", url, params={"typeDefName": type_def_name})
4551+
4552+
@dynamic_catch
4553+
def delete_type_def(self, type_def_guid: str, type_def_name: str) -> None:
4554+
"""Delete a type definition that was added through the API."""
4555+
loop = asyncio.get_event_loop()
4556+
return loop.run_until_complete(self._async_delete_type_def(type_def_guid, type_def_name))
4557+
4558+
@dynamic_catch
4559+
async def _async_delete_enum_def(self, enum_def_guid: str, enum_def_name: str) -> None:
4560+
"""
4561+
Delete an enum definition that was added through the API. This fails while any type definition has an
4562+
attribute of this type. Async version.
4563+
4564+
.. warning::
4565+
Observed live (2026-10-05) to fail with a 500, OMRS-CONTENT-MANAGER-500-001 "unknown TypeDef", even
4566+
after the only type using the enum had been deleted and while the enum was still listed by
4567+
``get_attribute_types``: an enum added through the API could not be removed again. Delete enum
4568+
definitions only if you can live with that. See PYEGERIA_ISSUES.md, ISSUE-125.
4569+
4570+
Parameters
4571+
----------
4572+
enum_def_guid : str
4573+
Unique identifier of the enum definition.
4574+
enum_def_name : str
4575+
Name of the enum definition (the server checks it against the GUID).
4576+
"""
4577+
url = f"{self.command_root}/open-metadata-attribute-types/enum-defs/guid/{enum_def_guid}/delete"
4578+
await self._async_make_request("POST", url, params={"enumDefName": enum_def_name})
4579+
4580+
@dynamic_catch
4581+
def delete_enum_def(self, enum_def_guid: str, enum_def_name: str) -> None:
4582+
"""Delete an enum definition that was added through the API."""
4583+
loop = asyncio.get_event_loop()
4584+
return loop.run_until_complete(self._async_delete_enum_def(enum_def_guid, enum_def_name))

0 commit comments

Comments
 (0)