I have been working on a site-local Shotgun cache in order to (1) eliminate the round-trip time to Shotgun's servers, and (2) allow for more expressive queries. This cache primarily operates as an API proxy; it satisfies any requests it can, but if it is missing anything it will forward the request to the real server, and cache the results for next time.

Schema changes are a major concern of mine. Instead of trying to perfectly anticipate how Shotgun will internally mutate data during a schema change, I have opted to simply discard any data affected by such a change.

This still requires me to be able to watch for these changes. My first thought was to watch the event log (by polling for new EventLogEntry entities), and this does show us a schema change, e.g. here is the entry for changing a field's type from "text" to "number":

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

{
    'type': 'EventLogEntry',
    'id': 1996928,
    'event_type': 'Shotgun_DisplayColumn_Change',
    'attribute_name': 'data_type',
    'entity': {'id': 2836, 'name': 'sg_text_field_1', 'type': 'DisplayColumn'},
    'meta': {
        'attribute_name': 'data_type',
        'entity_id': 2836,
        'entity_type': 'DisplayColumn',
        'field_data_type': 'text',
        'new_value': 'number',
        'old_value': 'text',
        'type': 'attribute_change'
    },
    'user': {'id': 108, 'name': 'Mike Boers', 'type': 'HumanUser'}
}

The problem is that this doesn't mention anywhere what entity type this field is on! Where is CustomEntitity02 in all of this?

"It's okay, Mike", you might say. "You can just query the DisplayColumn entity!". Lets try:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

>>> sg.find_one('DisplayColumn', [('id', 'is', 2836)])
Traceback (most recent call last):
...
shotgun_api3.shotgun.Fault: API read() invalid/missing string entity 'type':
Valid entity types:
["ActionMenuItem",
 "ApiUser",
 "AppWelcomeUserConnection",
 "Asset",
...

It turns out that DisplayColumn, along with PageSetting (and potentially more entity types) are considered private, and cannot be queried via the standard API.

So... how can we get that data?

Read more... (2 minutes remaining to read.)