Value filtering on non-key objects

[ericvoit]

Do we allow selection filters for on-change subscriptions to filter non-key objects based on a subset of values of these non-key objects? Doing this might confuse receivers as add & delete yang-patch operations will need to add and remove objects (and not just their changed properties) simply when a property changes.

[ericvoit]

Email sent to NETCONF Alias on Dec 22nd...
During Martin's recent review of yang-push, he argued that we should allow selection filtering using non-key properties. I believe he is correct. (In fact, this is what was supported in yang-push drafts -v00 through -v08.)

The reason we changed in -v09 is that if a non-key property changes, it might mean that the object as a whole is no longer selected. The result is that an entire subtree may no longer should be visible to the receiver. So the YANG patch sent from the subscriber to the receiver would need to add/remove entire subtree based on a simple value change for a non-key object in that subtree. The thought for -v09 was that if we restricted on-change selection filtering just to keys, this might be simpler/easier to implement. But per Martin's comments below, he pointed out other issues which come with placing such restrictions in filters.

Does anyone have an objections to once again allowing selection filters to be placed on non-key properties? (Alex & Andy, this was something we went back-and-forth on as part of the Dezign Team discussions.)

If there is no objection, this makes the conceptual selection process for creating an push-change-update:

(1) Just before a change, or at the start of a dampening period, evaluate any filtering and any access control rules. The result is a set "A" of datastore nodes and subtrees.

(2) Just after a change, or at the end of a dampening period, evaluate any filtering and any (possibly new) access control rules. The result is a set "B" of datastore nodes and subtrees.

(3) Construct a YANG patch record for going from A to B. If the record is non-empty, send it to the receiver.

---

I think it is time to close this issue, and allow such filtering

[abierman]

Hi, I do not object to non-keys in filters. In fact, I don't want the standards to constrain implementations. The subscription-hints should be able to handle the details of what filters are supported (although this is a hard problem to generalize) Andy

…

On Mon, Feb 5, 2018 at 6:39 PM, ericvoit ***@***.***> wrote: Email sent to NETCONF Alias on Dec 22nd... During Martin's recent review of yang-push, he argued that we should allow selection filtering using non-key properties. I believe he is correct. (In fact, this is what was supported in yang-push drafts -v00 through -v08.) The reason we changed in -v09 is that if a non-key property changes, it might mean that the object as a whole is no longer selected. The result is that an entire subtree may no longer should be visible to the receiver. So the YANG patch sent from the subscriber to the receiver would need to add/remove entire subtree based on a simple value change for a non-key object in that subtree. The thought for -v09 was that if we restricted on-change selection filtering just to keys, this might be simpler/easier to implement. But per Martin's comments below, he pointed out other issues which come with placing such restrictions in filters. Does anyone have an objections to once again allowing selection filters to be placed on non-key properties? (Alex & Andy, this was something we went back-and-forth on as part of the Dezign Team discussions.) If there is no objection, this makes the conceptual selection process for creating an push-change-update: (1) Just before a change, or at the start of a dampening period, evaluate any filtering and any access control rules. The result is a set "A" of datastore nodes and subtrees. (2) Just after a change, or at the end of a dampening period, evaluate any filtering and any (possibly new) access control rules. The result is a set "B" of datastore nodes and subtrees. (3) Construct a YANG patch record for going from A to B. If the record is non-empty, send it to the receiver. ------------------------------ I think it is time to close this issue, and allow such filtering — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#12 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABugjzWtIw0R_NmldS7_vvu1b1BpT_dQks5tR7tsgaJpZM4Q6W-Y> .

[ericvoit]

Sounds like we are all in agreement on this then. From: Andy Bierman [mailto:notifications@github.com] Sent: Tuesday, February 6, 2018 2:26 PM To: netconf-wg/yang-push <yang-push@noreply.github.com> Cc: Eric Voit (evoit) <evoit@cisco.com>; Author <author@noreply.github.com> Subject: Re: [netconf-wg/yang-push] Value filtering on non-key objects (#12) Hi, I do not object to non-keys in filters. In fact, I don't want the standards to constrain implementations. The subscription-hints should be able to handle the details of what filters are supported (although this is a hard problem to generalize) Andy

On Mon, Feb 5, 2018 at 6:39 PM, ericvoit ***@***.******@***.***>> wrote: Email sent to NETCONF Alias on Dec 22nd... During Martin's recent review of yang-push, he argued that we should allow selection filtering using non-key properties. I believe he is correct. (In fact, this is what was supported in yang-push drafts -v00 through -v08.) The reason we changed in -v09 is that if a non-key property changes, it might mean that the object as a whole is no longer selected. The result is that an entire subtree may no longer should be visible to the receiver. So the YANG patch sent from the subscriber to the receiver would need to add/remove entire subtree based on a simple value change for a non-key object in that subtree. The thought for -v09 was that if we restricted on-change selection filtering just to keys, this might be simpler/easier to implement. But per Martin's comments below, he pointed out other issues which come with placing such restrictions in filters. Does anyone have an objections to once again allowing selection filters to be placed on non-key properties? (Alex & Andy, this was something we went back-and-forth on as part of the Dezign Team discussions.) If there is no objection, this makes the conceptual selection process for creating an push-change-update: (1) Just before a change, or at the start of a dampening period, evaluate any filtering and any access control rules. The result is a set "A" of datastore nodes and subtrees. (2) Just after a change, or at the end of a dampening period, evaluate any filtering and any (possibly new) access control rules. The result is a set "B" of datastore nodes and subtrees. (3) Construct a YANG patch record for going from A to B. If the record is non-empty, send it to the receiver. ------------------------------ I think it is time to close this issue, and allow such filtering — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#12 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABugjzWtIw0R_NmldS7_vvu1b1BpT_dQks5tR7tsgaJpZM4Q6W-Y> .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#12 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AN9voBrALbC-3Rwol0Elt7EXjY7GTtD6ks5tSKdKgaJpZM4Q6W-Y>.

[ericvoit]

In a recent review, Tim Jenkins privately pointed out a bug in the process above. This bug has been fixed in the current -v15. Basically the process bug resulted from this thread last year when I forgot to document a step which has long been included in the draft text. Specifically the text is from Section 3.3 On-change considerations: "If an object has returned to its original value (or even has been created and then deleted) during the dampening-period, the last change will still be sent. This will indicate churn is occurring on that object."

To fix this bug the following text has been inserted as an independent step (4)...

(1) Just before a change, or at the start of a dampening period, evaluate any
filtering and any access control rules. The result is a set "A" of datastore nodes
and subtrees.

(2) Just after a change, or at the end of a dampening period, evaluate any
filtering and any (possibly new) access control rules. The result is a set "B" of
datastore nodes and subtrees.

(3) Construct a YANG patch record for going from A to B.

(4) If there were multiple changes made to a subscribed object between "A" and "B" which fully canceled each other out, then if access control permits read access, insert into the YANG patch record the last change made to that object.

(5) If the resulting patch record is non-empty, send it to the receiver.

Final note, while writing this email update, I did a text tweak from step (4) from what is in -v15. But the intent of the text is the same. And the process now returns to matching earlier versions of the draft, and historical documentation provided such as slides 29 & 30 from IETF96.
https://datatracker.ietf.org/doc/slides-96-netconf-5/1/

Eric