Define predicate as external function... but how?

[khanhhua]

In the readme we have 2 examples about rules and predicates (and grounded facts), namely contains_jar_direct and direct_conn. As far as I understand, these are grounded facts to be imported or referenced at runtime (like a connection to Postgres, or a Rest API call).

Could you direct me to concrete examples of how it is to be done? I can work directly with C, Go. Thank you.

[burakemir]

Proper external predicates called during evaluation are not supported yet (they will be). UPDATE: There is an external predicates API now ea0330c

There are a few ways / patterns / conventions which are often good enough in the meantime. I should document these recipes better.

Let me know if these examples help, or whether you really prefer runtime external predicates.

1. offline, full, ahead-of-time

The simplest is to retrieve the data and then populate the factstore with facts, before doing evaluation. Ths README was written with such a setup in mind, where we can retrieve all the data upfront. Example code that populates a factstore from a file. You could also just add the facts to the fact store one by one. When the data is changing, one can periodically repeat this, replacing factstore, redoing evaluation.

2. interactive, selective

For an interactive use case (you get a Mangle program which needs external data and need to build a database to compute the result), there are atoms that are fully determined external queries whose results fits in memory:

• parse the program/query, identify atoms that refer to "rpc predicates".
• run those rpcs, add results to fact store.
• afterwards, perform regular evaluation, which runs the program and query.

For type-checking, you want a declaration like this that defines a "mode". A "+" mode indicates input, a "-" mode indicates output.

To remember which is which for the modes, think of shooting an arrow: for input, you shoot the arrow and see the read-end, for output, the arrow-head is pointing at you.

Decl gimme_xyz(Key1, Key2, Result1, Result2)
  descr[
    rpc(),
    mode("+", "+", "-", "-"),
    doc("Returns all Result1,Result2 pairs for query Key1,Key2"),
    arg(Key1,
        "string indentifying environment, in frombulizer format"),
    arg(Key2,
        "the jwt token for auth"),
    arg(Result1,
        "corresponding foobar result first component")
    arg(Result2,
        "corresponding foobar result second component")
  ]
  bound[/string, /string, /string, .List</string>].

3. offline, selective, dynamic in phases

If database computed ahead of time, the full external data would be too much, the relevant data fits in-memory and your program needs specific subset but you only know what in the middle of evaluation, one can do something like this:

• split your program to work in two "phases":
  • the first "phase" of your program computes what is needed, producing facts external_xyz_query(Key1, Key2)
  • the second "phase" then works with external_xyz facts as if they are already there and extensional.
• in the go code that links the Mangle lib, you run evaluation twice:
  • the first evaluation. This produces external_xyz_query facts
  • the go code retrieves all external_xyz_query facts from the fact store, performs the external query, adds external_xyz facts to the fact store
  • then we run evaluation again, this time the rules can use external_xyz facts.

This can even be done in a loop, with caching, until no more facts are added.

If the data is huge, one can save the factstore to a file (or cloud service) between phases. This all needs dev work, but overcomes in-memory limitations.

[burakemir]

Update: there is an external predicates API now ea0330c

[burakemir]

I filed issue #59 for proper external predicates.

[kamahen]

Datalog is a subset of Prolog and SWI-Prolog has an extensive API in C, C++, Python that might give you some ideas (I re-worked the C++ API).

https://www.swi-prolog.org/pldoc/man?section=calling-prolog-from-c
https://www.swi-prolog.org/pldoc/doc_for?object=section(%27packages/pl2cpp.html%27)
https://www.swi-prolog.org/pldoc/man?section=janus-call-prolog
https://www.swi-prolog.org/FAQ/Python.md