PSOATransRun Development Agenda
This agenda complements the projects in PSOATransRun Development Tutorial, particularly Projects 1 and 4, as well as material in PSOA RuleML#Implementation. Most agenda items require knowledge of Java, several of ANTLR, some of Python. If you want to contribute to the open development of one of the below agenda items or to a new one, please contact Harold Boley.
- Source: https://github.com/RuleML/PSOATransRunComponents/releases
- Executable (with README): http://psoa.ruleml.org/transrun/
- 1 Documentation
- 2 Functional Programming
- 3 Expressivity/Efficency
- 3.1 Allow All Permutations of Descriptors in Psoa Atoms
- 3.2 Make KB Input via Command Line as Expressive as via Import Statement
- 3.3 omitNegMem as a Possible Default Option
- 3.4 Add Generic Inequality
- 3.5 Add Negation-as-failure
- 3.6 Extend towards Reaction RuleML
- 3.7 Add support for membership declaration in the "Forall" wrappers
- 4 Usability Increase
- 4.1 Display Instantiation and Version
- 4.2 Refine TPTP Instantiation
- 4.3 Enable Multi-Line Query Input
- 4.4 Display Blank-Separated Bindings in Interaction Loop
- 4.5 Allow Tracing
- 4.6 Refine Error Messages
- 4.7 Revisit Encodings and Reserved Names
- 4.8 Schema Querying
- 4.9 Preserve Comments in the Translated KB
- 5 Utility Programs
- 6 Upgrading PSOA RuleML Files
- 7 Test Cases
- 8 Technology-Stack Maintenance
1.1 PSOATransRunComponents README
Consider quick setup hints.
Mention used version of ANTLR as the Java-integrated parser generator etc. (cf. #Technology-Stack_Maintenance).
1.2 Best Practices Page
- When editing a PSOA RuleML file:
- Use a pretty-print format, where, e.g., each clause is indented by the same amount of whitespace (maybe none)
- Put clauses (rules and facts) with variables into Forall ... (. . .) wrappers
- Referring to #Unit_Tests, within new batch testing directories <testDir> such as testsynsem (for java -jar target\PSOATransRunLocal.jar --test -i <testDir>), in files of the form *-answerJ.psoa (do not use test's yes/no but) use Capitalized Yes/No (as displayed in the interaction loop)
2 Functional Programming
2.1 Refine Functional Expressions
This could become a project of its own:
- As usual, n-ary lists (RuleML/XML: slotless plexes, PSOA RuleML: single-tuple psoa expressions) can be realized as ('passive') binary cons (Relfun: cns) expressions, which ultimately could be given an n-ary surface syntax, also allowing rest parameters (RuleML/XML: repo, Prolog and Relfun: "|") as detailed in #Introduce_Tuple_Rests
- Flattening of ('passive' and 'active') expressions could be modeled on, and possibly combined with, existing unnesting of atoms
- (First-order) Functional Programming (with higher-order syntactic sugar) would then be possible as in Hilog and Relfun
2.2 Introduce Tuple Rests
John#Student(Mon Tue Fri)
John#Student(+[Mon Tue Fri])
John#Student(?d1 ?d2 ?d3)
John#Student(+[?d1 ?d2 ?d3])
John#Student(?d1 ?d2) John#Student(?d1 ?d2 ?d3 ?d4)
When 'unsigned' tuples (currently NOT used), tuple variables, and tuple rests (with Prolog/Relfun-like "|") are introduced,
tuple-variable (?R) queries
John#Student(| ?R) John#Student(?d1 | ?R) John#Student(?d1 ?d2 | ?R) John#Student(?d1 ?d2 ?d3 | ?R)
John#Student(+[| ?R]) John#Student(+[?d1 | ?R]) John#Student(+[?d1 ?d2 | ?R]) John#Student(+[?d1 ?d2 ?d3 | ?R])
would succeed with
?R=[_Mon _Tue _Fri] ?d1=_Mon,?R=[_Tue _Fri] ?d1=_Mon,?d2=_Tue,?R=[_Fri] ?d1=_Mon,?d2=_Tue,?d3=_Fri,?R=
For a psoa atom with multiple tuples, each would be allowed to have its own tuple rest.
Also see PSOA RuleML: Integrated Object-Relational Data and Rules, Section 6.
2.3 Alternative to External
[Postponed] Consider realizing a PSOA RuleML 1.0 change, replacing External with a more concise syntax or making External optional. Study consequences for http://wiki.ruleml.org/index.php/PSOA_RuleML#Monolithic_Syntax, http://wiki.ruleml.org/index.php/PSOA_RuleML#Semantics, and http://wiki.ruleml.org/index.php/PSOA_RuleML#Test_Cases. Optionality would allow omission of External wrappers by not only shortcutting external built-in pred calls like
> External(pred:numeric-equal(42 42)) Answer(s): Yes
> pred:numeric-equal(42 42) Answer(s): Yes
but also external built-in func calls like in
> 42 = External(func:numeric-add(26 16)) Answer(s): Yes
> 42 = func:numeric-add(26 16) Answer(s): Yes
[instead of current
> ?res = func:numeric-add(26 16) Answer(s): ?res=+(26 16)
Like in Prolog's "is" and unlike in External-using PSOA or "(...)"-vs.-"[...]"-using Relfun, 'active' funcs would not be 'applied differently' from 'passive' funcs but would need to be recognized from being in the context of a "=" rhs or an arg or slot-filler position.
For funcs, External thus currently not only marks a built-in but also indicates an 'active' call. Relying on pred/func prefixes as a replacement for External would burden prefixes with this extra meaning in contrast to the more simple ('user-package') meaning of other prefixes.
3.1 Allow All Permutations of Descriptors in Psoa Atoms
Right-slot normal form (PSOATransRun 1.3):
_John#_Student(+[_Mon _Tue _Fri] -[1995 8 17] _coursehours+>20 _dept+>_Math _gender->_male) _John#_Student(-[1995 8 17] +[_Mon _Tue _Fri] _gender->_male _coursehours+>20 _dept+>_Math)
Left-slot normal form (TBD):
_John#_Student(_coursehours+>20 _dept+>_Math _gender->_male +[_Mon _Tue _Fri] -[1995 8 17])
Free form (TBD):
_John#_Student(_gender->_male -[1995 8 17] +[_Mon _Tue _Fri] _dept+>_Math _coursehours+>20)
3.2 Make KB Input via Command Line as Expressive as via Import Statement
Currently, command-line input (-i,--input) refers to a KB in a local file but cannot refer to a KB in a global URL. An Import statement can be employed within an empty-KB ("include"-style) document to refer to a URL. But it would be better to refer to that URL directly from the command line.
3.3 omitNegMem as a Possible Default Option
Consider making memterm omission the default, thus changing this (in LP, "negative literals" refer to atoms in rule conditions and in queries):
-z,--omitNegMem Omit memterm in the slotribution of negative occurrences of psoa atoms with at least one dependent descriptor
However, for target engines such as Prolog that process conjunctions (as resulting from slotribution) left-to-right, it might be better for efficiency to keep (ground) memterms, in left-most positions, causing (type/class/predicate) failures as early as possible rather than spending unnecessary computation before encountering such failures further into conjunctions. Hence the current default, with possible "semantic query optimization" in the form of conjunction reordering, could be kept.
3.4 Add Generic Inequality
Generalizing the number inequality built-in, a possible string inequality built-in, etc., a generic predefined inequality should be added:
- mainly as another built-in via a prefix predicate
- perhaps as a primitive via an infix operator (~=)
Similarly to other PSOA built-ins and the Prolog is primitive as well as PSOA's generic equality (=) infix operator it should be able to evaluate expressions (at least) in its second argument or right-hand side, but also allow the use of non-numeric, non-string, etc. arguments, negating the result of their equality comparison. The (initial) built-in version should be restricted to testing (in conditions) its arguments for inequality, rather than a (later) primitive version that also could generate (in conclusions) pairs of unequal terms.
3.5 Add Negation-as-failure
Since Negation-as-failure (Naf) is not affected by the kind of (generalized psoa) atoms in its scope, in PSOATransRun[PSOA2Prolog,...] instantiations it can be orthogonally added by augmenting the Prolog converter to translate Naf to Prolog's Not.
3.6 Extend towards Reaction RuleML
Make Pure Production Rules available in an extended PSOATransRun instantiated for targeting Prova (PSOA Prova).
3.7 Add support for membership declaration in the "Forall" wrappers
This Many-sorted logic feature will allow to limit the domain of discourse of the variables-quantifiers. It will be especially helpful in cases of "free" variables which result in singleton variables and unwanted bindings. Furthermore, it improves the readability of the code, e.g instead of:
Forall ?x ?y ( p1(s1->?x s2->?y) :- And(p2(?y v2) ?x#p3) )
where ?x#p3 limits the possible bindings of the variable ?x, it will be possible to have the equivalent code:
Forall ?x#p3 ?y ( p1(s1->?x s2->?y) :- p2(?y v2) )
4 Usability Increase
4.1 Display Instantiation and Version
- [Done] On startup, unconditionally display (command-line) instantiation variety and (released) version number of PSOATransRun such as PSOATransRun1.3[PSOA2Prolog,XSBProlog] (vs. PSOATransRun1.3[PSOA2TPTP,VampirePrime]) followed by PSOATransRun1.4[PSOA2Prolog,XSBProlog] (vs., e.g., PSOATransRun1.4[PSOA2Prolog,SWIProlog] and PSOATransRun1.4[PSOA2TPTP,VampirePrime])) and then in smaller increments like PSOATransRun1.4.1[...,...]
- Also see:
- println("PSOATransRun 1.3"); // TODO: Define method in PSOATransRun class, called here to return version
- Also see:
- Instead of hard-coding version into a print statement, use global variable(s) or constant(s), which later could also again be printed in interactive sessions (cf. Project 2)
4.2 Refine TPTP Instantiation
- Explore running on Linux (very easy, currently via VirtualBox), macOS (very probable), Windows (very unlikely, since VampirePrime-needed C library is missing)
- Expand the --test option examples that can be run using --targetLang tptp rather than --targetLang prolog (already all except [External-wrapped] built-ins)
- Consider reactivating Web Hosting on Openshift
4.3 Enable Multi-Line Query Input
[Indefinitely postponed] Currently, multi-line interactive query input is not allowed because the first end-of-line (via the ENTER key or the clipboard) immediately starts parsing. Since consoles cannot distinguish end-of-line from ENTER, a clean solution needs to bypass the console and directly handle keyboard events from end users in Java, like GUI-based programs such as Skype do, which can distinguish ENTER from SHIFT-ENTER and allows clipboard pasting of multi-line input, where 'inner ENTERs' are passivated.
4.4 Display Blank-Separated Bindings in Interaction Loop
[Done] In the interaction loop, consider displaying a substitution (a binding for each query variable) not in a ","-separated but in a (more light-weight) " "-separated manner, e.g. for query
display the substitution
?what=_XMLBible ?situation=_oidcons(_own _Mary _XMLBible)
which is also the format for *-answerJ.psoa files. Besides interaction/batch uniformity, this permits copy&paste from the interaction loop to a corresponding *-answerJ.psoa file. Moreover, in Prolog the comma already has two other meanings.
4.5 Allow Tracing
While the --echoInput (-e) option shows the translated KB and query, it would sometimes be helpful to see a computation trace on the same level. For the Prolog instantiation, this could just be a Prolog trace under the PSOATransRun system (a trace can already be viewed by invoking a separate (traced) XSB Prolog engine with translation results). For the TPTP instantiation, this would likely be more involved because of the complete search performed by TPTP engines (such as VampirePrime).
4.6 Refine Error Messages
Make errors (and warnings) display more uniformly, perhaps all using the same prefix, e.g. "Error: " (and "Warning: "). Consider possible connection to option --denseErrorMsgs. Some msgs can be easily refined. Others (from multiple thrown exceptions received by a single catch) are not so easy.
Explain error texts like "node from line 0:0" or make them more comprehensive.
Consider (context-free, -sensitive, and semantic) validation, in connection with XML serialization (cf. PSOA_RuleML#Specification and PSOA RuleML API), e.g. for the presence and declaration/use match of Forall ... (. . .) wrappers (cf. #Best_Practices_Page).
4.7 Revisit Encodings and Reserved Names
Addition to Project 4 on fixing issue with (<- Interprolog <- XSB Prolog) back-translation of constant encodings by single-quote surroundings (for '_'-prefixed PSOA constants and '"'-surrounded PSOA strings in XSB Prolog).
4.8 Schema Querying
PSOATransRun currently supports only instance(-level), not schema(-level), querying. A subclass (##) fact is translated to a rule by PSOATransRun, which cannot be queried as a whole and requires at least one 'witness' instance to answer #-queries properly. Schema(-level) querying to directly retrieve ##-facts could be included in PSOATransRun.
4.9 Preserve Comments in the Translated KB
Currently, comments are ignored by the parser, while the translated KB has no empty lines between blocks of code corresponding to a rule of the original KB. One possible solution is to preserve original comments (and possibly empty lines) in the translated KB, thus improving the readability of the generated Prolog/TPTP code. This will require changes to the ANTLR lexer, e.g. send COMMENT tokens to a dedicated channel. Then, they can be retrieved from e.g. CommonTokenStream which contains all the tokens, including the tokens of the dedicated channel. An important note is that it may be necessary to discard non whole-line comments.
Handling multi-line comments or empty lines would be also desired. One possible option for multi-line comment preservation is to also support the Prolog/TPTP-compatible block-comment based (/*...*/) comments, as their handling would be easier to implement.
5 Utility Programs
5.1 Refine Unit Test Program
For background see PSOA RuleML Unit Tests, which sketches Systematics, Vocabulary, and more.
Consider refining the (Java) TestSuite program that runs X_Y_Z[I]-queryJ.psoa with expected X_Y_Z[I]-answerJ.psoa for X_Y_Z[I]-KB.psoa in subdirectory X_Y_Z[I] of the directory test, comparing each actual answer with the expected answerJ. Possibly generalize the program such that it can be used for structured directories like, for Specification of Deliberation RuleML 1.03#Testing_of_Relax_NG_Schemas, the 'test cases' in rnc-test-suites (also see the 'show cases' in exa), where the current functionality is applied to the innermost directories (of the above form), whose existing (e.g., *.ruleml, *.pl, and *.grailog) files, i.e. files other than *-queryJ.psoa, *-answerJ.psoa, and *-KB.psoa are ignored. This will help creating and maintaining #Test_Cases.
Besides having a test-collection directory containing test-case directories, as in the command (with the main test collection, which is simply called test)
--test -i ...\test
we should also allow an individual test-case directory, as in the command (with a test-case directory indivdir1)
--test -i ...\indivdir1
or in the command (assuming the test collection test contains a test-case directory indivdir2)
--test -i ...\test\indivdir2
A mid-term TestSuite-like program could relax the current file name conventions, so it could be applied to arbitrary *.psoa KBs: The "-KB" suffix for file names X_Y_Z[I]-KB.psoa would no longer be needed.
For a long-term TestSuite-like program, after (drawing on RuleML/XML) not only a <Query>-markup-corresponding syntax but also an <Answer>-markup-corresponding syntax is introduced for a "self-describing" presentation syntax inside files, the file names X_Y_Z[I]-queryJ.psoa and X_Y_Z[I]-answerJ.psoa would no longer be essential, but a (Reaction-RuleML-send-like) general messaging performative could be used. But then the queryJ/answerJ pairing would need to be done in another way, e.g. by combining queryJ/answerJ files into one messageJ file (disadvantage: sometime queryJ/answerJ could become useful in a decoupled way), or by pointing from the queryJ file to the answerJ file.
- Also see:
- File testCases = dir.listFiles(); // TODO: listFiles could be generalized to structured directories.
5.2 Refine Benchmark Test Program
Consider refining the (Python) program, for chain and its variants, that (1) creates an enumeration of PSOA KBs of increasing size for a single query with an obvious answer and (2) provides the execution environment for probing PSOATransRun response times over them.
6 Upgrading PSOA RuleML Files
6.1 RuleML/Assert Use
Document/Group --> RuleML/Assert
6.2 Empty KB Allowance
Because some users don't want any Import and some may want to work off-line (rather than connecting to http://psoa.ruleml.org), the following should not be the default but the file EmptyKB.psoa should be provided in the PSOATransRun distribution for manual command-line use.
Consider to allow starting without a KB option/parameter, to be automatically initialized as the empty KB (omitted Assert), EmptyKB.psoa (instead of error
C:\Users\...>java -jar PSOATransRunLocal.jar No input KB specified
RuleML ( % Empty KB but allowing built-ins and libraries Prefix(pred: <http://www.w3.org/2007/rif-builtin-predicate#>) Prefix(func: <http://www.w3.org/2007/rif-builtin-function#>) Prefix(math: <http://psoa.ruleml.org/lib/math#>) Prefix(phys: <http://psoa.ruleml.org/lib/phys#>) Import(<http://psoa.ruleml.org/lib/math.psoa>) Import(<http://psoa.ruleml.org/lib/phys.psoa>) )
7 Test Cases
7.1 Unit Tests
- The basis are the Ref. 18 unit tests: https://github.com/RuleML/PSOATransRunComponents/tree/master/PSOATransRun/test (not to be extended)
- Add a few extensible (Semantic Profile-insensitive) unit test directories, each grouping feature by feature (small bits of functionality at a time), corner cases, etc., possibly uncovering issues: https://github.com/RuleML/PSOATransRunComponents/tree/master/PSOATransRun/testsynsem (also see the other .../test* directories in https://github.com/RuleML/PSOATransRunComponents/tree/master/PSOATransRun)
- Commandline-option- (Semantic Profile-)sensitive unit tests
- staticOnly vs. Absent
- staticOnly vs. Absent
- Testing the TestSuite System: test-meta
7.2 Benchmark Tests
7.3 Deliberation/PSOA RuleML Intersection Tests
Create tests, in PSOA Presentation Syntax, where PSOA RuleML/PS already semantically intersects with Deliberation RuleML/XML (later: PS generated from XML with PSOA RuleML API):
- (Purely) Relational KBs (e.g. for Project 3's case study "A Rule-Based Approach for Air Traffic Control ...")
- [Done] own.ruleml: https://github.com/RuleML/PSOATransRunComponents/blob/master/PSOATransRun/test/own/own-KB.psoa
- [Done] discount_compact.ruleml: https://github.com/RuleML/PSOATransRunComponents/blob/master/PSOATransRun/test/discount/discount-KB.psoa
- [Done] LoanProcessor.ruleml: http://psoa.ruleml.org/usecases/LoanProcessor/LoanProcessor.psoa
- Cases where engines (e.g., PSOATransRun) for PSOA RuleML's slotribution produce answers equivalent to those of engines (e.g., OO jDREW) for Deliberation RuleML's slotted unification
7.4 Deliberation/PSOA RuleML Union Tests
[Postponed] Similarly, add tests preparing (semantically) the union of Deliberation and PSOA features, e.g. multiple tuples with positional rests. Maybe use Prolog lists with "|" for PSOA argument sequences, but this would degrade performance.
8 Technology-Stack Maintenance
Also requires documentation in support of maintenance.
Deliberately using an earlier version (ANTLR v3).
Recommended version range: @@@
Pop-ups about updates.
See Github Demo.
For the TPTP instantiation, the Web-based service (project PSOATransRunWebService) can no longer be found at http://psoa-ruleml.rhcloud.com [cf. item 1) at PSOA_RuleML#TPTP_Instantiation]. Perhaps Mark Proctor's Red Hat OpenShift (https://www.openshift.com) colleague Daniel McPherson can help again.
See if new version, supporting SWI, is also relevant for XSB:
8.8 XSB Prolog
For Windows, install XSB 3.6 rather than the more recent 3.7 or 3.8: