Other implementations may appear in later versions of this manual.

Partial orders

It is anticipated that these macros will be used in wffs, e.g., in the :definition argument to defrelation. Having defined a relation, R, that is a partial order, one might use it for such purposes as:

(sort seq #'(lambda (x y) (?? R x y)))

None of the arguments of these macros is evaluated. We use the term "ordering expression" to mean any of the following:

the NAME of a relation that is a partial order

a description, ((v1 v2) s.t. ...), that is a partial order

a list whose macroexpansion is an ordering expression

Macro (inverse-order ordering-expression)

This is just the inverse relation of the argument - if the argument relates x to y then the inverse relates y to x.

Macro (priority-order ordering-expression+)

A priority order P is a sequence of partial orders. The priority order relates x to y, if there is some member of the sequence that relates x to y, and no earlier member relates y to x.

Macro (order-by-class class-expression {ordering-expression})

A class-based ordering C causes members of the class (type) denoted by class-expression to precede non-members. Two members of the class are ordered with respect to one another by the order denoted by ordering-expression, if it is provided. Specifically, C relates x to y if and only if either, x is a member of the designated class and y is not, or both x and y are members of the class, and ordering-expression relates x to y. Class-expression may be either the name of a unary relation or a unary description. The default ordering-expression is the empty ordering -- ((x y) s.t. false).

Macro (image-order ordering-expression image-relation)

The image order orders values indirectly in terms of an ordering on values to which they are related. Image-relation may be the name of a binary relation or may be a binary description. Let R be the relation denoted by image-relation, and O be the relation denoted by ordering-expression. The image order relates x to y if and only if

(E (x' y') (and (O x' y') (R x x') (R y y')))

Macro (Lexicographic-order ordering-expression)

A lexicographic order orders sequences in terms of an ordering on their positionally corresponding members. Suppose O is the partial order denoted by ordering-expression, and x and y are sequences (lists or vectors). Then the lexicographic order relates x to y if, there is some index, i, such that O relates the i'th element of x to the i'th element of y, and for all smaller indices, j, the j'th elements of x and y are not related (in either order) by O.

Macro (literal-order equivalence . values)

A literal order orders the elements of values according to their position in the list. It relates x to y if the first occurrence of x precedes the first occurrence of y in values, or if x occurs in values and y does not. Equivalence must be the name of an equivalence relation, or a binary description that satisfies the axioms for being an equivalence relation. It is used to define occurrence in the list. (See Equivalence.)

Rules, Constraints and Atomic Transitions

The Atomic construct provides control over the size of database updates. The lisp macro



Macro (atomic form* {ifabort abortform*} {ifnormal normalform*})

evaluates form* in order (barring an abort, described below). However, all attempts to update the database are held off until the end, at which time they are all done "at once". This means that no rule can react to a situation in which some but not all of these changes have been made. Every primitive update, u (a ++ or -- ) is semantically equivalent to (Atomic u).

As a first order approximation (refined below), we can view an AP5 program as doing a sequence of atomic updates:

(loop while t do (atomic ...))

(let (automation-queue) [do atomic update] (loop while automation-queue do (funcall (pop automation-queue))))

Furthermore, AP5 expands each (do atomic update) into an inner loop something like:

(loop until [proposed atomic update consistent] do [attempt to fix atomic update]

At any time inside the form* of an atomic, execution of the macro



Macro (abort tag format-string . objects)

will cause an exit to the (outermost) atomic, with no changes to the database. In this case the elements of abortform* for that atomic are evaluated, and the value(s) of the last returned from the atomic. These may refer to

Variable abortdata

This is bound to a list which starts with the arguments to abort, and then contains additional information (which we will try to formalize in a later edition of this manual), including the data built up so far to represent this transition in the global-history list, documented in the section on transition history. The default ifabort code calls error on the arguments to abort, excluding the tag. The idea is that the tag should be used by abortform* as a classification of the problem, so it can decide how to recover. Of course, it is free to use any other data returned from the abort as well. For atomics without ifabort forms, including database updates that are not contained in an explicit atomic, the format-string provides a way to describe to the user what went wrong.

The following macro is useful for debugging aborted updates:

Macro (from-abort {({:abortdata abortdata} {:updatestatus updatestatus} {:delta delta})} . body)

In its simplest form, when you're in an error caused by an abort (so abortdata is bound), (From-Abort ()) simply puts you in the context of the abort and executes (BREAK). This allows you to evaluate expressions from the "last" inconsistent state before the abort. Updates are not allowed. A saved previous value of abortdata may be used as the value of abortdata and forms to be evaluated in the aborted context may also be supplied.

There are also facilities for retrying and correcting aborted transitions. These are not detailed here because we hope they will be self explanatory.

If there is no abort, then after the database is updated, normalform* is executed and the values of the last are returned from the atomic. If no normalforms are supplied, the values of the successful atomic are those of the last of form*.

Within an atomic, (Atomic form* ifabort abortform* ifnormal normalform*) is equivalent to (progn form* normalform*) , i.e., any abort will escape to the outermost atomic. Since atomics are dynamically scoped, the same piece of code might have different effects depending on whether it is executed from inside an atomic.

Macro (insist {explanation-string} . wff)

will cause the atomic to abort (in the end) if wff would not be true in the resulting database. This wff may refer to runtime values which are computed at the time of the insist. For example, if one has the lisp variable x bound to an object which should end up being in relation R, (Insist (R x)) will make sure that it is. The explanation is simply used as an error message if the wff turns out to be false.

Macro (inatomic)

returns a non-NIL value if executed inside an atomic and NIL otherwise. In particular, you can write code that is not allowed to be executed as part of an atomic by doing (when (InAtomic) (Abort ...)) .

Although we don't expect it to be needed very often, we mention that at the top level of an outermost atomic, the symbol REVEAL allows things that are done afterward in the atomic to "see the effects" of updates done before the reveal. The effects do not include any rules that would react to those changes. This might be useful if, for example, you want to create a rule and, in the same atomic transition, assert something about that rule. If you do

(atomic (defautomation rule1 ...) (++ P (theonly r s.t. (rulename r 'rule1))))

(atomic (defautomation rule1 ...) reveal (++ P (theonly r s.t. (rulename r 'rule1))))

For now we consider it an error for reveal to be used anywhere other than the top level of an outermost atomic. We may later define what it means in inner atomics, in consistency rule responses, etc.

Examining and undoing history

Function (history ...)

is the history browser. It uses the *query-io* stream to display information, ask questions and get answers. Many print control parameters can be set interactively. Initial values for these parameters can be passed in as keyword arguments. These default to the values of special variables which can be set or bound in order to make defaults more permanent. The browser offers two types of "undo", which we'll call "forward" and "backward". Backward undoing corresponds to "going back in time". Only the most recent event can be undone backward, after which it is "forgotten" and the previous event can be undone backwards. Backward undoing ignores rules. Undoing "forward" means doing a new atomic update with the opposite effect of the current event: delete all the stored tuples added and add all the stored tuples deleted. Forward undo's are done with rules in effect. Any event may be undone forward (and in fact the undo may later be undone). Note, however, that forward undo's may abort or trigger automations.

Note: See the note in the section about decaching.

If you plan to back up to a particular point, another mechanism is more useful:

Function (history-label name)

creates an event labelled with name.

Function (undo-to name)

undoes (backward) all events back to (and including) the one labelled with name.

This facility is used to provide a database transaction macro:

Macro (transaction {label} . body)

If the first item in a transaction is a symbol then it is treated as a history label for the beginning of the transaction. A transaction may not be executed inside an atomic. It may contain atomic transactions to be executed sequentially. At any time inside a transaction the form:

Macro (abort-transaction . values)

will cause any transitions done inside the innermost transaction to be undone, and the values will be returned as the values of the transaction. If no AbortTransaction (or throw outside the transaction) is done, the values of the last form in the transaction are used as the values of the transaction. N.B. Any non-local transfer of control from inside a transaction to outside causes an abort-transaction.

Macro (intransaction)

returns T from inside a transaction, NIL otherwise. Transactions may be nested.

Consistency Rules

Intuitively we want atomic transitions to be augmented only with updates that are needed to satisfy the constraints. Unfortunately there may be alternative ways to satisfy this informal specification. It is also desirable for the augmentation to be easily predicted by the programmer (it helps for it to be deterministic) and relatively efficient for the machine to compute. We provide here a heuristic approximation that in practice seems to meet these requirements.

When a set of changes is proposed as an atomic transition, AP5 determines whether the state resulting from that set of changes would be consistent (satisfy all constraints). If so, the changes are made and the transition is complete. Otherwise, the consistency rules associated with the threatened constraints (those that would be violated by the proposed transition) are executed in an environment that allows access to both the (inconsistent) proposed state and the (consistent) current state. The rules do not see augmentations proposed by other rules. If no rules propose any augmentations, the transition aborts (since it's inconsistent and nothing can be done to fix it). If any rule proposes an augmentation that's incompatible with another proposed update (either original or augmentation), the transition aborts. Otherwise, all the proposed augmentations are made to the proposed set of updates and the result is proposed as an atomic transition (which may trigger more consistency rules).

This algorithm is deterministic (if the rules are deterministic) and in fact the result does not depend on the order in which the rules are considered. However infinite loops are possible, e.g., if there's a rule which reacts to the addition of R(n) for any integer n by adding R(n+1). Next, it is obvious (but important to understand) that the code in a consistency rule must be able to tolerate an inconsistent database. Also, when several cycles of consistency rules are needed, the cycle in which something happens (and thus the length of the path from one rule to another) can make a difference.

As an example, suppose we start in a situation where the (unary) relation P is everywhere false. There are two rules, R1 and R2. R1 requires that

(A (x) (implies (P x) (E (y) (Q y)))) . It reacts to a threat of violation by doing (++ Q 1) . R2 requires that (A (x) (implies (P x) (Q x))) , and reacts to a threat of violation (for x) by doing (++ Q x) . Clearly, if we do (++ P 2) , the reaction of rule R2 will in some sense make the action of R1 unnecessary. However, the presence of both rules will cause the addition of two instances of Q. suppose, however, that R1 had been replaced by two rules, R1a and R1b, which communicate through a new relation P1: R1a requires that (A (x) (implies (P x) (P1 x))) , and reacts by doing (++ P1 x) , while R1b requires that (A (x) (implies (P1 x) (E (y) (Q y)))) , and reacts by doing (++ Q 1) . Now if we do (++ P 2) , only one instance of Q will be added. Rule R1a will be triggered and will cause (P1 2) to be added, but after that, R1b will never be triggered, due to the addition of (Q 2) .

One particularly important case is that if a rule creates some new object, and it is important that it not create two such objects, then its action should prevent it from triggering (as opposed to making an update that will cause some other rule to do something that will prevent it from triggering). Of course, if the action can be done many times without causing any trouble, such as making the same assertion, then a long path (in terms of consistency cycles) to the solution (in the sense that the rule will no longer trigger) still works.

Another point is that the inability to remove an update from a transition sometimes necessitates the toleration of redundancy. As an example, suppose we have a stored relation (R x y) , but we really only want to access its transitive closure, R*. Then we might want to try to keep R minimal, in the sense that there's no need to store (R x y) when (R* x y) can be derived in its absence. However, a rule that prohibited such redundancy would prevent otherwise legal updates, such as

(atomic (++ R a b) (++ R b c) (++ R a c))

(R a c)

note: We are considering how to provide the ability to install such a "preprocessor" for derived relations, but this is not yet provided.

In order to decide how to react to an inconsistent set of proposed changes, a consistency rule may want to see what has changed. For example, one interpretation of the constraint that nobody have more than one office is that whenever an office is given to a person, if he already had another office then he should be required to vacate it. In order to see what is changed, consistency rules are allowed to use a limited form of temporal reference in the wffs that they use for database access. In particular, the two special forms

(Previously wff) and

(Start wff)

are allowed as wffs. In the absence of these temporal operators a wff refers to the proposed, possibly inconsistent state. Previously refers to the (consistent) state before the atomic transition started. (Start wff) means (and wff (not (Previously wff))) , i.e., the wff is proposed to become true. Temporal reference is allowed in the following places:

In reactions of consistencyrules

In triggers of consistency and automation rules (introduced below)

In functions for generating automation invocations

In Trigger code (introduced later)

Macro (previously . body)

is also defined as a lisp macro. The forms are evaluated as of the previous state. The macro generates an error if executed outside one of the situations (listed above) where temporal reference is allowed.

There are subtle differences between consistency rules that maintain time sensitive as opposed to time insensitive conditions. For example, if we prohibit (Start P) , P is allowed to be true, but it may not become true after having been false. Also, the knowledge that a time insensitive wff was true (or false) before means that certain things need not be checked. For example, if we prohibit (E (x) (P x)) , then whenever P becomes true of anything, the condition becomes violated. On the other hand, if we prohibit (Start (E (x) (P x))) , then as long as there is already an example of P, adding a new one will not violate the condition. The great majority of consistency triggers will not use Start.

The normal ways to declare constraints are:

Macro (alwaysrequired name trigger {:repair repair} {:enforcement-level enforcement} {:reporter reporter} {:documentation documentation})



Macro (neverpermitted name trigger {:repair repair} {:enforcement-level enforcement} {:reporter reporter} {:documentation documentation})



The only difference is that AlwaysRequired creates a rule that reacts when the condition is (about to be) false, while NeverPermitted reacts when it's true. Functions can create rules with computed names, triggers, etc. by calling the following versions defined as functions rather than macros:

Function (alwaysrequire name trigger reaction enforcement {:reporter reporter} {:documentation documentation})



Function (neverpermit name trigger reaction enforcement {:reporter reporter} {:documentation documentation})

The macro versions do more at compile time (an advantage when you compile to a file). They also compile reactions that are specified as lambda-expressions.

Name is a symbol, trigger is a wff, and reaction is a lisp function. The rule may be interpreted as meaning that the wff should never be true (for neverpermitted) or false (for alwaysrequired). If it threatens to become true, then the reaction is executed in an attempt to "repair" the problem. If the trigger has the form (E vars ...) for neverpermitted or (A vars ...) for alwaysrequired (or macroexpands to such a form), then every binding of vars which serves as an example (and thus violates the constraint) will be passed to a separate invocation of the reaction. The reaction function is meant to do something to maintain the constraint (for this binding of vars).

Function (ignore ...)

is a function which does nothing with any number of arguments. This is useful as a reaction if you don't have a reaction to propose but do not want to automatically abort (it's possible that another rule might fix the problem).

Enforcement is one of { :total , :incremental , :none }, indicating the degree to which AP5 enforces the constraint. Total means that when the constraint is declared, AP5 checks to see that it is satisfied, [Actually, total means to insist that the constraint be satisfied in the state resulting from the addition of the rule.] and also notices (and takes the appropriate action) when it is about to be violated. Incremental means that AP5 assumes the constraint to be satisfied when it is declared but watches for violations. [In different versions, the rule itself may become active at different times, e.g., only after the atomic transition is complete, or after the consistency cycle in which the (proposed) database contains the rule. Of course, in the second case, it will be deactivated by an abort.] None means that AP5 simply assumes the constraint is always satisfied. Other enforcement levels may be supported in the future. Different levels are useful because sometimes it is impossible, impractical or unnecessary to enforce a constraint. The AP5 compiler will feel free to make use of constraints as assumptions, even if it is not enforcing them.

Reporter allows you to specify a function for reporting failures of consistency rules. If an atomic transition aborts because some rules could not be satisfied and if no IFABORT was supplied, this rule-specific error-reporting code is used (if present) to print error messages. The arguments to this function are the same as those for the reaction of the rule.

There's a constraint that only one rule have a given name. If you try to name a new rule with the same name as an old one, the old rule is deleted.

Ruletriggers are like wffdefns in that they are not allowed to contain arbitrary lisp forms to be evaluated. All arguments to relations must be bound variables or constants. (However, see the description of Memo.)

The RuleTrigger and Constraint-Enforcement-Level cannot be altered (other than by creating or destroying the rule). If you want to change them, you can redeclare the rule (with different values in those slots), which will have the effect of replacing the rule.

Example:

(neverpermit 'Two-offices-for-same-person '(E (x y person) (and (Office person x) (Office person y) (not (eql x y)))) #'(lambda (x y person) (declare (ignore x)) (when (?? Previously (Office person y)) (-- Office person y))) :incremental :reporter #'(lambda (ignore ignore2 person) (declare (ignore ignore ignore2)) (format *error-output* "~A should not have two offices" person)))

Note that when a state is proposed in which p has two offices, the rule will trigger twice - each office has a turn at being both x and y. If both offices are new, there is no reaction, and the condition remains violated - which causes an abort. An alternative reaction could have been

(when (?? Start (Office person x)) (-- Office person y))

Several constraint idioms have been identified. These have been (or will be) made into relations:

subtype constraints, e.g., every integer is a number - these are discussed in the chapter on types

type constraints on relations, e.g., the FirstName relation would only relate people to strings - these are also discussed in the chapter on types.

disjoint types, e.g., nothing is both a symbol and a number - these will again be described in the chapter on types.

count restrictions, e.g., no person can have more than one spouse

Notes:

- Declaration of a rule involves compiling various pieces of triggering code. This tends to take some time.

- It may also generate errors (aborts) on the grounds that the rule cannot be triggered or tested. In some cases it may actually be possible to do what AP5 says it cannot, and I'm willing to see examples. In other cases it is not possible, and so the rule cannot be implemented, at least with the current implementations of the relations involved. See the section on large wffs .

Consistency Checkers

Type (consistencychecker rule)

means that rule is one of these generalized constraints. Such rules may be declared by the macro

Macro (defconsistencychecker name description response {:reporter reporter} {:documentation documentation})

Description is either a description that could be used as an automation rule trigger (see the section on automations), or the special description "(nil s.t. true)". On every consistency cycle, every match of the description causes an invocation of response. Unlike a consistency rule, the fact that such an invocation takes place does not necessarily mean that a constraint has been violated. That is decided by the response. Like a consistency response, the response may make updates, do insist's and even abort. If it makes updates (even noops) AP5 considers the current situation to be inconsistent, i.e., it violates the constraint represented by this rule. Otherwise, if the value of the response is NIL, AP5 considers the rule to be satisfied, and any other value indicates that it is violated. This mechanism is clearly more general than consistency rules, but is usually not needed. Also, consistency rules have the advantage that the constraint is explicit in the trigger, while for consistency checkers the constraint is at least partly in the lisp code.

Count Constraints

Cardinality (or "count") constraints express facts such as "every employee has at most one office", or "every employee has at least one office". The general form is:

For every n-tuple of objects, [x 1 , ..., x n ] of types t 1 , ..., t n (actually, descriptions can be used as well as types), there are count-restriction m+n-tuples in the relation R, which can be formed by inserting m arbitrary objects in specified positions in [x 1 , ..., x n ].

The types and the positions at which objects are to be inserted are specified by "patterns" which are lists of length m+n, which must be the arity of R. The elements of the list are either types or descriptions, or the symbol OUTPUT which stands for an inserted object.

The currenly supported values for count-restriction are:

:none There are no such tuples, e.g., while employees may in general be assigned secretaries, there should not be any secretaries who have secretaries: the constraint for the pattern (SECRETARY OUTPUT) is :none.

There are no such tuples, e.g., while employees may in general be assigned secretaries, there should not be any secretaries who have secretaries: the constraint for the pattern (SECRETARY OUTPUT) is :none. :multiple The relation must contain at least one tuple, e.g., if every employee has an office, the constraint for the pattern (EMPLOYEE OUTPUT) would be :multiple.

The relation must contain at least one tuple, e.g., if every employee has an office, the constraint for the pattern (EMPLOYEE OUTPUT) would be :multiple. :optional The relation must contain at most one tuple, e.g., if no employee can have more than one office, the constraint for the pattern (EMPLOYEE OUTPUT) would be :optional.

Of course, "none" implies "optional", and "multiple" is incompatible with "none". However, "multiple" and "optional" are independent. This leads to two more possible cases:

:unique Exactly one (both multiple and optional), e.g., every person has one office.

Exactly one (both multiple and optional), e.g., every person has one office. :any No constraint, e.g., a person can have any number of children.

Examples:

if we want to restrict the relation Works-On%(person, project, number) so that every employee works some amount on some project, the count-restriction would be :multiple and the pattern would be (employee output output), i.e., for every employee there is at least one project and number such that Works-On%(employee, project, number).

If we want to say that there can be no more than one number for any given person and project, we can use the pattern (employee project output) and the count-restriction :optional .

Cardinality constraints are usually declared as part of a relation declaration, with the :count argument of DefRelation. In fact, the value of this parameter is, in effect, the result of appending together a bunch of argument lists to the Restrict-Cardinality macro described below (except that the name of the relation being declared is left out).

Macro (restrict-cardinality relname pattern {:countspec countspec} {:enforcement enforcement} {:replacing replacing} {:default default} {:too-many-repair too-many-repair} {:too-few-repair too-few-repair} {:delete-any-other-countspecs delete-any-other-countspecs})

is the macro that declares count constraints. Relname is the name of a relation and pattern is a list, each element of which is a typename, description, or the symbol output . Countspec is one of { :any , :unique , :optional , :multiple , :none }. Enforcement is an enforcement level. The default countspec is :any and the default enforcement is :incremental.

If countspec is either :optional or :unique, and replacing is non-NIL, then the rule will have a reaction that deletes old tuples in order to "make room for" new ones. For example, giving a person a new name will cause his old name to be removed if the constraint is declared as replacing. The other side of the coin is that if countspec is either :unique or :multiple, the default parameter can be used to provide a reaction function capable of supplying one or more tuples when the constraint is violated. The default function will be applied to an N-tuple of values that are instances of the types named in the N typed slots of the pattern. It should return, as multiple values, zero or more M-tuples, each M-tuple containing a value for each of the M OUTPUT slots of the pattern. The M+N tuple formed by merging the input with the outputs of the function will be added to the database as the reaction. [If the function wishes to supply NO tuples in a particular case, it should return NO values, not return NIL.]

Other reaction code can be supplied via the arguments Too-Many-Repair and Too-Few-Repair. (Unique constraints are represented as two rules, so both reactions can be specified independently.) The default reactions do nothing.

In the case of Too-Many-Repair violation of an upper bound constraint will result in application of the function. If countspec is :optional, it will be applied to the N values corresponding to the typed pattern elements for which there are too many tuples. If countspec is :none, it will be applied to N+M values. They will be the slots of a tuple that violates the constraint. The arguments will be passed in the order of the slots of the relation, immaterial of which slots in the pattern were typed and which were OUTPUT. The function, like any other reaction function, may assert and retract facts to reestablish consistency. [The Replacing keyword is a special case of a Too-Many-Repair; it is an error to have non-NIL values for both keywords.]

In the case of Too-Few-Repair, the violation of a MULTIPLE constraint will result in application of the function to N values -- the same ones described above for a Default function. The function, like any other reaction function, may assert and retract facts to reestablish consistency. [The Default keyword is a special case of a Too-Few-Repair; it is an error to have non-NIL values for both keywords.]

Delete-any-other-countspecs may be T, NIL, or a list of patterns. T means the same thing as (list pattern). The meaning is that all count constraints for this relation except those for the patterns listed are to be removed. However NIL means not to remove any other constraints.

In order to find out what cardinality constraints apply in a given case, the following are provided:

Function (cardinality-of-pattern rel-or-name pattern)

Function (cardinality-for-tuple rel-or-name i-o-pattern {output-indicator}) The first accepts a relation and pattern and returns a countspec that is implied by existing count constraints. The second is similar but the pattern is altered in that all the types are replaced by objects and the "output" slots are identified by being EQ to output-indicator, which defaults to the symbol OUTPUT but can be supplied if that is one of the objects you want to pass in. This returns a countspec for those particular objects, sort of like asking about all the tuples of the types of those objects and merging the results.