Entropies and Combinatorics of Random Branching Processes and Context-Free Languages

The entropies and combinatorics of trees that branch according to fixed but finite numbers of rules are studied. Context-free grammars are used to categorize the ways in which nodes branch to yield daughter nodes, thus providing an organized setting to examine the entropies for random branching processes whose realizations are trees and whose probabilities are determined by probabilities associated to the substitution rules of the grammar. Normalized entropy rates H are derived for the critical branching rate (p = 1) and supercritical branching rate (p > 1) processes. An equipartition theorem is proven for the supercritical processes proving that -generation trees normalized by their number of nodes have log probability converging to the entropy rate H with L, almost everywhere in the nonextinction set. A strong departure from classical theorems for Markov sources occurs for super-critical branching processes p > 1 as the typical sets have super-geometric growth rates. Defining the 5-typical set of trees to be the T-generation trees with log of their negative log probability within 5 of log p, then the typical set has probability equaling the nonextinction probability and log growth rate of pL. The combinatorics of the set of all trees that can be generated from the context-free substitution rules is also studied. It is proven that for all context-free grammars that are strongly connected and have at least one substitution rule with two daughters or more, the combinatoric growth rate of the set of trees is also super-geometric and equals the largest growth rate of any random branching process with the same substitution rules. Instances of regular, pseudo-linear and context-free grammars are studied for demonstrating the theory, and as a particular example it is shown that the arithmetic expression language has log-number of unique L-generation programs growing at a rate 1.75488^L.