DataGen Parameters

1. Overview
2. Predicting Attributes
   • Relevant Attributes
   • Datatypes
   • Attribute Domains
   • Disturbances
     • Masked and Irrelevant Attributes
   • Explicit Definition
3. Rules
   • Quantity
   • Distribution
     • Uniform, Random and Standard Normal
   • Conjunctions
   • Disjunctions
4. Tuples
   • Disturbances
     • Error Ratio
     • Missing-Values Ratio
5. General
   • Report Style
     • Plain
     • Verbose
   • Randomness Style

---

Overview

The DatGen parameters modify both the characteristics of the generated domain theory and the way the theory is applied to generate events. Rules are composed of head and body were the head is a single predicted attribute-value and the body is composed of one or more predicting attribute-values.

---

Predicting Attributes

In synthetic classification data sets there is one predicted attribute and the remaining predicting attributes. The quantity and distribution of the predicted attribute is determined by the rule base. Predicting attribute characteristics include their quantity, datatype and domain size. Furthermore some common real world disturbance should also be modeled including missing relevant attributes and completely irrelevant attributes.

• Relevant Attributes This parameter sets the number of relevant attributes which DatGen can compose a rule base from. If this number is smaller than the allowable conjunction size per rule then the number of relevant attributes will be the default.

• Datatypes Each attribute can have either a nominal or ordinal datatype. Nominal attributes do not contain an ordering. They include, for example, the name of cities. Ordinal attributes do have an ordering. They include, for example, a persons age. Currently this parameter can only be set in an explicit definition. A disjunctive rule term based on an ordinal attribute, such us A=(7 or 8 or 9) will have its members form a sequence. Because nominal values do not have an ordering there's no sense of sequence available. The default datatype is nominal.

• Attribute Domains This parameter defines a range from which predicting attribute domains can be chosen.

• Disturbances Real world databases can have several flaws. These include missing and irrelevant attributes.

  • Masked Attributes: Databases normally do not collect all the relevant predicting attributes required for classification. This parameter sets the number of predicting attributes which will go undisplayed. As the number of masked attributes increases the predictive power of a knowledge discovery system will deteriorate.

  • Irrelevant Attributes: This is the number of attributes which will only have a random relationship with the predicted attribute. A good discovery tool should be able to work around these easily.

• Explicit Definition To more explicitly synthetically model some real-world model, predicting attribute characteristics can also be explicitly defined. A colon separated list is used to specify all the attributes. Each attribute can have several comma separated parameters.
  
  Attribute Paramaters
  • #: Domain (defaults to global domain setting)
  • #: Disjuncts Associated in Rule-Terms which use this Attribute
  • R | I: Relevant (default) or Irrelevant Attribute
  • N | O: Nominal (default) or Ordinal Attribute Datatype
  • V | M: Visible (defaul) or Masked Attribute
  
  

  Eg. 10,3,R,N,V:22,R,O,V:33,5,I,O,V:44,2,R,M,V

---

Rules

The generated domain theory is composed of conjunctive normal form rules. These parameters set the number of such rules, their qualitative structure and how they will be invoked to create events.

• Quantity: This parameter sets the number of CNF rules in the rule base. This also defines the domain size of the the predicted attribute. In other words this is the number of classes in this data set.

• Distribution: The representation of classes can take many shapes, from equal to standard normal. How often a rule is invoked to generate events will determine the distribution of the classes.
  
  
  • rules invoked sequentially.
  • rules invoked randomly.
  • rules invoked randomly but with a standard normal bias.

• Conjunctions: Rule bodies are composed of terms. This parameter specifies the allowable range of terms per rule. For example the rule body attribute1=valueA AND attribute2=valueB has one conjunction, while the single term attribute1=(valueA OR valueB) has no conjunction.

• Disjunctions: Rule terms can have more than one possible value. This parameters specifies a range of values per term. For example the term attribute5=(valueA OR valueB) has a single disjunction, while the term attribute3=valueD involves no disjunction.

---

Tuples

Once the predicting attributes and the rule base are defined the system can proceed to generate data tuples (records). As the number of tuples is increased discovery tools will be able to make better predictions because of the increased information.

• Error Ratio: The error ratio determines the number of predicting attribute-values which take on a random value. This is used to simulate noise in the data, which occurs for example when a value is mistyped.

• Missing-Values Ratio: The missing-values ratio determines the number of predicting attribute-values which do not contain a value. This is used to simulate failure to enter information. The current strategy is to only miss values from a tuples attribute if that attribute is not required to classify it. This would model settings were an analyst stops gathering information because a classification is already at hand.

---

General

• Report Style: Reports can be plain or verbose.
  
  
  • Plain reports include only a simply structured synthetic data set and rule base which can be passed to a knowledge discovery tool.
    • Rule terms of masked attributes have a * beside their name
    
    
  • Verbose reports include the parameters used for the query, a glorified synthetic data set and a rule base.
    • Irrelevant attributes have a * beside their name
    • Masked attributes have a * for all their values
    • Erroneously entered values have a * beside them

• Randomness Style: By default the program uses nondeterministic randomness by selecting the dayofyear/hour/minute/second integer as a seed. Pseudorandomness may be useful to test that a program which uses the synthetic data will generate the same results given the same data. Note that different hardware/software combinations will create different sequences from each other.