RaceSearch

Overview | Package | Class | Tree | Deprecated | Index | Help | Weka's home

PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## weka.attributeSelection
Class RaceSearch

    java.lang.Object
      weka.attributeSelection.ASSearch
          weka.attributeSelection.RaceSearch

All Implemented Interfaces:
  java.io.Serializable, RankedOutputSearch, OptionHandler, RevisionHandler, TechnicalInformationHandler

    public class RaceSearch

extends ASSearch
implements RankedOutputSearch, OptionHandler, TechnicalInformationHandler

Races the cross validation error of competing attribute subsets. Use in conjuction with a ClassifierSubsetEval. RaceSearch has four modes:
forward selection races all single attribute additions to a base set (initially no attributes), selects the winner to become the new base set and then iterates until there is no improvement over the base set.
Backward elimination is similar but the initial base set has all attributes included and races all single attribute deletions.
Schemata search is a bit different. Each iteration a series of races are run in parallel. Each race in a set determines whether a particular attribute should be included or not---ie the race is between the attribute being "in" or "out". The other attributes for this race are included or excluded randomly at each point in the evaluation. As soon as one race has a clear winner (ie it has been decided whether a particular attribute should be inor not) then the next set of races begins, using the result of the winning race from the previous iteration as new base set.
Rank race first ranks the attributes using an attribute evaluator and then races the ranking. The race includes no attributes, the top ranked attribute, the top two attributes, the top three attributes, etc.
It is also possible to generate a raked list of attributes through the forward racing process. If generateRanking is set to true then a complete forward race will be run---that is, racing continues until all attributes have been selected. The order that they are added in determines a complete ranking of all the attributes.
Racing uses paired and unpaired t-tests on cross-validation errors of competing subsets. When there is a significant difference between the means of the errors of two competing subsets then the poorer of the two can be eliminated from the race. Similarly, if there is no significant difference between the mean errors of two competing subsets and they are within some threshold of each other, then one can be eliminated from the race.
For more information see:
Andrew W. Moore, Mary S. Lee: Efficient Algorithms for Minimizing Cross Validation Error. In: Eleventh International Conference on Machine Learning, 190-198, 1994.

BibTeX:

     @inproceedings{Moore1994,
        author = {Andrew W. Moore and Mary S. Lee},
        booktitle = {Eleventh International Conference on Machine Learning},
        pages = {190-198},
        publisher = {Morgan Kaufmann},
        title = {Efficient Algorithms for Minimizing Cross Validation Error},
        year = {1994}
     }

Valid options are:

     -R <0 = forward | 1 = backward race | 2 = schemata | 3 = rank>
      Type of race to perform.
      (default = 0).

     -L <significance>
      Significance level for comaparisons
      (default = 0.001(forward/backward/rank)/0.01(schemata)).

     -T <threshold>
      Threshold for error comparison.
      (default = 0.001).

     -A <attribute evaluator>
      Attribute ranker to use if doing a
      rank search. Place any
      evaluator options LAST on
      the command line following a "--".
      eg. -A weka.attributeSelection.GainRatioAttributeEval ... -- -M.
      (default = GainRatioAttributeEval)

     -F <0 = 10 fold | 1 = leave-one-out>
      Folds for cross validation
      (default = 0 (1 if schemata race)

     -Q
      Generate a ranked list of attributes.
      Forces the search to be forward
      and races until all attributes have
      selected, thus producing a ranking.

     -N <num to select>
      Specify number of attributes to retain from
      the ranking. Overides -T. Use in conjunction with -Q

     -J <threshold>
      Specify a theshold by which attributes
      may be discarded from the ranking.
      Use in conjuction with -Q

     -Z
      Verbose output for monitoring the search.

     Options specific to evaluator weka.attributeSelection.GainRatioAttributeEval:

     -M
      treat missing values as a seperate value.

Version:
  $Revision: 1.26 $
Author:
  Mark Hall (mhall@cs.waikato.ac.nz)
See Also:
  Serialized Form

Field Summary
`static Tag[]` | `TAGS_SELECTION`
`static Tag[]` | `XVALTAGS_SELECTION`

Constructor Summary
`RaceSearch()`

Method Summary
`java.lang.String` | `attributeEvaluatorTipText()` Returns the tip text for this property
`java.lang.String` | `debugTipText()` Returns the tip text for this property
`java.lang.String` | `foldsTypeTipText()` Returns the tip text for this property
`java.lang.String` | `generateRankingTipText()` Returns the tip text for this property
`ASEvaluation` | `getAttributeEvaluator()` Get the attribute evaluator used to generate the ranking.
`int` | `getCalculatedNumToSelect()` Gets the calculated number of attributes to retain.
`boolean` | `getDebug()` Get whether output is to be verbose
`SelectedTag` | `getFoldsType()` Get the xfold type
`boolean` | `getGenerateRanking()` Gets whether ranking has been requested.
`int` | `getNumToSelect()` Gets the number of attributes to be retained.
`java.lang.String[]` | `getOptions()` Gets the current settings of BestFirst.
`SelectedTag` | `getRaceType()` Get the race type
`java.lang.String` | `getRevision()` Returns the revision string.
`double` | `getSelectionThreshold()` Returns the threshold so that the AttributeSelection module can discard attributes from the ranking.
`double` | `getSignificanceLevel()` Get the significance level
`TechnicalInformation` | `getTechnicalInformation()` Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
`double` | `getThreshold()` Get the threshold
`java.lang.String` | `globalInfo()` Returns a string describing this search method
`java.util.Enumeration` | `listOptions()` Returns an enumeration describing the available options.
`java.lang.String` | `numToSelectTipText()` Returns the tip text for this property
`java.lang.String` | `raceTypeTipText()` Returns the tip text for this property
`double[][]` | `rankedAttributes()` Returns a X by 2 list of attribute indexes and corresponding evaluations from best (highest) to worst.
`int[]` | `search(ASEvaluation ASEval, Instances data)` Searches the attribute subset space by racing cross validation errors of competing subsets
`java.lang.String` | `selectionThresholdTipText()` Returns the tip text for this property
`void` | `setAttributeEvaluator(ASEvaluation newEvaluator)` Set the attribute evaluator to use for generating the ranking.
`void` | `setDebug(boolean d)` Set whether verbose output should be generated.
`void` | `setFoldsType(SelectedTag d)` Set the xfold type
`void` | `setGenerateRanking(boolean doRank)` Records whether the user has requested a ranked list of attributes.
`void` | `setNumToSelect(int n)` Specify the number of attributes to select from the ranked list (if generating a ranking).
`void` | `setOptions(java.lang.String[] options)` Parses a given list of options.
`void` | `setRaceType(SelectedTag d)` Set the race type
`void` | `setSelectionThreshold(double threshold)` Set the threshold by which the AttributeSelection module can discard attributes.
`void` | `setSignificanceLevel(double sig)` Sets the significance level to use
`void` | `setThreshold(double t)` Sets the threshold for comparisons
`java.lang.String` | `significanceLevelTipText()` Returns the tip text for this property
`java.lang.String` | `thresholdTipText()` Returns the tip text for this property
`java.lang.String` | `toString()` Returns a string represenation

Methods inherited from class weka.attributeSelection.ASSearch
`forName, makeCopies`

Methods inherited from class java.lang.Object
`equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Field Detail

### TAGS_SELECTION

    public static final Tag[] TAGS_SELECTION

### XVALTAGS_SELECTION

    public static final Tag[] XVALTAGS_SELECTION

Constructor Detail

### RaceSearch

    public RaceSearch()

Method Detail

### globalInfo

    public java.lang.String globalInfo()

  Returns a string describing this search method

  Returns:
    a description of the search method suitable for displaying in the explorer/experimenter gui

### getTechnicalInformation

    public TechnicalInformation getTechnicalInformation()

  Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.

  Specified by:
    `getTechnicalInformation` in interface `TechnicalInformationHandler`
  Returns:
    the technical information about this class

### raceTypeTipText

    public java.lang.String raceTypeTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setRaceType

    public void setRaceType(SelectedTag d)

  Set the race type

  Parameters:
    `d` - the type of race

### getRaceType

    public SelectedTag getRaceType()

  Get the race type

  Returns:
    the type of race

### significanceLevelTipText

    public java.lang.String significanceLevelTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setSignificanceLevel

    public void setSignificanceLevel(double sig)

  Sets the significance level to use

  Parameters:
    `sig` - the significance level

### getSignificanceLevel

    public double getSignificanceLevel()

  Get the significance level

  Returns:
    the current significance level

### thresholdTipText

    public java.lang.String thresholdTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setThreshold

    public void setThreshold(double t)

  Sets the threshold for comparisons

  Specified by:
    `setThreshold` in interface `RankedOutputSearch`
  Parameters:
    `t` - the threshold to use

### getThreshold

    public double getThreshold()

  Get the threshold

  Specified by:
    `getThreshold` in interface `RankedOutputSearch`
  Returns:
    the current threshold

### foldsTypeTipText

    public java.lang.String foldsTypeTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setFoldsType

    public void setFoldsType(SelectedTag d)

  Set the xfold type

  Parameters:
    `d` - the type of xval

### getFoldsType

    public SelectedTag getFoldsType()

  Get the xfold type

  Returns:
    the type of xval

### debugTipText

    public java.lang.String debugTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setDebug

    public void setDebug(boolean d)

  Set whether verbose output should be generated.

  Parameters:
    `d` - true if output is to be verbose.

### getDebug

    public boolean getDebug()

  Get whether output is to be verbose

  Returns:
    true if output will be verbose

### attributeEvaluatorTipText

    public java.lang.String attributeEvaluatorTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setAttributeEvaluator

    public void setAttributeEvaluator(ASEvaluation newEvaluator)

  Set the attribute evaluator to use for generating the ranking.

  Parameters:
    `newEvaluator` - the attribute evaluator to use.

### getAttributeEvaluator

    public ASEvaluation getAttributeEvaluator()

  Get the attribute evaluator used to generate the ranking.

  Returns:
    the evaluator used to generate the ranking.

### generateRankingTipText

    public java.lang.String generateRankingTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setGenerateRanking

    public void setGenerateRanking(boolean doRank)

  Records whether the user has requested a ranked list of attributes.

  Specified by:
    `setGenerateRanking` in interface `RankedOutputSearch`
  Parameters:
    `doRank` - true if ranking is requested

### getGenerateRanking

    public boolean getGenerateRanking()

  Gets whether ranking has been requested. This is used by the AttributeSelection module to determine if rankedAttributes() should be called.

  Specified by:
    `getGenerateRanking` in interface `RankedOutputSearch`
  Returns:
    true if ranking has been requested.

### numToSelectTipText

    public java.lang.String numToSelectTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setNumToSelect

    public void setNumToSelect(int n)

  Specify the number of attributes to select from the ranked list (if generating a ranking). -1 indicates that all attributes are to be retained.

  Specified by:
    `setNumToSelect` in interface `RankedOutputSearch`
  Parameters:
    `n` - the number of attributes to retain

### getNumToSelect

    public int getNumToSelect()

  Gets the number of attributes to be retained.

  Specified by:
    `getNumToSelect` in interface `RankedOutputSearch`
  Returns:
    the number of attributes to retain

### getCalculatedNumToSelect

    public int getCalculatedNumToSelect()

  Gets the calculated number of attributes to retain. This is the actual number of attributes to retain. This is the same as getNumToSelect if the user specifies a number which is not less than zero. Otherwise it should be the number of attributes in the (potentially transformed) data.

  Specified by:
    `getCalculatedNumToSelect` in interface `RankedOutputSearch`

### selectionThresholdTipText

    public java.lang.String selectionThresholdTipText()

  Returns the tip text for this property

  Returns:
    tip text for this property suitable for displaying in the explorer/experimenter gui

### setSelectionThreshold

    public void setSelectionThreshold(double threshold)

  Set the threshold by which the AttributeSelection module can discard attributes.

  Parameters:
    `threshold` - the threshold.

### getSelectionThreshold

    public double getSelectionThreshold()

  Returns the threshold so that the AttributeSelection module can discard attributes from the ranking.

### listOptions

    public java.util.Enumeration listOptions()

  Returns an enumeration describing the available options.

  Specified by:
    `listOptions` in interface `OptionHandler`
  Returns:
    an enumeration of all the available options.

### setOptions

    public void setOptions(java.lang.String[] options)
                    throws java.lang.Exception

  Parses a given list of options.

  Valid options are:

       -R <0 = forward | 1 = backward race | 2 = schemata | 3 = rank>
        Type of race to perform.
        (default = 0).

       -L <significance>
        Significance level for comaparisons
        (default = 0.001(forward/backward/rank)/0.01(schemata)).

       -T <threshold>
        Threshold for error comparison.
        (default = 0.001).

       -A <attribute evaluator>
        Attribute ranker to use if doing a
        rank search. Place any
        evaluator options LAST on
        the command line following a "--".
        eg. -A weka.attributeSelection.GainRatioAttributeEval ... -- -M.
        (default = GainRatioAttributeEval)

       -F <0 = 10 fold | 1 = leave-one-out>
        Folds for cross validation
        (default = 0 (1 if schemata race)

       -Q
        Generate a ranked list of attributes.
        Forces the search to be forward
        and races until all attributes have
        selected, thus producing a ranking.

       -N <num to select>
        Specify number of attributes to retain from
        the ranking. Overides -T. Use in conjunction with -Q

       -J <threshold>
        Specify a theshold by which attributes
        may be discarded from the ranking.
        Use in conjuction with -Q

       -Z
        Verbose output for monitoring the search.

       Options specific to evaluator weka.attributeSelection.GainRatioAttributeEval:

       -M
        treat missing values as a seperate value.

  Specified by:
    `setOptions` in interface `OptionHandler`
  Parameters:
    `options` - the list of options as an array of strings
  Throws:
    `java.lang.Exception` - if an option is not supported

### getOptions

    public java.lang.String[] getOptions()

  Gets the current settings of BestFirst.

  Specified by:
    `getOptions` in interface `OptionHandler`
  Returns:
    an array of strings suitable for passing to setOptions()

### search

    public int[] search(ASEvaluation ASEval,
                        Instances data)
                 throws java.lang.Exception

  Searches the attribute subset space by racing cross validation errors of competing subsets

  Specified by:
    `search` in class `ASSearch`
  Parameters:
    `ASEval` - the attribute evaluator to guide the search
    `data` - the training instances.
  Returns:
    an array (not necessarily ordered) of selected attribute indexes
  Throws:
    `java.lang.Exception` - if the search can't be completed

### rankedAttributes

    public double[][] rankedAttributes()
                                throws java.lang.Exception

  Description copied from interface: `RankedOutputSearch`
  Returns a X by 2 list of attribute indexes and corresponding evaluations from best (highest) to worst.

  Specified by:
    `rankedAttributes` in interface `RankedOutputSearch`
  Returns:
    the ranked list of attribute indexes in an array of ints
  Throws:
    `java.lang.Exception` - if the ranking can't be produced

### toString

    public java.lang.String toString()

  Returns a string represenation

  Overrides:
    `toString` in class `java.lang.Object`
  Returns:
    a string representation

### getRevision

    public java.lang.String getRevision()

  Returns the revision string.

  Specified by:
    `getRevision` in interface `RevisionHandler`
  Overrides:
    `getRevision` in class `ASSearch`
  Returns:
    the revision

Overview | Package | Class | Tree | Deprecated | Index | Help | Weka's home

PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD