public class AdaptiveLogisticRegression extends Object implements OnlineLearner, org.apache.hadoop.io.Writable
OnlineLogisticRegression
learners. Each
member of the pool has different learning rates. Whichever of the learners in the pool falls
behind in terms of average log-likelihood will be tossed out and replaced with variants of the
survivors. This will let us automatically derive an annealing schedule that optimizes learning
speed. Since on-line learners tend to be IO bound anyway, it doesn't cost as much as it might
seem that it would to maintain multiple learners in memory. Doing this adaptation on-line as we
learn also decreases the number of learning rate parameters required and replaces the normal
hyper-parameter search.
One wrinkle is that the pool of learners that we maintain is actually a pool of
CrossFoldLearner
which themselves contain several OnlineLogisticRegression
objects. These pools allow estimation
of performance on the fly even if we make many passes through the data. This does, however,
increase the cost of training since if we are using 5-fold cross-validation, each vector is used
4 times for training and once for classification. If this becomes a problem, then we should
probably use a 2-way unbalanced train/test split rather than full cross validation. With the
current default settings, we have 100 learners running. This is better than the alternative of
running hundreds of training passes to find good hyper-parameters because we only have to parse
and feature-ize our inputs once. If you already have good hyper-parameters, then you might
prefer to just run one CrossFoldLearner with those settings.
The fitness used here is AUC. Another alternative would be to try log-likelihood, but it is much
easier to get bogus values of log-likelihood than with AUC and the results seem to accord pretty
well. It would be nice to allow the fitness function to be pluggable. This use of AUC means that
AdaptiveLogisticRegression is mostly suited for binary target variables. This will be fixed
before long by extending OnlineAuc to handle non-binary cases or by using a different fitness
value in non-binary cases.Modifier and Type | Class and Description |
---|---|
static class |
AdaptiveLogisticRegression.TrainingExample |
static class |
AdaptiveLogisticRegression.Wrapper
Provides a shim between the EP optimization stuff and the CrossFoldLearner.
|
Modifier and Type | Field and Description |
---|---|
static int |
DEFAULT_POOL_SIZE |
static int |
DEFAULT_THREAD_COUNT |
Constructor and Description |
---|
AdaptiveLogisticRegression() |
AdaptiveLogisticRegression(int numCategories,
int numFeatures,
PriorFunction prior)
Uses
DEFAULT_THREAD_COUNT and DEFAULT_POOL_SIZE |
AdaptiveLogisticRegression(int numCategories,
int numFeatures,
PriorFunction prior,
int threadCount,
int poolSize) |
Modifier and Type | Method and Description |
---|---|
double |
auc()
What is the AUC for the current best member of the population.
|
void |
close()
Prepares the classifier for classification and deallocates any temporary data structures.
|
State<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> |
getBest() |
List<AdaptiveLogisticRegression.TrainingExample> |
getBuffer() |
EvolutionaryProcess<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> |
getEp() |
int |
getMaxInterval() |
int |
getMinInterval() |
int |
getNumCategories() |
int |
getNumFeatures() |
PriorFunction |
getPrior() |
int |
getRecord() |
State<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> |
getSeed() |
int |
nextStep(int recordNumber) |
int |
numFeatures()
Returns the size of the internal feature vector.
|
void |
readFields(DataInput in) |
void |
setAucEvaluator(OnlineAuc auc) |
void |
setAveragingWindow(int averagingWindow) |
void |
setBest(State<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> best) |
void |
setBuffer(List<AdaptiveLogisticRegression.TrainingExample> buffer) |
void |
setEp(EvolutionaryProcess<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> ep) |
void |
setFreezeSurvivors(boolean freezeSurvivors) |
void |
setInterval(int interval)
How often should the evolutionary optimization of learning parameters occur?
|
void |
setInterval(int minInterval,
int maxInterval)
Starts optimization using the shorter interval and progresses to the longer using the specified
number of steps per decade.
|
void |
setPoolSize(int poolSize) |
void |
setRecord(int record) |
void |
setSeed(State<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> seed) |
void |
setThreadCount(int threadCount) |
static int |
stepSize(int recordNumber,
double multiplier) |
void |
train(int actual,
Vector instance)
Updates the model using a particular target variable value and a feature vector.
|
void |
train(long trackingKey,
int actual,
Vector instance)
Updates the model using a particular target variable value and a feature vector.
|
void |
train(long trackingKey,
String groupKey,
int actual,
Vector instance)
Updates the model using a particular target variable value and a feature vector.
|
void |
write(DataOutput out) |
public static final int DEFAULT_THREAD_COUNT
public static final int DEFAULT_POOL_SIZE
public AdaptiveLogisticRegression()
public AdaptiveLogisticRegression(int numCategories, int numFeatures, PriorFunction prior)
DEFAULT_THREAD_COUNT
and DEFAULT_POOL_SIZE
numCategories
- The number of categories (labels) to train onnumFeatures
- The number of features used in creating the vectors (i.e. the cardinality of the vector)prior
- The PriorFunction
to useAdaptiveLogisticRegression(int, int, org.apache.mahout.classifier.sgd.PriorFunction, int, int)
public AdaptiveLogisticRegression(int numCategories, int numFeatures, PriorFunction prior, int threadCount, int poolSize)
numCategories
- The number of categories (labels) to train onnumFeatures
- The number of features used in creating the vectors (i.e. the cardinality of the vector)prior
- The PriorFunction
to usethreadCount
- The number of threads to use for trainingpoolSize
- The number of CrossFoldLearner
to use.public void train(int actual, Vector instance)
OnlineLearner
train
in interface OnlineLearner
actual
- The value of the target variable. This value should be in the half-open
interval [0..n) where n is the number of target categories.instance
- The feature vector for this example.public void train(long trackingKey, int actual, Vector instance)
OnlineLearner
train
in interface OnlineLearner
trackingKey
- The tracking key for this training example.actual
- The value of the target variable. This value should be in the half-open
interval [0..n) where n is the number of target categories.instance
- The feature vector for this example.public void train(long trackingKey, String groupKey, int actual, Vector instance)
OnlineLearner
train
in interface OnlineLearner
trackingKey
- The tracking key for this training example.groupKey
- An optional value that allows examples to be grouped in the computation of
the update to the model.actual
- The value of the target variable. This value should be in the half-open
interval [0..n) where n is the number of target categories.instance
- The feature vector for this example.public int nextStep(int recordNumber)
public static int stepSize(int recordNumber, double multiplier)
public void close()
OnlineLearner
close
in interface Closeable
close
in interface AutoCloseable
close
in interface OnlineLearner
public void setInterval(int interval)
interval
- Number of training examples to use in each epoch of optimization.public void setInterval(int minInterval, int maxInterval)
minInterval
- The minimum epoch length for the evolutionary optimizationmaxInterval
- The maximum epoch lengthpublic final void setPoolSize(int poolSize)
public void setThreadCount(int threadCount)
public void setAucEvaluator(OnlineAuc auc)
public int numFeatures()
public double auc()
public State<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> getBest()
public void setBest(State<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> best)
public int getRecord()
public void setRecord(int record)
public int getMinInterval()
public int getMaxInterval()
public int getNumCategories()
public PriorFunction getPrior()
public void setBuffer(List<AdaptiveLogisticRegression.TrainingExample> buffer)
public List<AdaptiveLogisticRegression.TrainingExample> getBuffer()
public EvolutionaryProcess<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> getEp()
public void setEp(EvolutionaryProcess<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> ep)
public State<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> getSeed()
public void setSeed(State<AdaptiveLogisticRegression.Wrapper,CrossFoldLearner> seed)
public int getNumFeatures()
public void setAveragingWindow(int averagingWindow)
public void setFreezeSurvivors(boolean freezeSurvivors)
public void write(DataOutput out) throws IOException
write
in interface org.apache.hadoop.io.Writable
IOException
public void readFields(DataInput in) throws IOException
readFields
in interface org.apache.hadoop.io.Writable
IOException
Copyright © 2008–2017 The Apache Software Foundation. All rights reserved.