Skip to content

LogisticRegressor

Jump to bottom
Nicolas Raymond edited this page Oct 29, 2020 · 2 revisions

LearningModels.LogisticRegressor

Logistic regression model that proceeds to sequential learning via gradient descent.
The model can be found in Bishop, Pattern Recognition and Machine Learning, p.205-206.

LogisticReg

The letter sigma stands for the sigmoid function.

Constructor

LogisticRegressor(phi, eta0=1, learning_rate='adaptive')
  • phi : basis function applied to inputs' original features vectors.
  • eta0 : initial learning rate value at the start of the training.
  • learning_rate: learning rate schedule ('constant', 'adaptive' or 'invscaling').

Public methods

  1. train
  2. reset_weights
  3. loss
  4. copy
  5. predict
  6. plot_model
!Note that L is the original number of features and M is the number of features after transformation with phi.

train

train(self, X, t, nb_epoch, minibatch_size=1, weight_init=None)

Trains the logistic regressor according to the observations in the dataset X and their respective targets in t.
The weights are updated sequentially using the gradient of the cross-entropy loss.

CrossEntropy

  • X: N x L numpy array with training observations.
  • t: N x 1 numpy array with training labels.
  • minibatch_size: size of minibatches used is gradient descent.
  • weight_init: starting point of SGD in weight space, if weight_init=None, each of them are sampled from a standard normal distribution.
  • return: updated weight vector (M x 1 numpy array).

reset_weights

reset_weights()

Resets the weights of the model randomly by sampling each of them from a standard normal distribution.

loss

loss(X, t, return_predictions=False)

Computes the cross-entropy loss associated with our current model according to the observations in X and their respective labels in t.

  • X: N x L numpy array with training observations.
  • t: N x 1 numpy array with training labels.
  • return_predictions: bool that indicates if the numpy array of predictions is returned.
  • returns: loss (float), predictions (numpy array).

copy

copy()

Creates a deep copy of the logistic regression model.

predict

predict(x)

Predicts the probability of an input x to have label 1.

  • x: 1 x L numpy array.
  • return: predicted probability (float between 0 and 1 included).

plot_model

plot_model(X, t, title=None, save=False, save_path='', filename='model', save_format='.pdf')

Plots the the prediction curve of our model (only available with 1-D or 2-D feature space).
The x-axis labels will be associated to w_t_phi.

  • X: N x 1 numpy array with training observations.
  • t: N x 1 numpy array with training labels
  • title: title of the figure
  • save: bool indicating if we want to save picture or not
  • save_path: path indicating where we save the file if it is saved
  • filename: name of the file if it is saved
  • save_format: saving format

Clone this wiki locally