Week 4 Learning

• Machine Learning: computers use data to preform tasks with no explicit instructions
• Supervised ML:

  Provides computer with input-output pairs, labelled data. Computer expected to grow capable of mapping inputs to outputs.
• Classification:

  Computer grouping input into categories.

  
• Computers are capable of understanding multiple dimensions in a plotted graph
• Nearest Neighbour Classification:

  Algorithm selects nearest labelled group to assign to new input point

  K-nearest Neighbour Classification:

  variation that considers most common class of k nearest data points, to increase accuracy

  
• Perceptron Learning:

  uses linear regression to draw a boundary line on plotted data, with the aim to divide as evenly as possible
• Perceptron Learning Rule:

  given data point (input x, output y), update each weight according to:

  
• perceptron rule uses a hard threshold; logistic regression uses a soft threshold

  
• support vector machines can also handle non-linearly separable data

  
• Support Vector Machine:

  attempts to find maximum margin separator

  
• Regression:

  supervised learning problem, mapping inputs to outputs of a continuous value
• Hypothesis Evaluation:

  attempts to minimise the loss function

  there are different types of functions that can be used
• 0-1 Loss Function:

  commonly used in discrete classification

  L(actual, predicted) = 0 if predicted is correct, else 1
• L1 Loss Function:

  L(actual, predicted) = |actual - predicted|

  takes into account difference so predicted does not need to be exactly the same as actual

  L2 squares this, to be more strict

  
• Overfitting:

  fitting a model too closely to a specific set of data and thus failing to generalise (for new input)
• Regulization:

  cost(h) = loss(h) + y complexity (h)

  [can chang value of y, depending on how much complexity should be penalised]
• Holdout Cross-Validation:

  Separate data into a training set to learn from and test set to evaluate

  k-fold cross-validation:

  prevent reduction in data being used to learn from, data is split into k sets to experiment on k times (meaning each set is used as training set once)
• Scikit-Learn:

  popular python library for working with ML algorithms
• Reinforcement Learning:

  agent provided with numerical 'rewards'/'punishments' to learn how to complete a task
• Markov Decision Process:

  decision-making model to represent states, actions and rewards

  
• Q-Learning:

  method for learning function Q(s,a) to estimate value of an action within a state (by considering rewards/punishments)
• Agent 'explores' to gain knowledge and can 'exploit' previous knowledge, when repeating

  Problem is that agent may exploit a path it knows eventually leads to goal, without exploring alternatives (thus possibly missing optimal path)
• ε-Greedy ("epsilon greedy"):

  aims to prevent loss of optimal path due to exploitation, by exploring a random move a select frequency

  P(1 - ε) chose best move

  P(ε) chose random move
• Unsupervised Learning:

  provides unlabelled input data
• Clustering:

  grouping similar objects
• k-means Clustering:

  dividing into k clusters by originally assigning cluster centres randomly then repeatedly moving to the average data point within each cluster