Image for post
Image for post

Resources frame machine learning as something complex and something fancy and something futuristic and exciting. Instead, it is rather kind of depressing. Think of machine learning as something not fancy at all — and something that only requires spatial reasoning.

Let’s begin with some anonymous definitions of machine learning.

First Definition:

Machine learning is an application of artificial intelligence that provides systems with the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it to learn for themselves.

The process of learning begins with observations or data such as examples direct experience or instruction. In order to look for patterns in data and make better decisions in the future based on the examples that we provide the primary aim is to allow the computers to learn automatically without human intervention or assistance and adjusts actions accordingly. …

Using TensorFlow

Aim: It is important to know the effect of loss functions in algorithm convergence. Here we will illustrate how the L1 and L2 loss functions affect convergence in linear regression.

Getting ready

We will use the iris dataset, that is built into the Scikit Learn. Specifically, we will find an optimal line through data points where the x-value is the petal width and the y-value is the sepal length. But we will change our loss functions and learning rates to see how convergence changes.

How to do it…

  1. We load the necessary libraries, start a session, load the data, create placeholders, and define our variables and model. One thing to note is that we are pulling out our learning rate and model iterations. …

Using TensorFlow

Getting ready

We will loop through batches of data points and let TensorFlow update the slope and y-intercept.

We will implement a matrix decomposition method for linear regression.
Specifically, we will use the Cholesky decomposition, for which relevant functions exist in TensorFlow.

What is the Cholesky decomposition?

The Cholesky decomposition or Cholesky factorization is a decomposition of a Hermitian, positive-definite matrix into the product of a lower triangular matrix and its conjugate transpose.

Getting ready…

Implementing inverse methods in the previous blog (Linear Regression with TensorFlow)can be numerically unfit in most cases, especially when the matrices get very large. Another approach is to decompose the A matrix and perform matrix operations on the decompositions instead.

Using the Matrix Inverse Method

Linear regression may be one of the most important algorithms in statistics, machine learning, and science in general. It’s one of the most used algorithms and it is very important to understand how to implement it and its various flavours. One of the advantages that linear regression has over many other algorithms is that it is very interpretable.

Using the Matrix Inverse Method

we will use TensorFlow to solve two-dimensional linear regressions with the
matrix inverse method.

Getting ready

Linear regression can be represented as a set of matrix equations, say Ax =B. Here we are interested in solving the coefficients in matrix x. …

Combining Everything Together

Create a classifier on the iris dataset.

Getting ready

The iris data set is described in more detail in Working with Data Sources in Getting Started with TensorFlow. We will load this data, and do a simple binary classifier to predict whether a flower is a species Iris setosa or not. To be clear, this dataset has three classes of species, but we will only predict whether it is a single species (I. setosa) or not, giving us a binary classifier. We will start by loading the libraries and data, then transform the target accordingly.

Source: TensorFlow Machine Learning Cookbook

How to do it…

  1. First, we load the libraries needed and initialize the computational graph. Note that we also load matplotlib here, because we would like to plot the resulting line…

TensorFlow updates our model variables according to the prior described backpropagation, it can operate on anywhere from one datum observation to a large group of data at once. Operating on one training example can make for a very erratic learning process while using a too large batch can be computationally expensive. Choosing the right type of training is crucial to getting our machine learning algorithms to converge to a solution.

Getting ready…

  1. In order for TensorFlow to compute the variable gradients for backpropagation to work, we have to measure the loss on a sample or multiple samples.
  2. Stochastic training is only putting through one randomly sampled data-target pair at a time. …

Implementing Back Propagation

One of the benefits of using TensorFlow is that it can keep track of operations and automatically update model variables based on backpropagation. We will look into how to use this aspect to our advantage when training machine learning models.

Getting ready

Now we will introduce how to change our variables in the model in such a way that a loss function is minimized. We have learned about how to use objects and operations, and create loss functions that will measure the distance between our predictions and targets. Now we just have to tell TensorFlow how to backpropagate errors through our computational
graph to update the variables and minimize the loss function. …

Loss Functions

Implementing Loss Functions

Loss functions are very important to machine learning algorithms. They measure the distance between the model outputs and the target (truth) values.

Getting ready…

In order to optimize our machine learning algorithms, we will need to evaluate the outcomes. Evaluating outcomes in TensorFlow depends on specifying a loss function. A loss function tells TensorFlow how good or bad the predictions are compared to the desired result. In most cases, we will have a set of data and a target on which to train our algorithm. The loss function compares the target to the prediction and gives a numerical distance between the two.

How to do…

Key components of how TensorFlow operates

we will introduce the key components of how TensorFlow operates. Then we
will tie it together to create a simple classifier and evaluate the outcomes. We will be learning about the following:

  1. Operations in a Computational Graph
  2. Layering Nested Operations
  3. Working with Multiple Layers
  4. Implementing Loss Functions
  5. Implementing Back Propagation
  6. Working with Batch and Stochastic Training
  7. Combining Everything Together
  8. Evaluating Models

In this article, we will cover the first 3 points.

In the previous article, we learned how TensorFlow creates tensors, uses variables and placeholders, we will introduce how to act on these objects in a computational graph. …


Bhanu Soni

Data Scientist at NCS-IT, UK

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store