Machine Learning in the browser: Softmax Regression

Wednesday, July 17th, 2019

Machine learning has enabled developers to build adaptive software applications which don't require custom engineered code. In web browsers machine learning algorithms can help to enable offline JavaScript applications that otherwise need to call server API for machine learning. This can also be used in offline cross-platform JavaScript mobile apps and with desktop(Electron) applications which use Chrome as container. In this blog post I have discussed the use of a supervised learning algorithm known as softmax regression. Developers can use this to have multi-class classification given training data.

Problem Statement

Consider a reminder application that reminds users of certain task and users can choose to snooze(for 2 min), accept, reschedule and cancel the task. Each task can be grouped into categories of tasks like office, health, food ...etc. So the logs from the reminder application can be used to model if the user is likely to follow through the reminder. We can use X as category of reminder with reminder time then use Y to represent either category using one-hot encoded representation. For this exercise we only consider the log data and the UI can have any implementation as desired.

Fig 1.0

Figure 1.0 shows how the features will be used from the reminder application log data.

In this blog post we assume that the data is already available from a reminder application so we are not concerned with how we get it but what is done with the log data.

Step 1: Transforming data

If the log data is already present then it might be in the form of array of JSON objects as follows:

[{
  reminderTimeStamp: 1519211809934,
  reminderCategory: 'office',
  finalAction: 'snooze',
  reminderText: 'Meeting with QA',
}, {
  reminderTimeStamp: 1553018400000,
  reminderCategory: 'health',
  finalAction: 'accept',
  reminderText: 'Gym at 6:00 pm',
}]

Since the log data is verbose compared to the features required for our model so we will only pick certain properties from the data. The day of the week will be extracted from the timestamp. So if the timestamp is 1519211809934 then the day is Wednesday which translates to third day of the week '0010000'. In this case we assume that there are only 4 categories which are food, health, office and social. These categories will be stored in an array so food will have index of 0, health will have index 1, office will have index 2 and social will have index 3. This way category like food can be represented as one-hot encoded vector like '1000', which has 1 in the 0th index. Similarly there are only 4 actions which are snooze, cancel, reschedule and accept. These actions will be stored in an array as well so each action can be converted to a one-hot encoded vector. For instance 'accept' can be converted into '0001' where it has 1 on the 3rd index.

The transformed data for the above data sample looks like this:

let categories = ['food', 'health', 'office', 'social'];
let actions = ['snooze', 'cancel', 'reschedule', 'accept'];

let x = [[0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0],
	 [0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0];
let y = [[1,0,0,0],
	 [0,0,0,1]];

This is just a snippet of the overall data. You can download the data from github

Step 2: Setting up the Algorithm

For this blogpost we will use machine learning library from intellingine.com which I implemented but any popular library can be used for the task. Firstly the softmaxregression folder from the library should be placed in the project directory under libs(it could be any other folder). Then the required JavaScript files can be imported as below:

<!--index.html-->
<!doctype html>
<html>
<head>
</head>
<body>
<script src="./libs/softmaxression/lib/q.js"></script>
<script src="./libs/softmaxregression/lib/math.js"></script>
<script src="./libs/softmaxregression/softmax.js"></script>
<!--Include the main.js file where you use the algorithm.-->
<script src="main.js">
</script>
</body>
</html>

After setting up files the algorithm can be setup before any data is trained. Hyperparameters for the algorithm can be set like momentum, learning rate, batch size, regularization, weight initialization range...etc. To instantiate the algorithm use the syntax as below:

  let sft = new window.SoftmaxRegression({
        'notify_count': 100,
        'momentum': 0.5,
        'parameter_size': [11, 4], //[number of input features, total number of output classes]
        'max_epochs': 100,
        'weight_initialization_range': [Math.random()*-1, Math.random()],
        'threshold': 0.1,
        'batch_size': 64,
        'iteration_callback': callback,
        'learningRate': 0.0009,
        'regularization_parameter': Math.exp(-5)
    });

Step 3: Training

After the data is transformed to one-hot encoded matrices it can be used to train the algorithm using the syntax below:

    sft.startRegression(matX, matY);

After training completes it will save the weights and biases to local storage. Which can be used later.

Step 4: Prediction

Prediction is simple and uses the syntax as below:

let prediction = sft.predict(matX);

After the prediction is successful it returns a promise with the softmax normalized array. It is evident that the 2nd index is the highest so the one-hot encoded array will be [0,0,1,0]. This means that it will be rescheduled.

You can find the code for this demo on github.