Week 3: Experiments with DNNs and Logistic Regression

This week we put together the code to generate training data out of phrase files using the information such as number of phonemes and booleans as per the number of neighboring phonemes in the phrase. These features were used to learn the intelligibility score of every word in a phrase using DNN and Logistic Regression.

We started the discussion on Gitter regarding the approach to be followed to overcome the issues noticed while analyzing DNN results.

Here's the distribution of samples across both classes for a single phrase "We drank tea in the afternoon and watched TV"

We performed several rounds of softmax classification. Here's the initial results shared by James:

Using Logistic Regression:

Using DNNs (without dropout regularization):

We applied dropout regularization and it improved DNN results further:

DNN with 50% units dropped from intermediate layers during training (still using softmax activation across the network) 

 DNN with 50% units dropped from intermediate layers and ReLU used as activation function after input and intermediate layers.

Here's James' version with 0.3 dropout across the network  

In the coming week we will work on the following items:

1. Deal with the narrow range of DNN probability for some words

2. Download the audio files from the CSV files provided by James and process them as per the assignment (further details in next blog).