Evaluation metrics matter: predicting sentiment from financial news headlines

Andrew Moore

SCC, Lancaster University

The SemEval Task 5 track 2 competition this year was to predict the sentiment with respect to a company in financial news headlines. We came 5th out of 45 competitors using a Bi-Directional Long Short-Term Memory (LSTM) deep learning model. We describe this approach and how we practically implemented it and the other methods we attempted. We describe the experience of participating in the event and how the evaluation metric used to evaluate the models should reflect the real world problem being solved. This talk we hope will be interesting to all wanting to see how easy it is now to implement deep learning models and how we can learn from others mistakes with regards to setting up an evaluation task to solve a real world problem.

Week 19 2016/2017

Thursday 16th March 2017

Management school LT9