Machine Learning

Image Captioning

A CNN and LSTM network that generates natural-language captions for images, trained on Flickr8k.

A football game photo with the generated caption and reference captions above it

A golden retriever running on grass with the generated caption and references

A snowboarder mid-jump with the generated caption and references

Diagram of a pretrained CNN feeding image features into an LSTM caption generator

1 / 4

As part of CSE 190: Neural Networks at UCSD, we built a deep neural network that automatically captions images.

A convolutional neural network (CNN) extracts features from the image, which are passed to a recurrent neural network (RNN) that generates the caption. The network was trained on the Flickr8k dataset.