Project Timeline

How we managed to create this AI product in three weeks:

July 6 - July 7

Drafting Project Ideas

At AI Camp, we were given the freedom to choose our own project related to computer vision and object detection. As a team, we brainstormed ideas ranging from a dog species detector to a social distancing detector. We wanted to choose a product that was both impactful and feasible given the time constraints. Ultimately, we decided to create a model that could detect and identify letters from the American Sign Language alphabet.

July 7

Collecting Data

Great machine learning projects rely on great data. We began our project by searching online for data. We came across an American Sign Language dataset on Kaggle and thought that would be good start!

July 8 - July 10

Labeling Data

We used Labelbox to label the photos in our ASL dataset. The labels are useful because they specify the part of the image that contains the ASL and identify the letter. Initially, we labeled 8,100 images (300 for each letter and 300 for the 'space' character).

July 13 - July 14

Training & Evaluating Model

We then prepped for the training by downloading the labeled images, converting them into the YOLO format, configuring YOLO, and installing darknet. We split our labeled data into train, validation, and test sets and trained our model using YOLO. We evaluated our model and found that it performed well on images from the Kaggle dataset; however, it performed poorly on most other images. To make our model more generalizable, we decided to go back and collect more, diverse data. 

July 15 - July 17

Collecting More Data & Retraining the Model

To improve the performance of our model, we collected more images from more online datasets, Google Images, and we even uploaded our own photos of ourselves signing! We labeled these images, resplit our data, and retrained our model. We found that the model performed much better after retraining.

July 20 - July 23

Deploying Model

After creating our model, we then deployed it onto this website for the public to use. We have thought of some potential nexts steps for our project. It would be awesome to add live video compatibility for our ASL detection model - that way, users could translate in real-time! Furthermore, we could expand beyond letters, adding words and phrases to create a more robust and useful translator.