How Easy It Is to Trick AI? Playing with Image Recognition Models via Teachable Machine!

December 16, 2022

In a study called "Women Also Snowboard: Overcoming Bias in Captioning Models", the authors Hendricks et al. describe their shocking discoveries as automatic image captioning models were more likely to generate captions including "he" or "him" with pictures of a person snowboarding even the picture contains a woman (1). Another study titled "Predictive Inequity in Object Detection" by Wilson et al. also portrays horrific results where object detection algorithms used in surveillance systems were more likely to accurately detect white people more than people of color (2). It is clear that AI can be wrong and hurtful toward certain races and genders much more than you think!

To drive the point home, let's try to experience how faulty AI can be by ourselves! Recently, I came across Teachable Machine while I was learning about algorithmic bias in class. This website is a part of Google Creative Lab, which Google describes the project as "a web tool that makes it fast and easy to create machine learning models for your projects, no coding required" (3). With this website, I have created some models that I will use to demonstrate how easy it is to trick an image recognition machine learning model. You can also access the Teachable Machine website here.

The first model is an image recognition model to classify images of people with black and white hair. For this model, I trained the model using 20 pictures of people with black hair and 20 pictures of people with white hair. Here, I also demonstrate two ways that I "tricked" the model to give inaccurate results!

Can you think of another way that we can trick the model? Where do you think the model can improve to account for these inputs to make sure that AI will give the most accurate results? (Maybe the problem might also come from the way the model was trained?).

The second model is an image recognition model to classify images of people wearing and not wearing a mask. The training method for this model is similar to the first one. Here, I also show two ways that I "tricked" my model. Try to recreate them!

* For this model, please access the model with this link. The button below does not work due to technical difficulties. *

Can you think of another way that we can trick the model? Where do you think the model can improve to account for this input to make sure that AI will give the most accurate results? (Maybe the problem might also come from the way the model was trained?).

There is an interactive game that portrays this similar idea called "Facework". The author Kyle Mcdonald described this game to be "a world where face analysis is key to the latest gig economy app" and he hoped to "playfully grow an intuition for what it means to see like a machine, and to understand how machines can fail" (4).

References

1. Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018). Women also snowboard: Overcoming bias in captioning models. Computer Vision – ECCV 2018, 793–811. https://doi.org/10.1007/978-3-030-01219-9_47
2. Wilson, B., Hoffman, J., & Morgenstern, J. (2019). Predictive Inequity in Object Detection. ArXiv. https://doi.org/10.48550/ARXIV.1902.11097
3. Google. (n.d.). All experiments - experiments with google. Google. Retrieved December 16, 2022, from https://experiments.withgoogle.com/experiments?tag=Teachable%2BMachine
4. Mcdonald, K. (n.d.). Kyle Mcdonald. Kyle McDonald. Retrieved December 16, 2022, from http://kylemcdonald.net/