A new tool that uses artificial intelligence to label people’s photos is showing how weird and biased AI can be.
The ImageNet Roulette website created by AI Now Institute co-founder Kate Crawford and artist Trevor Paglen has taken Twitter by storm since its launch on Monday. The experiment by the two is set to showcase the dangers of using datasets with inherent biases to train AI models.
SponsoredAny AI-based system needs to feed off data. ImageNet Roulette’s AI was trained on ImageNet, a database compiled in 2009 of 14 million labeled images.
ImageNet is one of the most comprehensive training datasets in the field, in part because it’s free and available to anyone. The creators of ImageNet Roulette trained their AI model on 2833 sub-categories of “person” found in the ImageNet dataset.
When a user uploads a picture, the application first runs a face detector to locate any faces. If it finds any, it sends them to the model for classification. The application then returns the original images with a bounding box showing the detected face and the label the classifier has assigned to the image.
Classification results range from accurate and boring to funny and outrageous, which is why it caught so much attention on social media throughout the week.
Sponsored Sponsored
AI Systems Have Challenges to Overcome
The main objective of Crawford and Paglen is to show how miss-fitted data used to train AI algorithms can produce weird and undesired outcomes. ImageNet Roulette is currently on display as part of an exhibition in Milan.
Want to see how an AI trained on ImageNet will classify you? Try ImageNet Roulette, based on ImageNet's Person classes. It's part of the 'Training Humans' exhibition by @trevorpaglen & me – on the history & politics of training sets. Full project out soonhttps://t.co/XWaVxx8DMC pic.twitter.com/paAywgpEo4
— Kate Crawford (@katecrawford) September 16, 2019
The creators highlight how — simply by introducing a number of offensive categories, like misogynistic or racist terminology — the AI model will start delivering likewise results.
SponsoredOn the one hand, it sheds a light on how problematic AI systems can be with fundamentally bad data. On the other, it also means that AI systems could be manipulated by feeding them altered data to get the desired outcome.
Crawford tweeted that, although ImageNet was a major achievement for artificial intelligence by being such a huge database, the project revealed fundamental problems with bias when it comes to classifying human beings.
While ImageNet Roulette is a fun and gimmicky tool, it should raise some questions on how well these systems can be effectively used in real-world scenarios.
Did you try the ImageNET Roulette? Let us know what you think of the experiment and share your results!
Image courtesy of Twitter/@katecrawford, Shutterstock.
