The Subtle Art of Fooling AI

Explore How Small Image Edits Can Outsmart AI

TL;DR

When trying to trick models with slight changes to images, scattered changes are harder for most models to detect compared to grouped ones. However, models using certain image processing methods are more susceptible to grouped changes in rectangular patches.

Detailed explanation

Imagine you have a coloring book and some invisible ink. You want to see if anyone would notice if you made tiny changes to the pictures. You have two ways to do this: You can sprinkle little dots of ink all over the picture, or you can color in little shapes like a small line or square. The goal is to see which way is harder for people (or in our case, AI models) to catch.

The Problem

AI models, like those that recognize faces or read text from images, can sometimes be fooled by making tiny changes to the images they see. Scientists wanted to find out: Is it harder for the AI if the changes are dotted all over (sparse) or grouped together in shapes (contiguous)?

The Study: Sparse vs. Contiguous Changes

The scientists experimented with two types of tricks – sparse and contiguous:

Sparse Changes: It’s like sprinkling tiny dots randomly across the picture. These dots are hard to spot because they are spread out. Contiguous Changes: These changes are made by coloring small shapes like a row, column, or rectangle (patch). They stand out more because all the changes are close to each other.

What They Found

AI models that use CNN (Convolutional Neural Network) to process images are more easily tricked by contiguous shapes like patches. Think of it like coloring a small square in the invisible ink—it’s harder for the AI to recognize the changes if they’re shaped like this.

AI models that use transformers (ViTs) to process images are tricked by sprinkling tiny dots in different places because they process different parts of the image separately.

Why It Matters

These findings are important for improving AI in activities like driving cars, security systems that recognize faces, and even in apps that help doctors read medical images. By knowing these tricks, developers can make AI more robust and accurate.

So, in short, it’s all about where and how you make those tiny changes in the picture to outsmart the AI. The dotted changes or grouped shapes can each be a superpower, depending on the type of AI model you’re up against!

Read the full research on arXiv