Art or Not

Developing an Artificial Intelligence Art Critic

In 2016, a pair of glasses were made famous at the San Francisco Museum of Modern Art. Left on the floor by a teenage prankster, many museum-goers believed they were art and, whilst maintaining a safe distance from the ‘artwork’, began taking pictures on their phones. If we can be fooled that easily, how can you tell whether something really is art or not? This, and similar dilemmas prompted researchers at SensiLab to develop an “independent arbiter” to help settle the “art or not?” question.

Let the Machines Decide

Art or Not is a personal AI-powered art critic that helps you figure out whether things around you are art. You simply open the app, take a picture of the possible artwork, and get its immediate opinion on whether “A kid could do that!” or it’s “Totally Art”, and many other opinions in-between. Additionally, Art or Not also finds artworks that are most visually similar to the image you’ve just snapped and shows you those works along with links to the museum or gallery collection that holds them. You can also easily share your discoveries on social media.

Critical Decisions

The app doesn’t have a philosophical understanding of what Art is, rather it relies purely on visual information to make its determination. Whereas a content-based search – Googling ‘apple’ or browsing Related Pins on Pinterest – can give you images with similar metadata, the app uses a neural network to break down images into a thousand visual features and compare them to images it knows are of art.

To achieve this, we trained a convolutional neural network on over 100,000 artworks, including sculptures, paintings, art objects and photographs — drawn from museum collections in the public domain — to give the network a visual understanding of what ‘Art’ looks like. When a new image is captured, the app uses a technique called “Visual Similarity Search” to figure out how similar the captured image is to other artwork images it has ‘seen’ before. If the captured image shares enough characteristics of known art, it is deemed “art” by the app.

It is important to point out that to be judged “Art” by the app, the image in question does not have to be of any known art image the neural network has seen before. So, for example, you can create your own pencil doodle and ask the app to see if it’s art, or not.

Art Beyond Gallery Walls

Along with being a helpful critic, the app is also a very useful companion as it can pull up ‘similar’ looking artworks to things you capture. Taking a photo of a person can pull up renaissance portraits, capturing a cup can show you pictures of historic sculptures — it lets you use the world around you as a way of discovering art.

We have been surprised by how the app can be used to settle arguments about a new artwork at the local gallery and also to inspire people to think more critically about how they approach the experience and appreciation of art.

The next step in the project might be to offer a more detailed critique of the work, for example using a modified version of GPT2 trained on a corpus of art criticism text. The app might also be able to provide more explanation behind its decision, helping budding artists to improve their work, even offering advice on what changes to make.

Art or Not is available from the Apple App Store for free. It currently only runs on iOS devices.