Our engineers are always keeping up with the latest technology trends to see if there are ways to apply them within the museum. One of the most promising lately has been image detection technologies, which have grown a lot in recent years. Amazon, Google, and Microsoft currently all provide services where you can upload an image, and they’ll suggest words of things that can be found in the image—like “suit,” “jar,” or “children.” This works really well with crisp photographs, but not so well with paintings and sculptures.
In thinking about how we could use image detection effectively, we started by looking for a way to identify the dominant colors (the ones featured most prominently) in our artworks, a tool we’ve seen a few of our peers introduce. After lots of tinkering, we were able to programmatically determine dominant colors for every object in our collection that’s published to the web. “Programatically” means no one had to look at any images and tag them with colors but that code was written to do this work. That’s a big deal, because it means as new objects are published or rephotographed, we can automatically keep this data up-to-date. Our algorithm constantly polls the collection looking for new and updated images and determines the dominant colors of those works.
With the data in place, we then focused on how to get this tool on the website and create the best user experience. Our design team came up with a simple interface and iterated over our engineering team’s implementation until it felt just right, making sure it worked well on both desktop and mobile.
We also looked at how results are filtered and decided that the best approach was to sort the results by how saturated a work is with that color. More saturated works will appear first, trickling down to works that match, but are less saturated. That makes for a really satisfying experience where you first see works that are really the color you chose.
We then had a ton of fun conducting usability tests with colleagues around the museum to identify tweaks we could make to improve the experience. People were so excited about the tool! It uncovers works deep in our collection that you may not see any other way—across departments, time, and media. It’s a fun way to dig into our collection.
Here are some color searches that the museum staff members have enjoyed:
“I chose this blue because it reminds me of the electric blue in Hélio Oiticica’s prints, a piercing blue that can’t quite be reproduced in print or online. In my #444fe7 search I found this color in blue glass paperweights and a Qing Dynasty robe. It also turned up in Marc Chagall’s America Windows (1977) and a beautiful watercolor by Gladys Nilsson, The Great War of the Wonder Women. I love encountering this deep, electric blue in works when I see them in storage or in the galleries, and it was great to be able to pull them all together and see how a single color reappears in different places and times in our collection.”
—Lauren Makholm, assistant director of production, Publishing
“My kid is a new reader so searching the site by color opens the collection up in a way that he can access. Yellow is my kid’s personal favorite, and part of my job is working with gold, which is another reason for the color choice. This would be a fun way for my kid and I to discover art and artist we don’t know in our collection before a visit. I would love see the Neil Winokur photographs. I also see many paintings that I’ve done work on like David Wojnarowicz’s Queer Basher/ Icarus Falling and Joseph Albers’s Homage to the Square: Yellow Signal.”
—Christopher Brooks, conservation technician, Frame Conservation
“I chose this pink shade to search our collection. I like the color search because it lets me explore across the relatively silo-ed curatorial departments—the many ways and media in which this color is interpreted or used as a visual tool give more context to the pieces and to my cultural expectations and experiences of the color. Visually, it’s also deeply satisfying and strangely calming to me to see the variations of pink, the contrasting and complementing colors alongside it.”
—Sarah Gray, associate director of Member and Donor Services
“This saturated color—orangey-red or reddish-orange (and some of its pinkish varieties)—has been my go-to color since college, at least. I take pictures of flowers of this color when I travel, I happily discover it in the kitchens of my friends, and I love it for sweaters or t-shirts or bathing suits or dresses. The works that first come up on the Art Institute’s search are highly graphic, eye-catching posters by designer John Massey, a painting by Howard Hodgkins and one by Hans Hoffman, a work by Jenny Holzer, a poster by Allen Ruppersberg, and a drawing by Juan Muñoz, among others—all modern and contemporary artists who know(knew) the power of communicating at a distance and who, in the case of Hoffman and Hodgkin, love(d) saturated color.”
—Jacqueline Terrassa, Woman’s Board Endowed Chair of Learning and Public Engagement
What are you waiting for? Go to our Collection page, click on “Show filters,” select color, and spin away!
—nikhil trivedi, web architect, Experience Design
