Explainer: AI in Vernon Browser

Artificial Intelligence (AI) is a rapidly emerging technology. We’ve used AI in Vernon Browser since 2018 to automate our image analysis. This article gives some examples of how AI is used in the museum sector and in Vernon Browser.

Repairing photographic images

Image analysis is particularly powerful with AI as it’s very good at detecting patterns.

You can use AI to repair images. Within the museum industry, you need to be very careful to clearly note any images which have been augmented with AI.

So you might use AI in a very specific context, such as an exhibition where you show an original photograph alongside a repaired image to allow the visitor to get a clearer view.

Predicting events

AI can also be used to predict things that are going to happen.

Dexibit is an Auckland software company that gathers information on visitors and combines that with other resources like weather forecasts to make predictions.

One example is how typically rainy days or public holidays may affect your visitor numbers. That may allow you to plan how many staff you need to have available on a particular day.

Creating content

You can also use AI in content creation. ChatGPT is one example of a tool which has very powerful capabilities for generating content based on its existing set of knowledge.

For example, you could ask ChatGPT to give you examples of games that teenagers might play in a museum. It will generate a whole set of ideas which you could then use as a starting point to design some games for teenagers in your own museum.

Chatbots

AI is also used in conversational contexts. Chatbots on websites can provide answers to common questions such as, what are your opening hours?

Chatbot technology can also be used for more complex activities. The British Museum Chatbot Guide has been built by an independent developer accessing public resources that are available from the British Museum.

Here’s an example chat dialogue where somebody’s asked questions about one of the Lewis Chessmen.

Image analysis: patterns, keywords, and colours in Vernon Browser

Vernon Browser uses an AI-powered image recognition product called Google Cloud Vision to analyse images for patterns and subjects

For example, the Art Gallery of Western Australia has images that Vernon Browser has tagged with the keyword ‘circle’.

Vernon Browser can also add subject keywords that describe what’s actually depicted in an artwork.

Here’s another example from the Art Gallery of Western Australia, Landscape with farm by Charles Ginner. Since ‘Landscape’ is in the title, we could have easily found that by searching, but that may not always be the case.

If we scroll down, we can see in the subject auto tags that the AI thinks this is a landscape in a rural area containing a field and a hill.

By clicking on the tag Natural Landscape, we can find a whole lot of different artworks, many of which don’t have any keyword in the title or object description to say that they’re a landscape.

For example, the first record in the results shows that Naples is the landscape depicted, but we couldn’t have found this artwork without AI adding that subject tag.

Vernon Browser also analyses colours. So you can browse your collection by particular colours that group records together. As well as providing colour swatches as a navigation aid on the website, Vernon Browser also indexes the precise colour name and colour group as text keywords. For example, Khaki and Brown. These keywords help visitors find the objects they’re searching when using keyword searches.

Here’s one of our US sites, the Farnsworth Art Museum, looking at a selection of their artworks with a particular shade of blue.

Biases in AI

However, there are biases within AI and we have to be aware of that whenever we’re using these tools. Those biases can come primarily from the content which is teaching the AI how to match up things, how to find patterns, and so on. The content we’re feeding to AI may already be a biased subset. Biases can also come in the design and development.

Depending on who’s building those tools, they already may have particular biases about how they interpret the world and interpret the content.

Bias in AI: What it is, Types, Examples & 6 Ways to Fix it in 2024 (AI Multiple Research website)

Administration tools in Vernon Browser

To cope with those biases, we’ve built administration tools in Vernon Browser. Let’s look at examples of how we can modify keywords and colours.

Editing subject keywords

Within our demo website, we have a fish specimen record that’s been kindly given to us by Queensland Museum.

And we can see that the AI has added subject keywords for that record.

So the AI has added tags for Organism, Fish, and Ruler.

Most of these natural history records have a ruler for scale in the image, which isn’t necessarily useful for the public user. So we might decide that keyword isn’t relevant for this record or possibly for our whole website.

From the dashboard of the Vernon Browser administration site, you can browse your records by the image auto tags.

And we’ll see that one of the most common tags on the entire site is Ruler. That’s because every one of those natural science images includes a ruler.

We can select the Ruler tag to see the records that include it.

You can delete a tag on a particular record or across the whole site by selecting one of the records.

We can also see how confident Google Cloud Vision is about the accuracy of the subject keyword. So that percentage is a confidence level that Google has returned to us.

We can click the Edit link next to those auto tags to manage them.

Previously there was a tag for Office ruler which has been deleted. We might also want to delete Ruler, either for the current record or from all records.

Editing colour analysis

Vernon Browser’s colour analysis looks at the broadest parts of an image that have the same colour. However, that may not match human perception.

In this Pacific Holiday artwork by Rodney Fumpston, we can see there’s some very large blocks of colour, plus a very small area of bright pink. Although that area of pink is small, it’s very recognisable to the human eye.

However, the colours that have been detected don’t include that bright pink. And the last of those colours, Peru Brown, has been detected interspersed in the grain of the pink background.

So we could modify those swatches. We can log into Vernon Browser and find that record from the Browse Data section.

And then just like we did with the image auto tags, we can edit the colours that it’s detected.

Of course, this is something you’d only do in an exceptional case. You wouldn’t normally have the resources to be checking even a small fraction of these records. But you may have an iconic picture where you really want that to be found by a particular colour element that’s not dominant but is very recognisable.

So these are the four colours we have. Next to each colour is an option to delete it. So we’ll delete the Peru Brown colour.

We can also add colours. So we can manually add the shade of Hot Pink so that people can find the record using that swatch.

We can use the Colour Presence slider to give a rough estimate of how much of the image has that particular colour. So let’s say 6% has that colour.

The colour swatches are updated and available for users to browse.

So that gives you a broad overview of the two areas where we’re using AI in Vernon Browser to help people discover and explore things in your collection.

Learn more about Vernon Browser

• About Vernon Browser
• Client websites
• Vernon Browser Help
• Image auto tags
• Colour swatches