Custom Vision Model for mushroom classification using Bing Images

  • Article

Creating a mushroom classifier

I'm no mycologist, but am a fun-guy (badum...tish).  I know there are a few different types of mushroom about in the wild and some should never be eaten.  Here, I'd like to build an image classifier that can identify a few different types of mushroom.  I'm a busy guy, so naturally, I don't go around taking photos of every mushroom I come across, therefore I don't have a large photo library of different types of mushrooms.  However, Bing has millions of images already indexed and stored and has an API that I can leverage to return them.  I can then use these images to create my own mushroom classifier within Microsoft Custom Vision - using the Custom Vision API.

First, we need to grab some images of different types of mushrooms

I've created a simple cross platform CLI to do this: https://github.com/daltskin/Bing-Image-and-Custom-Vision-CLI/tree/master/BingImageCLI

To use we simply provide the arguments for the Bing Search criteria and Bing Search API key eg:

BingImageCLI.exe -k yourkey -s "Amanica muscaria" -l ShareCommercially -p c:\photos\mushroom -m 50 -fmax 4000000BingImageCLI.exe -k yourkey -s "Coprinopsis atramentaria" -l ShareCommercially -p c:\photos\mushroom -m 50 -fmax 4000000BingImageCLI.exe -k yourkey -s "Geastrum triplex" -l ShareCommercially -p c:\photos\mushroom -m 50 -fmax 4000000BingImageCLI.exe -k yourkey -s "Lycoperdon perlatum" -l ShareCommercially -p c:\photos\mushroom -m 50 -fmax 4000000BingImageCLI.exe -k yourkey -s "Psilocybe cubensis, magic" -l ShareCommercially -p c:\photos\mushroom -m 50 -fmax 4000000BingImageCLI.exe -k yourkey -s "Agaricus bisporus" -l ShareCommercially -p c:\photos\mushroom -m 50 -fmax 4000000

This will download over 200 images (some may fail with 403/404s) to the c:\photos\mushroom folder each within their own subfolder of category eg:

You'll want to quickly scan through all the downloaded images, deleting any rogue ones - as some images may have been incorrectly labelled or will simply not be useful for your classifier eg:

Next, we need to create a Custom Vision model for our mushroom classifier

I've created another cross platform CLI to do this: https://github.com/daltskin/Bing-Image-and-Custom-Vision-CLI/blob/master/CustomVisionCLI/Readme.md

To use we simply provide the arguments for the Custom Vision API eg:

CustomVisionCLI.exe -k yourkey -p c:\photos\mushroom" -n Mushroom

This will upload all of the remaining images above, labelling them at the same time and will then train the model.  After a couple of minutes you can then test the new classifier with a previously unseen image.  You can either do this in the Customvision.ai portal, or again from the CLI tool eg:

CustomVisionCLI.exe -k yourkey -p c:\dump\unseen\mushroom.jpg" -n Mushroom -q

[caption id="attachment_2305" align="alignnone" width="180"] Closed cup mushroom from the local supermarket - not used in the training data[/caption]

[caption id="attachment_2295" align="alignnone" width="300"] Agaricus bisporus correctly identified[/caption]

More details and demo here: https://github.com/daltskin/Bing-Image-and-Custom-Vision-CLI/