Mission Hack- Cognitive Services, Bots, IOT and gaming fun


Guest blog by team Mission Hack from Cambridge Hack by Johannes Goslar Microsoft Student Partner at Oxford University

clip_image002

So the idea

In the beginning, we had a motivated team of 4 randomly mixed together people, set out on a mission to not cure cancer, but visually identify different skin conditions, we built a hardware prototype and reduced the cable clutter of a raspberry pi when attaching a camera, speakers and a microphone.

The Pivot

But we only had 24 hours, we needed more data and more time to do that. So then one had 4 slightly less motivated people frantically searching for another idea integrating their newly learned custom vision experience and their refreshed chat bot construction.

The new Plan

After some Red Bull, the usual backup plan looked more and more appealing: Build a game with that. In 18 hours. (While getting some sleep). Some brainstorming (and inspiration from previous games at other hackathons (game designer standard procedure: copy/modify/mutate)) brought us to the final conclusion: “Mission Hack”.

As the game is location bound, you sadly cannot immediately jump into it, but watching this video hopefully helps:


Mission Hack Summarised:

All participants get the same new mission every 5 minutes.

What’s the mission: Take a picture of the asked item, which might be: stress balls, google keyrings, crisps backs, red bull stacks, fire exit signs (safety is important), Microsoft swag, toilet signs…

We trained the custom vision endpoint to differentiate these (and notice if you take a picture of a picture).

The first person to do so gets a point and no one else can claim the current mission.

Wait for the next mission.

Repeat.

Let me show you our fabulous pitch

clip_image004

Our treasurehunty design based on old books (and WordArt).

clip_image006

Mission Hack will grow soon (at the next hackathon) to allow the full takeover of all events you are visiting.

clip_image008

It’s 2 simple steps

Find the items

Locate the beacon

Crack the…

It’s 3 simple steps

Find the items

Locate the beacon

Crack the code

The Code

Which code you may ask now, this code:

clip_image010

No hackathon is complete without some hardware, so instead of finding items and taking the picture, another mission type was to find the hidden Raspberry Pi and decipher its binary code.

Now you are probably asking, how did that all work together. Simple Solution: Microsoft Azure.

clip_image012

Mission Hack is basically a plain chat bot, built on the Function Bot template. The custom vision service is accessed via http calls, and as the function bot itself is a standard function app, it was easy to add another http function the pi could connect to.

The bot framework made it quite easy to connect it to the events slack and our fresh Facebook page. It sadly couldn’t convince the Facebook people at the hackathon to review our bot and enable it for public audiences, we had to do that on our own.

So how does the code work:

clip_image014

Well, at least, this is the most important bit. Bugs are mine, no code review was done, it did the job for the moment, don’t replicate at home. (Or if you want to do so: https://github.com/TernaryMed/ternaryBot (ignore the commit log))

Back to the presentation:

clip_image016

Top right: Jeff, our ML sorcerer from Edinburgh, also handling the mysterious beasts of the function app/slack/Facebook connection wizardry.

Top left: Johannes from Oxford, absolutely not likely to drink some Red Bull as evident in the first picture of this blogpost

Bottom left: Matt, our metal bending and resistoring Raspberry Eater from Cambridge, occasionally turning on LEDs by pure will power (or pressing the on/off-button)

Bottom right: Liz, tamer of the Custom Vision incantations and definitely the person who took the most images of all hackathon visitors

clip_image018

If you were a participant at the hackathon you might still try out Mission Hack at the event’s slack. And if you think, that definitely didn’t work, you are not totally wrong as evident:

clip_image020

Notice to self, don’t add random bots to general channels at 3am without testing or explanation (or at all).

What we Won

ARM liked our use of hardware and we won the ARM hardware prize, which will enable even more interactive hardware next time.

Microsoft liked our use of Azure, awarded use the Azure Challenge prize and made sure we continue using Azure, which truthfully didn’t take any convincing at all. I will definitely use it for the next hackathons.

Hackathons

If you are interested in using Azure, making a game and joining my team (and also are going to START HACK in February or VHACKS in March), feel free to message me at https://twitter.com/ksjogo

Comments (0)

Skip to main content