Pedagogo

Developing an intelligent assessment tool

We delivered an intelligent assessment tool for end of apprenticeship assessments, saving Pedagogo time and money.

Brief

Future focused design

Pedagogo required an intelligent assessment tool to deliver end of apprenticeship assessments. The tool has been designed for the future with upcoming phases making use of AWS machine learning and facial recognition.

Section 1

The challenge

Pedagogo, a startup company, chose us to deliver an assessment tool that tests everything an apprentice has learnt during their apprenticeship, so an assessor is able to gauge their level of ability and knowledge. Amy, the project manager who led this project says; “There are a lot of competitor sites out in the marketplace but none have this technology, they may be able to do the questionnaire element but not the video interview, that would have to be done face to face.”

Amy led teams throughout The Virtual Forge, along with involvement from our sister company Dreamtek, to produce a product that will revolutionise the assessment of apprentices. Using Adobe Connect allowed video conferencing for assessments and AWS for their facial recognition and ML tools. When the system launches it will enable users to register each apprentice under facial recognition. As the number of assessments occurring at any one time varies, the AWS solution seamlessly scales up and down, providing all users with on-demand access to the system whilst delivering a sizeable cost saving.

Why adobe connect?

One of our back end developers, Carlos explains; “Adobe Connect was the fastest and most suitable solution to use and it worked within the projects time constraints. It runs on Windows, Mac and Linux, therefore enabling users with different operating systems to easily access the system.”

The key Adobe Connect features used, include:

  • The ability to share files during an assessment
  • The ability for the apprentice to share their desktop in order to share a presentation
  • Video conferencing for the assessment

These useful Adobe Connect features allow for moderation during the assessments. The system allows for interactive chat and participants can put their hand up to ask a question, all much more advanced than usual video conferencing. The capability of Adobe Connect also allows for quality assurance and Carlos says “the assessor can record the assessment to evaluate later. The assessor and apprentice both join the session and the chat is recorded, later the assessor reviews and scores.”

Why AWS?

We've been an Amazon Web Services (AWS) partner for 4 years. Developing Pedagogo’s product required a level of scalability which AWS is able to provide. The front end and database are hosted on AWS virtual machines, with additional levels of security for the facial recognition service. This service allows the system to recognise a face that is logging in and cross reference with a profile image that is stored on S3, their storage system. Whenever an apprentice logs in, the system performs a facial recognition check, ensuring that the correct person is logging in, not allowing imposters to take important assessments, avoiding cheating.

AWS machine learning (ML) allows for exciting possibilities with the assessment of the apprentice’s interviews. In the current phase of the project multiple choice assessments are able to be marked by the system using ML, saving human time and reducing human error. Another useful feature of AWS is Amazon QuickSight, a fast,cloud-powered BI service that makes it easy to build visualizations, perform ad-hoc analysis, and quickly get business insights from data. Data is easily accessible from the cloud, analysis can be run and visualised, providing a useful assessors tool.

Pedagogo 3

Development

Pedagogo is an ongoing project, having started in January 2018. During phase 3 of the project we will start to really make use of the exciting capabilities of AWS machine learning. With potential for voice interviews to be transcribed into text and marked by the system using ML. The system will have to learn the pattern of assessment and answers and when in operation it will further reduce human time cost and take away the element of human error by marking free text assessments elements of the assessment.

There may also be the opportunity for Pedagogo to package up their unique product and sell on to apprenticeship organisations, recruitment agencies and colleges. There will be the option of purchasing the product in a white label format so that it can be bespoke to the particular purchaser.

Pedagogo 4

The Benefits

Using the capabilities of Adobe Connect and AWS, organisations assessing individuals will see many benefits from using Pedagogo;

  • A reduction in cost – due to the scalability of the AWS system. Remote assessments would allow organisations to only pay for the time used for the assessment not a full day.
  • Location is irrelevant – remote assessments using the Adobe Connect tools cut down travel for assessors and apprentices. There is also no need for a specialist assessor to be based in your area.
  • Security – AWS facial recognition means that only the correct user can login for assessments.
  • Reduction in time required for end of apprentice assessments.
  • Amazon Quicksight – cloud storage and tools for analysing and visualising data, enabling an easy way to present the results of apprentices assessments.
Pedagogo 5

Why the Virtual Forge?

We have an incredible wealth of expertise across The Virtual Forge and almost every department in our team has been involved in the Pedagogo project, and as Amy says, “we are so proud of the work we’ve completed”.

Our Co-CEO Matt Wicks has scoped the project through phases 1 and 2, along with our Directors Garrett and Leonel. Matt with his expertise in machine learning has been leading the machine learning and facial recognition aspects of the project. What a team effort!!

Pedagogo 6

Contact

Get in touch with The Virtual Forge