After Skymoons Edinburgh closed, I joined Sublime Digital in Glasgow in 2018.
A spin-off from the successful Soluis Group, Sublime specialises in creating immersive experiences using technology such as Virtual and Augmented Reality.
Mobius / Edify
My current project at Sublime is Edify, previously called Project Mobius.
Project Mobius started as an Innovate UK funded partnership with the University of Glasgow to create a suite of ten Virtual Reality applications to be used to teach a range of subjects at the university.
My main responsibility was to design and build the Mobius Platform, a common codebase that all the applications are built on.
One of the goals of the project was to store huge amounts of data about how the student approached the lesson and interacted with the various elements in the VR space. The anonymised data was to be studied later to look for patterns and draw conclusions about how students learned in a VR environment.
The type and amount of data that was required (ranging from the basic 3D location of the headset and controllers, to the movement of levers and switches, and events specific to each application) made it difficult to use any of the available VR plugins for Unity that existed (eg SteamVR). Instead, the decision was taken to build our own platform from the ground up, giving us the most flexibility.
Mobius Platform is an application framework within Unity that allows an application to be built easily. The features it boasts are extensive, but include:
- Controller support: Button mappings, haptic feedback.
- Interactive objects: Pick up objects, operate levers and buttons, etc.
- Movement: The user can teleport around the space.
- World space UI: Complex world-space UI that can be used with a VR controller. Context menus for objects that
- Front end: A series of customisable menus that can be used to configure the lesson and start the VR session.
Once work was completed on the platform, the ten applications were developed on top of it. I was the sole developer on two of these applications, and completed ~75% before the Covid-19 pandemic struck, completely changing our plans.
- Printing Press: Teaches typesetting skills and lets the student operate a 15th Century Gutenberg Press.
- Disease Diagnostics: The student performs a Zika virus diagnosis in a molecular biology lab, learning proper pipetting techniques and the process of preparing a solution for a qPCR machine.
- Battling Infection: Demonstrates the process of fighting a salmonella infection using the body’s immune system and takes a tour through various parts of the body including the intestines, a lymph node, and the blood stream.
Edify – a Covid-19 response
When Covid-19 struck and the world started to work from home, we quickly realised that having students gather in labs full of VR headsets and share them was unworkable for the foreseeable future. Project Mobius was quickly pivoted into an online teaching system that allowed the lecturer to use the power of a VR environment and share it with students that were watching remotely using video conferencing software such as Teams or Zoom.
A new application was added to the suite of ten that would work as a general teaching space, allowing the lecturer to bring in their own content from various sources; 3D models, websites, PDF documents, videos, etc. These could be annotated with lecture notes and presented in sequence. The environment can be switched (eg a lecture theatre, a medical lab, a moon base, etc). Cameras can be placed and selected for broadcast. Students can watch over video, chat via a connected mobile app, or even jump into the environment themselves in multiplayer VR.
To support our vision of this application, the Mobius Platform needed to be extended and support for the new features added to the existing applications as well as the new one.
- Networking and was required for the mobile chat application, and to multiplayer was needed to support spectators in their own VR setups at home.
- Recognising that not all lecturers had access to VR hardware, we added support for ‘desktop mode’ which simulates the VR experience using a keyboard and mouse.
- Broadcasting to Teams and Zoom, etc. The applications needed to appear as a webcam in the settings of any video conferencing software.
- Viewing Teams and Zoom windows from within VR so they could interact with the students while wearing the VR headset.
I led development on the changes to the Mobius Platform while mentoring and leading the junior members of the team, which was growing fast from a recruitment process that had started before the pandemic.
The academic response from Edify has been fantastic. It is currently entering beta testing, with the University of Glasgow and several other institutions that have been brought in via an invitation to submit ideas for future lesson applications.
Prior to Mobius / Edify, my first project with Sublime was to develop an integration with Autodesk’s cloud-based Forge platform.
Sublime operated several “shared immersion” portals, which house several projectors inside a geodesic dome to provide a VR-like experience to several users at once. This project was to bring architectural models into this experience, and make them interactive.
I first developed a .NET library in C# to connect to Forge and access architectural models and metadata in BIM format. This library allowed a user to login, scan through files stored in their Forge account, then download and cache the models in a format that was optimised for loading into a game engine.
Next, I developed a Unity application that used this library and presented the models in 3D on the inner walls of the portal’s dome. Using a single Vive VR controller as a pointer, or an Xbox wireless gamepad with a virtual cursor, the user could quickly find themselves standing within their models of landscapes and buildings.
I added several features that are common to architectural model viewers, such as:
- Slicing: Using a custom Shadergraph shader, the models could be presented as cross-sections.
- Measuring: Selecting two points within the model would display the distance between them. Selecting a single point would use the face normal of the selected point and measure the distance to the opposing geometry.
- Metadata: Selecting an object would display a searchable dialog of metadata for that object. Frequently this would be details of materials used in construction.
I also prototyped two further versions that were not picked up due to time constraints.
- One that connected to a plugin Sublime had developed for Revit, an industry standard application for architectural modelling. The plugin allowed my application to show the model that was currently loaded in Revit, and dynamically update as the user made changes. This would allow instant immersive 3D visualisation of a building as it was being modified and tweaked.
- Another that used a VR headset as an additional display. The ability to display simultaneously on any combination of the monitor, in the portal, and on a VR headset opened the application up to more uses. In smaller exhibitions where a portal was unavailable, VR could be used. In larger exhibitions it allowed a single user in VR to be given control over the experience, meaning the guests in the portal would not be allowed to “see behind the curtain” with the user interface.