Last Updated on May 31, 2024

With all the hype about how ‘easy’ it is to make incredible 3D interactive products online, some clients are asking themselves – why don’t we just make these 3D scenes ourselves? For example, why don’t we just download Unreal, or Unity or another – and make these incredible 3D interactive models with features.

Some try it, and then see just how hard it actually is…

The great news is that there are so many parts to it, and an infinite depth you can travel in making features, behaviours and connecting systems – we are more than happy to share and collaborate.

So here are just some of the steps to make 3D software and 3D RT models, especially geospatial accurate interactive software.

There is a lot of very expensive and convincing marketing from the very awesome Epic Inc that saying ‘look how easy it is to do, and it’s free – just download and go’. Sadly it’s not so simple. The estimates for how much “Grand Theft Auto V” cost to develop and market is some $200m – $250m. That’s a lot of money….

Here some of the steps to making a 3D interactive model and scene. Nothing too complex, just a standard engineering RT visualisation. We suggest you pick out the elements that you can work on, and we support you with the other elements.

Convert 2D to 3D geospatial – modelling and understanding drawings, data & schematics

Clever modeller designer technical geniuses understand complex schematic drawings across industries, CAD drawings, rail schematics, non-geospatial diagrams, pen sketches and whiteboard scribbles – and convert this into real and accurate 3D models. But for interactive scenes, this modelling has to be very disciplined to manage 3D geometry – even despite the hype of ‘trillion triangle geometry handling’ with nanite and advanced graphics cards.

Professional 3D modelling itself in itself is an advanced skill with millions of buttons to select and consider in niche software packages.

Clean / convert 3D to RT suitable – optimise and manage design / procedural packages

With designers like architects and engineers doing BIM and Digital Engineering and making 3D data in 3D packages – this is good news. But. Those packages are often parametric or procedural, and need to be converted to polygonal geometry if its to run in context across kilometres of space. One Bentley Inc rail staunchen can cost you 100million triangles. With 10,000 staunchens – you cant afford 1 zillion (that’s above a trillion yes?) triangles before you even think about terrains, trains, houses and stations.. So optimise you must. And that’s not just reducing polygonoids, thats scene batching for drawcall management – an entire science in itself.

So your clever 3D guys don’t just know how to use 3DS Max or Maya, but also 12D, Bentley, AutoCAD, Navisworks and all their eccentricities (and expensive licenses). And then you’d better understand scripting because the way layers and sub-layers, directories and data structures all get managed is a nightmare of unnecessary complexity of historical reasons in the industry.

Convert & manage 3D survey – make RT suitable

Modern technology gives us the wonders of point clouds and photogrammetry. This is some of the most transformative data yet – and it’s pretty cheap to capture and organise. But it can be terabytes, petabytes, of information and is so big and clunky that most people can’t use it. It sits in a cupboard being awesome on a hard drive.

Clever 3D data wranglers need to understand this data, be able to work with it, have a system to process it, tile it, LoD it, thin it, structure it, capture its colours, maintain its accuracy, geolocate it in the same coordinate system as other data – then have a system to publish it into a real-time (RT) 3D space.

But often this data needs to be cut and sliced into other data – such as for background/foreground data. So your tool makers need to make tools to cut and edit large data sets inside a workable environment and integrate into other 3D geometry.

Develop 3D data scripts – to manage large and complex data flows

Many functions and data transforms require repetitive works. E.g. moving 100,000 objects from a Navis model to somewhere else. The way the data moves around is often limited and ‘particular’ in different softwares – either stripping data or adding junk data to layers, elements and sub-files. You really don’t want to be doing this manually, as labour is the most precious resource. Thus, you need people who can think “this is scriptable” and define the script, then you need people who can write the scripts. There’s a range of scripting languages, depending on the place the action needs to occur. There are a bunch of free scripts online, but like most free things, you get what you pay for. Scripts need to be tested and verified, then shared workflows setup.

3D scene model, integrate, materials and optimise for RT – 3D modelling

Then, after all this processing and wrangling of data, some people are actually doing modelling. 3D Modelling is actually pretty complicated, especially for organic surfaces. If everything was rectilinear (unlike the surface of the earth), then it would be so much easier. Within all the above data, someone needs to weave these pieces together, model new pieces, and connect them all together. And that’s just geometry.

Then there is texturing and materials – a massively deep cavern of wonders. Shaders and materials are absolutely fascinating and there are many books and blogs written on it. Materials are barely a consideration in CAD and BIM – there are a massive part of the game in 3D RT worlds. If you want to do it right, you want technical artists who just do magic with materials….

3D modelling plugins – for rapid processes

There are specific 3D modelling tools and plugins that make specific functions easier and faster. Examples are rail-clone and forest-pack. These are pretty industry standard for certain functions, such as running items along splines, or distributing assets in spaces. They need to be purchased and maintained, learnt and utilised.

For our RT scenes we have made better, or more customised, tools for our needs to use in editor time. But we still need these tools and others for non-RT production pipelines.

3D assets and libraries – build, manage, materials, rigging, optimise, store

You want to develop and maintain 3D asset libraries. Yes, you can buy some online – but they are almost never right. Too detailed or too poor, wrong formats and scale, poorly rigged or not. As you work over the years, you need to develop and protect an asset library.

Trees, for example, are a total nightmare to build. Just try one. I have made all these mistakes, and lost weeks of a highly skilled modeller in a single tree. You can buy SpeedTree – another product to manage and learn. Then there are 3D characters – the hardest of them all. Walk cycles, animations, rigging…. again you can buy them, and there’s 100 considerations in amongst that – not least being IP. Any humanoid immediately opens the uncanny valley issue – and moving to animated, moving characters opens up all those extra animation oddities.

Note that “materials” are unique in all workflows unfortunately. For each RT package (eg Unity vs Unreal) and pre-rendered pipelines (VRay, Corona) – each has its own unique materials that need to be rebuilt per pipeline. Its tragic and true.

3D Model composition and publishing – to RT space (editor)

Now your 3D specialists need to check every triangle, make sure all vertices and faces are welded, no double-ups and z-fighting for coplanar geometry – just good modelling. And our guys will show you any number of examples of poor quality modelling. They love gridded, clean, optimised geometry for 3D RT scenes. If you are doing pre-rendered, it doesn’t matter – as long as it looks pretty. For RT, every element has to be near perfect because the user can go anywhere and see everything.

Then you need to check that all things publish correctly and are composited. There’s always a lot of buttons to press and check.

Your smart 3D guys also need to know when to do 3D works in modelling packages, and when to do them in RT editor packages. Because some functions are easier or better for performance here, and some there. It’s all very specific. It also depends on the skills of your other team members in writing editor tools for your workflows. If you can make smart tools that do things easier, then off you go.

Scene editing – in 3D RT editor

Assuming you have artists and developers who are familiar with the RT space – there are thousands of buttons and commands to use. Unreal editor itself is pretty complex even before you start making fancy scenes and scripts and actually ‘doing’ anything useful. But you will learn that.

Firstly you want a technical artist who can just optimise the scene for real-time (RT) use. There’s a heap of scene editing tools in Unreal or Unity you can use that have some advantages, but aren’t for hardcore modelling.

Optimise and publish 3D to interactive scene – unreal developer

Publishing and actually making an interactive scene, even just with a single camera, is a function that requires skill. With an advanced editor like Unreal, its not just ‘push a button. That would more suit a polished product like TwinMotion or Ensapce – which are great, but you can’t add features. Like anything in the world, there’s a pay-off.

The good news is that publishing scenes themselves, ignoring functionality, can be a fairly junior editor developer task. There’s a few hundred button clicks to be followed in a logical process to get the thing working – and presto – you have your first interactive 3D scene…. but…

Environment Art, materials, lighting and colourisation

… why does my scene look so… boring? Well… there’s a lot of reasons. But firstly you will need technical artistry on the editor side to lift the lighting and materials. Don’t be too upset when your first scene is flat is a pancake or a CAD drawing (see notes on materials). You have to want to make your scenes look amazing, and put the labour into doing it and learning all the nuances of lighting.

When you are working with survey data like aerial photography or photogrammetry, even LiDAR, one hand can be tied behind your back because you have an existing environmental art effect and style that the rest has to work to. So this takes particular skills. Normal and bump maps, reflections, depth shaders, edge blurring and 100 nuances that sit in the blurry interface of the engine you are using and photoshop or 3d texturing tools.

Behaviours and animations – complex asset behaviours like rail, traffic, bikes… coder-art blur

A special mention is needed for complex assets – like a train or articulated truck or similar. Firstly making the 3D asset is harder than you think. In our world, train manufacturers, for example, won’t share drawings for some weird reason, so we have to use other reference material. And then you have to make these assets behave realistically. Even making a train run on a track is a definite thing. Acceleration and deceleration curves, articulation around corners, multiple cabins and more.

Getting complex objects this behaving correctly is a blurry space between artist and coder, back and forth they go resolving all the little issues. Then if you want to add real doors opening and interactivity – you are jumping deeper into code and art synergies that is compounded by the number of other things going on in the project. The more interactivity you have, the more code gets in the way of itself – this is why most games are so simple in generic behaviour – e.g. point and shoot, change gun and walk… point and shoot.

Develop 3D runtime tools

Now the real hard stuff, is actually making 3D tools and functionality. Making a system. Now any one function in itself is not that bad. And the false notion in our heads that we can just ‘grab it off the marketplace’ ignores compounding effects in code. So if you want something simple, like walk and run, all good. If you want to “make” things and “save” things – then you need a pretty complex system, serialisation and a system architecture.

For simple products, it is definitely best to stick to blueprints and ‘hack’ them together. The metaphor used in code world is once it works once, with all the sticky tape and string holding the code together, then don’t touch anything!

But if you want multiple functionalities, then you need to invest heavily in smart systems and processes and people to make this thing work. Every single new function will almost certainly affect every other, and so you need to be very careful. A lot of time big fixing and reporting. Also a lot of time instructing coders who really often need very specific instructions and use cases to show them what you mean. The good ones are kind of autistic geniuses often – and very literal. They need a lot of support and management.

System Architecture and Management

So now you have a code base to manage, a system to architect and maintain. You’ll need bitbucket and github and jira or their equivalent. You’ll now be a software development house!

In conclusion

This is about half of what you need to do to be making interactive real world architectural or engineering models in real data with some functionality. A summary of some of the capabilities you might need and a recipe for success.

When you have had a try yourself, like some of our clients have, and perhaps wasted 2-3 years and a million dollars, do feel free to give us a call and see if we can help you out in building your realtime 3D engineering, planning and architectural 3D environments.