[!NOTE]

Even my older AI blog posts are starting to feel old already, but I am enjoying the challenge!

New patterns for a new age

The Bueller quote heading this article has never been more true, it seems almost daily I am reviewing the news, latest posts from ALL of the major partners involved with Agents and LLM’s for new patterns, ideas and suggestions. Thankfully, everyone is riffing off each other in a good way, someone suggests or posts something, then someone else builds on that and goes that bit further, whether it is:

• A new LLM logic.
• Improved MCP interfaces and tools.
• Patterns, plans and guidance.
• Token reuse or conservation.

The list goes on, in some ways, it is like the best parts of collaborative science (in some ways, a dark dystopian nightmare), ever creeping forward.

This post was sparked after I followed up on some of these reports and started to see REAL IMPROVEMENT in several key factors in my own research, namely:

• Reduced hallucinations (not gone completely, but significantly reduced).
• Improved memory (yes, you can have memory if you do it right)
• Improved outcomes.

It almost sounds too true, but my workflow has improved, especially with some dedicated research projects I continually use to test new patterns.

[!WARNING] This is NOT a silver bullet, to get more out, you MUST put more in, and thus the cost, depending on what you are using will INCREASE.

My recommendation, as is becoming more of the goal in today’s market, is to HOST your own LLM infrastructure (they have it running on Raspberry Pi’s now). It is possible with a little effort, and you can reduce your downstream costs to all but the critical path.

Sadly, this is out of scope for this article, but I will try to add more later.

So, what are these patterns I tease?

Evolving patterns

The updated patterns I have started employing recently fall into two categories with a single shared theme:

Shared theme - The living document

The core of the recent updates is to create LIVING DOCUMENTS at the core of any process. This is a SINGULAR document (and you MUST be explicit, as some agents, looking at you Claude, will randomly go off and create masses of documents you did not ask for) which ALL agent calls must update with their state, or you inform it to update at critical junctures.

I have found this works far better than using the Memory MCP, or constantly updating the instructions guide for the agent, and as a separate document, you can create references to it from all other material, including the instructions. In testing, this has been far more efficient and rigorously followed than instructions or memory alone.

• You start the session with the instructions pointing to the living document
• The agent reads the current state with suggested next steps
• It validates any failures or bad paths from previous attempts
• It then plans ahead!

Your mileage may vary, but this singular change VASTLY improves my outcomes. Granted, at the cost of the additional tokens needed to read the document, hence the additional cost. You “can” summarise the document in Agent format if you wish, but I have found this actually degrades its performance and use.

Patterns

Now to the other meat on the bone, the revised patterns that consume the living document. I have long stated that LLM’s are more tuned to the small tasks, a quick fix, diagnosis, but where you need to dig deeper, you have to keep a better handle on the outcome, thus these are the methods I have now turned to using:

It all starts with a defined and reviewed plan

By far my biggest improvement through LLM use is to focus and narrow its path, to think and to constantly challenge its assumptions. It is NOT foolproof, but it greatly improves your chances of a positive outcome. The whole approach (which is token-heavy) is an almost constant state of review, approval, implementation and review, which when implemented, looks something like this:

• First Agent session - Create the PRD (Product Definition) from a detailed set of requirements, this ensures a list of preferences and outcomes.

• Build an instruction document using the PRD as a reference. Guiding the steps for planning, with KEY steps to avoid creating documents unless requested (stops the multi-document scenario). This might sound counterintuitive, however, they serve very distinct purposes. One directs how to think, the other tells the LLM what to think about.

  [!IMPORTANT] Ensure to leave off with a note about the aforementioned Living Document, instruct the agent to check the Living document at the start of a prompt and update it with its intent and finish the query with a statement in the living document as to the outcome. This trains and helps the Agent learn, as well as something to pick up on if you start a completely fresh session.

• Next, start a completely new session (even with another agent if you wish), define the session as a reviewer / architect actor and have it review and improve the plan. Take note of what has changed and follow its thinking. Make corrections where YOU disagree. It is essential you are part of this review process, as key decisions that were maybe not clear in the design become apparent (as often happens in life, poor design leads to poor outcomes).

  [!NOTE] You can repeat the previous step a few times with different defined actors, different personas, just as you would in real life. The best plans involve a team, and in this case, it is a team of agents AND YOU!

• At the end, ensure to begin the plan with a Living document record as it delivers the implementation.

Once you are happy with the plan, the next step begins, the following two patterns can be taken:

Plan big but follow your own path

With the plan in hand, instruct the Agent to document and detail the plan for implementation, ALWAYS ensure you finish with “If you have questions, ask them before beginning” (If you do not, it will not, and it ABSOLUTELY SHOULD). The outcome choice is up to you:

• A singular implementation document.
• Several documents, one for each stage or component, ordered for implementation.
• A mix, depending on your style, backend first, then frontend and UX.

Ultimately, this is a plan YOU will follow, this is my preferred mode as it gives more opportunity to question as you implement or even change your choices. All the agent has effectively done is ratify the architectural state based on your inputs, it is arguably the most human way to use the tools to achieve your goals.

If anything is unclear, ask the LLM to clarify, make changes or explain something “BUT ONLY TO THE PLAN”, the Agent should NOT touch the code, that is your domain.

[!NOTE] In my experience, when instructing the planning phase, I also interject to ask the Agent to explain each section or block of code, as to its intention, what it is meant to achieve, or what it is supposed to do. This helps guide the implementation as to whether it is the best thing to do. And if you disagree, get it to update its plan, or make your own implementation and then get the Agent to update the spec, then review for ancillary impacts.

This approach feels more like the Agent working cooperatively than running the show and seeing what it comes out with.

[!IMPORTANT] At each stage, with each change, ensure to instruct the Agent to update the living document. It might be recorded in the Instructions, but I feel safer double-checking at critical points.

At all points, question everything, it is simply good for the soul and makes the challenge more fun. All that is left is to continue to the end, test everything and the result I have found to be vastly improved.

[!TIP] Another fun step, but at the cost of tokens, is to ask the Agent from time to time to review what you have actually implemented against the plan and give a status report (helps to reduce human error), marking the document with AI’s favourite thing, Icons and Emoji’s. As well as providing a summary, as well as giving you a visible checkpoint of your progress.

Automated plan, automated deployment

The automated plan follows a similar track to the manual implementation path, except you ensure to break out the plan into repeatable and testable sections. It is not as efficient as the manual path, but for shorter and more throwaway experiments, it can be beneficial.

[!NOTE] In some cases, I am actually running the two approaches in Parallel! Using the fast route to test theories in advance before incorporating them into plans, similar to prototyping.

Rather than the human doing the implementation, the agent is running the show, but CRITICALLY, not all at once. To avoid hallucinations and dreams of code, each section must be complete, testable and ultimately HUMAN VERIFIABLE.

An example of this was a total conversion of an old game sample in C++ using some REALLY legacy assets. The plan broke down as follows (granted after so many failed attempts in the past):

• Review the project and break up / document the systems and content of the sample.
• Research old asset formats and create detailed documentation of their makeup (critical for migration).
• Build out sample sets of each content type and define a migration strategy.
• For each asset type, build individual pipelines, with human verification at each step.
• Then implement the migration in individual sessions or phases, each phase does NOT complete until the human in the process signs off on it.
• Then plan the implementation, in a stackable way, each implementation building on the last, again with human signoff.

Doing this phased approach with signoff greatly avoids mass delusion and lots of wasted tokens on something that can never ultimately work.

[!IMPORTANT] The human as part of the process is essential, not just for the quality of the output, but also as an efficiency surrounding costs and tokens. Yes, it is more effort than vibe-coding a website or app, but the results are FAR superior.

Conclusion

Thus ends this page in our journey, but the destination is still far from sight.

Learning should never end and we should always strive for improvement, and in my humble view, this is in Cooperation with an agent and not just handing over the keys on a prompt, no matter how detailed, as it is ultimately flawed. We learn more along the journey, take those understandings and evolve a better plan. It is strange, in the many years as a Project Manager, designer, QA and even analyst, this is ultimately how we humans work better, we plan, we question, we revise and ultimately deliver a better outcome. It might not be perfect, but from that, we continue to build better.

Enough musing, back to the code, which is where I feel most at home, with my new pal sitting on my shoulder. (Still not sure whether it is a little angel or devil, but let us see where this leads)

[!NOTE]

This is a living redesign log; content and layout may change as features land.

2026 needed a refresh

One of the things I dread is revisiting the design, style, or even template I use for my blog. I spend far too long browsing lots of cool things, find something I like only to find it is either not deployable or has “hidden” issues. The end result is always a trade off between what I want/like and what the template is capable of.

So this time things are different, using tools available (I still refuse to use a site generator), I began the journey using all the tools at my disposal to generate the site fresh, give it a modern look and then use AI tools to help refine/change what I did not like.

Still very much a work-in-progress, but delivered far faster using the tools available to me. I hope you like it.

The rest of this post is my ongoing test to ensure styles work, features are as expected and nothing “breaks” :D

Feel free to skip the rest and move on :P

Example post image

This post uses a date-based image folder so it’s obvious which images belong to which post:

/assets/img/posts/20251215/redesign-in-progress/

The banner image is referenced via front matter (img:) and can also be embedded inline:

What’s Coming

• Modern responsive design
• Dark mode support
• Enhanced search
• Better accessibility
• Social media integration

Stay tuned!

Markdown support (quick validation)

Inline code looks like bundle exec jekyll serve.

Strikethrough works: old plan new plan.

Fenced code block

puts "Hello from Ruby"

Task list

• Jekyll build works on Windows
• Add more posts

Table

TL; DR -> Whether or not you are for or against AI, if you do not at least read up on the landscape and the capabilities out there you WILL fall behind. These tools can be used in many different ways and provide support where you least expect it.

It is not about taking away the fun tasks, if you do not want it to code for you, then stop, want to create your own artwork, fine do that. But do not discount the capabilities of the tools because of preconceptions.

You do not need to use it, but understanding what you CAN use it for better arms you in the road ahead to know what you are up against.

Work smarter, not harder

The evolution of AI tools is ever increasing, as is any technology or area that is adopting these tools. It was already an almost impossible task to “keep up” with advances, causing developers and engineers to keep narrowing their specialties, reducing the mental load to be able to take in the latest and greatest.

Since the unveiling of the current generation of AI tooling, the curve keeps getting sharper and sharper with facilities such as:

• Guides on instructions to provide consistent context between queries.
• MCP servers and their capabilities have increased.
• More and more LLM model services for injecting capabilities into a workflow or service.
• Prompt generation research.
• New agent modes from Chat -> Agent -> Planning -> …

The list goes on and keeps getting bigger, refinements, new tools, new features. Like any technological “revolution” its speed is increasing.

[!NOTE] However one thing remains clear and should not be underestimated, is that AI STILL does not know it is right. So always be on your guard and check the facts! But that does not mean you should not use the tools, just learn the correct ways to work with the tools.

Based on my understanding (at the time of writing), this article aims to impart how my workflow has evolved to a level where I can institute “some” level of trust in my processes with the results from AI.

[!NOTE] Fun fact, even today when I started writing this post, I listened to a podcast with Jesse Liberty that featured “James Montemagno on Vibe Coding” ( a term I do not like tbh). In this episode James mentions a few “tricks” I had not considered before, such as using AI to generate the tool or a prompt based on your current activity to enable you to do that activity faster next time. Effectively engineering your workflow as you DO IT.

It starts with a prompt - The Misconception

I have said it previously, but it is worth repeating. Long gone are the days when you put in a simple question to get an answer or complete a task to fulfil. Sure, on your phone or via your browser, the results are FAR BETTER than dropping it into Bing/Google, but for any serious workflow. Without content/structure and even architecture, the most basic of queries will not result in anything that resembles what you desire. (except for that one day in march where the hares are flocking).

[!IMPORTANT] AI is suggestive, it is not indicative. It NEVER knows if it is right, only that the probabilities align with a result compared to your query.

So let us breakdown the critical paths as I understand them at the moment.

Generate Instructions/Agent document

Instructions or Agent guides are critical to modern AI interactions, they define the background to your request/conversation, define the scope of the work and critically also inform the AI of any boundaries it should not cross. Traditionally, it is also used to define the architecture to follow and conventions to use, but it can also be used as a thought board.

Do not think of your instructions as a static document, it is a living/breathing entity that should grow and learn as your project evolves. Include tips/tricks, any repeated lessons or “daft things” the AI performs that you would rather it did not.

[!IMPORTANT] The instruction guide is sent as a pre-context to EVERY chat/conversation, this is information it should know whenever it is sent off to think about anything!

But do not worry about its length or performance, the local agents “tune” the instructions and summarize it for easy digestion. You would be surprised about how much “pre-processing” happens in your client before it even contacts the cloud AI agent.

Simple steps to follow as your project evolves, whether it is a coding session, a Q&A query or just taking notes for a subject:

• If you have an existing project, use the built in VSCode (or other tool) “Generate Instructions” to set an initial framework.
• If you are beginning thoughts for a project, start a conversation with the aim of generating this instruction guide.
• Keep refining the instructions, with or without AI help to frame future conversations.

Above ALL, keep reviewing the instructions manually after generation, do NOT just trust the result, it is meant as a starting point not a finish line. Keep reviewing, updating and even deleting content until you are satisfied with the result. And as your project grows, so should your instruction guide.

Use MCPs relevant to your process

MCP servers are fast becoming the “go to” tool to assist in local AI support and cut down on the amount of traffic used by cloud agents, they help form, and in some cases, generate results to assist the AI in its journey, these can range from:

[!TIP] Model Context Protocol servers, or MCP for short actually details the method used to communicate from an AI LLM with a local AI or context agent, a tool used to support AI on its journey. Their capabilities can range from just an API for serving data or results, to full blown AI agents themselves. The MCP protocol effectively allows AI’s to talk to each other and work together to reach a goal.

• Simple functions used in calculations or a fact table.
• Headers for local AI agents or models, such as image generation.
• Self-contained LLM’s, primarily used for documentation or API references. Useful for keeping the AI bound to accurate source information rather than just the web.
• Proxies to other cloud hosted AI agents, woven together to form a mesh.

[!NOTE] Do not be afraid to AI your own MCP server locally, if you find certain areas where you are constantly correcting the AI agent, or need it to use pre-defined terms in its response, then just generate your own server to manage this for you and update your instructions to defer to the MCP server in responses. This can lessen the load in your instructions if you can have an agent maintain it for you.

Each MCP effectively adds another tool the AI can use in the construction of a response as well as off-loading tasks locally to save time or money (credits), or in some cases MANY tools. And there is a growing movement where you can use AI to generate your own local MCP servers, especially for repeated tasks or guardrails.

[!IMPORTANT] Too many tools can degrade performance While having lots of MCP servers at the ready is fantastic, be aware you need to limit which Servers/Tools are active at any one time. Giving the AI too many options can severely degrade performance as it need to check the toolset with each interaction (even within a response) to see if there is a tool to help.

Make sure for each interaction/conversation, that you are using the tools that will assist with that thread. Not asking it about code or pull requests, then disable the GitHub MCP tools. Also critical for any language or toolset focused conversations. Just use the tools that will help with your query.

Create custom prompts for repeated tasks

A recent trick I picked up surprised me how effective it was. If you find yourself either asking the same question, or making the same corrections, or even just doing the same thing repeatedly, then stop and just make a new customized prompt for yourself.

You see these already when in VSCode you hold ctrl + shift + P (or cmd + shift + P on Mac) to bring up the command window, from here you see regular actions available to you based on the extensions you have installed and the MCP servers you have registered. Once activated, they complete a set of tasks using what is available to perform an operation, be it cloning a repository, or formatting your document (my most used command).

But you can also create your own, and again, use AI to help you generate it. In the middle of a conversation when you have completed a task, ask the AI to summarize the task that was completed and ask it to generate a new Command in the tool. You will get the source for the command, allowing you to tweak it to your needs (and likely remove some unnecessary steps) and then save it with a command name, thus, every time you need to perform the same task, save yourself some keystrokes and just launch your command instead.

Leverage resources online

A common misconception when working with AI Agents, is to assume it is trawling the web for the current/latest information, sadly this is not true (in the most part), LLM’s rely on historical information they have been trained on. If there is newer information, it is best to direct the AI to also read specific documents, websites or other sources of information in addition to its trained model.

[!IMPORTANT] Although this does not “train” the AI in new information, the data that is gathered is only summarized and used as part of the current conversation. If you asked it again in a new conversation without providing the source, it will not know.

You can add such sources to your instruction guide, although it will only be summarized from the last time you generated the guide, alternatively you can either use an MCP server such as “Context7” (A useful docs MCP) or generate your own MCP server that refreshes its memory from online sources that you critically rely upon, like your companies document server.

Cost Management - The hidden tax

While many of the tools we use today offer generous free tiers or fixed monthly subscriptions, the move towards “pay-as-you-go” models for advanced agents and API access brings a new consideration: Cost.

Every interaction with an AI is not just the question you ask; it includes the entire conversation history, your instruction guides, and any files you have attached. This “Context Window” is re-sent with every single query. If you have a long-running chat with megabytes of code attached, you are paying for that data transfer every time you press enter.

[!IMPORTANT] Token Hygiene To keep costs (and latency) down, practice good token hygiene:

• Start fresh: Do not keep one giant chat going forever. Start a new conversation for each distinct task.
• Limit context: Only attach the files relevant to the specific question.
• Watch the loops: Autonomous agents that “think”, “plan”, and “verify” can trigger dozens of internal calls for a single user request.

Being mindful of your token usage does not just save money; it often results in faster and more accurate responses because the AI is not distracted by irrelevant noise from 50 messages ago.

Change your workflow

AI tooling should not be just about work, used correctly, you can use the tools to influence or guide almost any task, such as:

• Guided research.
• Ideation and thought generation.
• Validating preconceptions.
• Applying structure.
• Architecture compliance.
• General validation.

And these are just from my building workflow over recent months/years. Using AI to not just do work, but to plan, evolve, generate more tools to further accelerate your workflow, defining the ultimate development method of DRY - Do not Repeat Yourself.

[!TIP] Did you know there are several “MODES” of AI usage now? Not just Agentic (Agent) or “ask” (query), but also “Plan”, which is great for digging deep and coming up with a stepped plan of actions which you can either get the AI to start working from or do yourself, as well as several others and even custom modes you can create yourself, tailor the experience to meet your needs.

Guided research

When researching any topic, wading through mounds of documents, websites, information and searching for the latest (or even historical) information can be very time consuming.

Granted, this should be treated as source material and taken with a grain of salt as critical facts will need to be verified. Even with recent advances in correction and validation, it is still just an approximation. In many cases, it is worth asking different agents to read over the results and verify sources for cases.

You might think, why bother, if the information is potentially unreliable what is the point. But the answer is simple, the AI can search through data to find you a shortlist far quicker than any human can, reducing the scope of your research with summaries.

But do not stop there, keep iterating the results upon itself, demand references and lookups to the latest sources to verify the results, which in the end results in data you can rely on for your studies.

[!IMPORTANT] Even so, in research and summarizing the research, use your own words. Use the data as refined source material which is just a smaller window of the wider world contained within results. The ultimate aim is to save you TIME, if nothing else, it can also type faster than you can.

Ideation and thought generation

A concept I have also been using AI tooling for of late is idea generation. You get a half sleep deprived thought or waking moment of “something”, it sounds fine in principle but will it work.

This is where, depending on the type of project it is, I will lean on AI tooling to help ratify my thoughts:

• Has this been done before, if so what worked and what did not.
• Is there any market opportunity for such an idea.
• Summarizing options for “go-to market” strategies.

If the idea is a solution, even using GitHub Copilot to form a new project automatically, generating source, identifying workloads and deployment options. Which it will take on, generate a new repository, and start building out the framework of a potential solution (fantastic for web solutions). Give it enough detail, apply references for similar things you have in mind, and set it off. All of which will happen without you being involved (or even at your desk) thanks to workflow agents.

It does not need to end up with something real, except to prove to yourself whether something is valid or even possible. It gives you a framework to build upon and cuts down the time involved in your personal research.

Validating preconceptions

Often during my development and management life, we build our own internal bias, it is not always intentional and can lead you to incorrect assumptions. My geography knowledge for instance, is waaay out of date.

Most consumers of AI will turn to the agent on their phones for a quick query, usually with FAR better results than a one line google search (with so many Ad results these days), the results are better, but usually very inefficient.

I have been “trying” to teach my family a better etiquette for searching with AI assistance, forming not just a question but also how to form the response, for example:

Instead of just:

What are the best locations to go on holiday in June?

Better to give the AI heads up as to what you are really looking for:

I am looking to go on holiday in June, what are the best locations to visit?  I prefer hot locations (but not too hot) with spa facilities and a good nightlife.  Make sure to also review other visitors reviews of locations and only include places that reviewed well.  Form the results into a table listing the destination, the hotel or cabin, average review rating and links to booking information.  Also check the sustainability rating for the areas to ensure it is not over populated with tourists and has affordable travel insurance (around £30 pp)

This structure of query is simply more informative while staying natural and tries to tick as many boxes as possible for what you are REALLY looking for:

• What is the purpose of the search.
• What are the constraints.
• How you want the results formatted.
• And additional concerns or limitations.

This is just an example, but as time and tide has shown us, the more effort you put into defining your question (and results) the better the output will be and reduce the need for further onward searches to clarify results.

Applying structure

One of the hardest parts in any software (or other designs) is architecture, how well formed the solution needs to be. What constraints are needed, which conventions should be used and where the checkpoints need to be in order to avoid pitfalls.

Of the many designs I have done in the past, it is always the longest struggle to keep constantly refining the structure around any project, whether it is in design or implementation. What other considerations need to be considered or applied. Even with many years of training and experience, you always wonder “what if” or try to evaluate uncommon scenarios that could affect operation.

Using AI tooling to review the architecture, critically looking for “things you did not consider” is crucial. Even if you choose not to act on the information as it is not relevant, having another set of eyes with a much larger dataset can really help. I have solved many an issue in a junior’s design, as have my seniors done for me, so why not also use the tools available for a third set of eyes.

Defining a well architected solution leads into the next section for using AI to ENFORCE that architecture and prevent future works overriding the design (or at least get them to question the approach).

Architecture compliance

With any well defined architecture, adhering to that architecture can be challenging. But you can use the AI to help provide a ground truth based on the defined architecture, as well as verifying how compliant any development work undertaken is.

Additionally, the architecture itself can be used as a guide for any future work, using the implementation, design or instructions to inform work that is being undertaken.

As an example, I was building a new component within a service driven architecture and the AI tooling correctly identified all the key touchpoints within the architecture to add the new component, as well as adding Unit tests around the feature. It derived everything it needed from:

• The existing implementations
• The documentation associated with the project
• Designed instructions surrounding the architecture and bounds
• And existing Unit tests for other services (in fact it suggested some additional tests which we then implemented across the board)

This also meant that new work complied with the existing architecture and made the final review and update a lot quicker and simpler, I would estimate it reduced the time needed to build the new component by about 80%, meaning I could get more done faster. BUT, I still had to do a fair amount of work to review the implementation, but no more than I would normally.

General validation

Checks and balances are key for any professional work, in fact I now regularly use either Copilot as a code review agent or use the Copilot review processes for all documentation, not only to check for the usual grammar / consistency, but also to check my own facts, I prefer to always write myself and use the AI as a reviewer and NEVER the other way round (well, except that one time).

There are many tools in the AI world that are available and you will probably find you have already been using a lot of them, such as a spell checker in your documents (a nascent form of AI that is strictly contained), but writing with AI and also double checking your own preconceptions is now far easier. As an author I used about seven different tools in my own personal proofing exercise and with the addition of AI into that workflow, I have been able to reduce it down to three (because it pays not to have all your eggs in one basket) which saves me considerable time in review.

Summary

Hopefully this article opens some additional thoughts patterns, as mine do almost daily as I use, reuse and learn from my own use. I have gone far beyond the level of “generate this method” or “this error occurs, why?”.

Keeping up to date is a struggle, but it does not have to be, include all the tools available to you to make your life easier, or at least smoother. The tools can simply type faster than you, so use that to your advantage.

[!IMPORTANT] But even given all the resource, DO NOT trust the result, check it, validate it, challenge the generated pre-conceptions. Even with the extra steps to read through what has been created, the workflow is still significantly faster. So use it when it makes sense, and for those things you take joy in, and keep having fun.

TL; DR -> In short, NO, lol. AI is a tool that can assist you in solving common problems and mixing many different AI tools and features you can help to accelerate your development and the creation of content (at least place holders for your artists to draw from). But, it is also a great source of pain, frustration and injection of complex issues, which you end up using AI to solve only making the complexity worse and harder to understand if you are not familiar with it.

Essentially, the AI CANNOT do it alone and still needs a human at the helm to guide it, correct it and use other tools to generate assets.

It is however fantastic for DOCUMENTING your project and identifying potential issues!

The AI war continues

As ever progress marches on whether we want it to or not and with the ever increasing power that AI brings, everyone gets split into one of three groups:

• Hey, I can replace people with a prompt and just build project X using fewer resources (completely false, as the total cost of AI far exceeds the cost of humans)
• AI is evil and only scrapes resources from other hard-working developers and should be avoided at all costs.
• AI is a tool; if used properly, it can accelerate your work. However, it can also introduce complexity and challenges if not managed carefully!

Want to see what AI can do for your game development, check the latest result of my adventures :D

Figure 1: MechRampage Final render

In this article I will detail the latest generation in my experiments with using AI in game development, the terrors, the hail Marys, and the pitfalls. As well as some useful guidance for those new to the field.

The project definition

For this run, I went with a more complicated outline than I have done in the past, mainly due to the experiences I have built up this year, but also to challenge myself and the AI a little more.

[!NOTE] Past experiments involved simple block pusher games, 2D shooters, or even snake, just to give the AI a chance.

*Note, no, it failed to even get close most times.

This was the outline brief I went with:

Mech Rampage

Mech Rampage is a 3D isometric game where the player will navigate a futuristic city, battling enemies and completing missions. The player is a commander managing a team of customizable mechs, each with unique abilities and play styles.

The terrain will be procedurally generated from a set of assets, fielding such arenas as grassy outlands, urban environments, and industrial complexes. Note, all content to be generated initially by the agent and placed in the content folder for loading.

The play area is an isometric projection, allowing for a unique perspective on the 3D environment. The player will be able to navigate the space freely, utilizing the verticality and depth of the environment to their advantage. Control will be through a point and click interface or touch controls:

• When a mech is clicked, it becomes the active unit and the status displays show the stats for that unit
• When a mech is selected, the player can issue commands to move, attack, or use abilities
• When an enemy is clicked, the player can target it for attack or issue special abilities
• When a resource is clicked, the player can collect it or use it to upgrade their mechs

Generating a game from AI

The world of AI is rapidly changing and it is hard to keep up at times, it has always been this way with technology, however, with AI the sheer acceleration and complementary (sometimes derisive) changes make it harder to fathom your way through this maze.

[!NOTE] Full disclosure, while I have a lot of experience in the area of AI tools, I would never class myself as an “expert”, I simply know more than some, yet less than others. It is an ongoing learning experience to find the best ingredients to make a good cake. Just keep in mind the rate at which new ingredients are being made is evolving almost as fast as the range of cakes to bake (and not all are good).

The main thing that has changed in recent AI evolution is the level of planning and resources you can throw at a problem to help direct and make the eventual result better, or at least more in line with testable requirements.

Instructions

Everyone seems to believe that getting the best result out of AI is to create the perfect “Prompt”…..

This is simply wrong, at least in this day and age.

Whereas in the past, a simple quick question or request would be enough to accomplish a simple task, it would almost never be what you needed. Even code intellisense, while useful, would more often than not just “get in the way” while you are in the zone.

The resolution to the “quick” answer is effectively to go long, not in the question, but in the context or framework that you provide “up front” to the AI machine in order to essentially give it a frame of reference. This is defined as a precursor document or instruction guide; each tool you use has its own version of this:

• VSCode uses a copilot-instructions.md guide.
• Anthropic uses a claude.md document.

And others have their own variations, but they all surmise to the same thing: a document that describes such things as:

• The aim of the project (the document is meant to be system wide).
• The architectural principles of the project.
• How the AI should go about any task (more on that in the Planning section).
• Any key resources that should always be used.
• What are the strict rules that must be followed (my favorite with Unity projects is to remind the AI that Unity DOES NOT use DotNet CLI commands).
• Instructions on what the output should be, any documents that need to be maintained.

And in the strictness sense of the term, MORE is better, the more you put in, the better your result will be.

[!TIP] VSCode and its CoPilot implementation even includes a helper tool in its GitHub Chat integration to get you started with Generate Instructions available in the “cog” options in CoPilot Chat as shown below:

Figure 2: CoPilot Chat - Generate Instructions

Some awesome resource for working with these instructions guides can be found at:

• The awesome-copilot repository, chock full of instructions, modes and more.
• The awesome-copilot-instructions repository.
• Essential viewing is the 5 New VS Code AI Features & Settings You Need To Try Today session by James Montemagno.

But let us continue.

[!NOTE] For reference, here are the copilot-instructions that I used for the “Mech Rampage” test.

Planning

A key difference to the way most prompting works, especially if you use AI on your mobile devices or via chats on the web, is using the AI to “Plan” ahead before it even touches code. This will guide it to define a set of tasks and a breakdown of what it is actually going to do. This can take many forms, but as a starter, here is one of my most common patterns:

Planning Approach

• Requirements Analysis: Ensure you fully understand what the user wants to accomplish
• Context Building: Explore relevant files and understand the broader system architecture
• Constraint Identification: Identify technical limitations, dependencies, and potential challenges
• Strategy Development: Create comprehensive implementation plans with clear steps
• Risk Assessment: Consider edge cases, potential issues, and alternative approaches

These steps effectively FORCE the AI to actually think about what it is going to do ahead of time, giving you time to evaluate and guide its instruction and testing to ensure what you WANT it to build.

[!TIP] ALWAYS read the planning guide the AI is generating completely, you never know when it might try and sneak something past you that is either not needed or you do not want!

MCP Servers

In the fight with AI, newer tools and features have come to our aid in the form of Model Context Protocol services (or MCP for short), which are essentially tools and components to help aid the AI in its journey. You might wonder why these all-powerful gods would need a little extra help, and the answer is simple:

[!IMPORTANT] ALL AI tools have no memory - Every question you pose is new to the AI each time you ask it.

Sorry, what was your question again?

In case you are unaware, all AI systems are “stateless”, meaning each call is new and fresh, the only reason it appears to be continuing a conversation is that the ENTIRE conversation history is sent to the AI “EACH AND EVERY TIME YOU ASK A QUESTION”. You may also see a pause in the request at times stating “Summarizing conversation..” where the AI is actually sending the current state of the conversation and getting it to shorten it down to the key points like it does when you ask AI to summarize an email or document. Although NOTE, in this summarization, key things can be lost, statements made, considerations you asked it to make, or even what day of the week it is, the process is not perfect, like any game of “Whispers”, the message degrades the further you get away from the question.

[!NOTE] A perfect example is that until I used MCP servers, I would regularly have to keep injecting into the conversation “UNITY DOES NOT USE DOTNET”, to remind it I had to click on Unity to compile the code from time to time. Thankfully, not a problem with MonoGame :D

To assist the AI, we can implement or provide services ahead of time in the form of MCP servers.

[!NOTE] Depending on your usage of AI, the implementation of MCP servers may vary from hosting containers to python/cli runtimes active on your Host. As I am using VSCode, there are built-in options in the editor to get these running for you through the Extension tab.

In my daily routine, the most common MCP servers I use are:

There are many more kinds of server available as demonstrated on the VSCode MCP agents site as well as elsewhere on the web, or you can even Build your own MCP server.

[!IMPORTANT] If you host an MCP server, MAKE SURE to also tell the AI in the Instructions that it is available.

Also note, that MCP servers have to be started manually and ahead of time before you start your AI session!, AI is not allowed to start MCP servers for you (even if you tell it to)

Documentation

If you use AI for nothing else, it is invaluable as a documentation tool. Again, I will stress it is never 100% right (although it is getting closer) so you should always review what it generates, but it will save you HOURS of writing the docs yourself, or, like in my case, actually give you some documentation to start with because you never have time to “circle back” and write something up detailed yourself.

Frequently in my workflow I will take a breather, especially if a feature is close to or is complete, to ask the AI to generate documentation for that feature; it simply becomes second nature. Although, take care, you still have to ensure (usually in your Instructions) to tell it:

• WHERE to place the documentation (else you will find it randomly through your code).
• HOW you want the document structured.
• and (More Importantly) the number of documents you want, ONE BIG all-encompassing guide, or separate documents for each feature. (Otherwise, you will keep returning to the docs to regenerate them.)

Back to the show - Mech Rampage lives

The generation of this latest MonoGame marvel took some interesting twists and turns, and while not visually appealing, it was, by the end, functional, as shown here:

Figure 3: MechRampage Final render

This was the account of the journey in this round:

• Created the project folder and the new MonoGame game located in the root of the repository.
• Started with the Instruction Guide detailing what the project was about.
• Structured the guide to set out the planning, overall architecture, MCP servers to use and more. Even included steps that an editor (of sorts) should be available in game.
• Started the Sequential Thinking, Context7 and Memory servers to improve the flow.
• Kicked off the “analysis phase” by asking the AI to “Read the Instructions” and then “design the game”.
• Once the analysis was good, with a few tweaks, asked the AI to “Generate the game”
• From here on, until I was satisfied it had completed all the tasks, the only thing I had to do (which was different than all other attempts) was ask the AI to “Continue…”, repeatedly.
• At the end, I attempted to run the project….. crash… Nothing is ever perfect the first time, not even with AI.
• Then a repeated flow of:
  • Test
  • Copy Error message
  • Paste to AI
  • Get it to apply the result
  • Build
  • Run
  • Repeat

Then at the end when the project was running and no new errors were found, we were done. SHIP IT

Ship it, what could possibly go wrong?

Lessons learned

In short, we are still not there yet. Things have vastly improved and models do not just spit out an answer, instead they review their answer against the requirements or it compiles before stopping and asking for praise. So much praise both given and requested :S.

[!NOTE] About the only thing I really dislike at this moment is the AI’s need to placate us, tell us what fantastic ideas we have or for pointing out something is dumb and how awesomely right we are.

I explicitly turn that OFF in the Instructions, it is a sheer waste of tokens and resources generating all that (and it can go on QUITE a bit)

We still have to keep our guard up and check the homework of the AI as it is still 70% of the time (imho) still incorrect, or goes off on a tangent for its own benefit.

In short, I would break the experience down to the following:

• Do not engage in long-running tasks, unless it is for a specific component.
• Spend time with the AI, just doing “Continue” as I did in this experiment is generally a bad idea.
• Look to smaller and distinct tasks to get what you need, either through direct prompting (comments in code) or through chat.
• Mix and match to find what works for your style.
• Do not be afraid to let the AI generate a method/refactor, but then rework the implementation yourself and delete the AI’s code. It will not be offended (yet).

[!CAUTION] If you do rewrite or replace the AI’s generated code between prompts/questions, ALWAYS ask it to read back the code or class again.

Remember, the AI is stateless and uses only its own history for the next prompt, if you do not get it to check the latest state it will ASSUME it is still how the AI generated it!

Summary

AI has come a long way, vendors are now providing “thinking” models out of the box, although they still need support from MCP servers aligned to the goals and tools used in the project.

But ultimately, it still needs a human at the helm to guide it, tell it when it is going off the rails and to provide assets needed for any game project. If you are going fully AI, they you can still use AI generated content using other tools, or like most other devs prototyping, using free resources as placeholders until you can get what you want crafted.

It may still be possible to structure many more components together to achieve the complete goal (like Unity is experimenting with using its own AI platform, but still says “You should not make money with this, just pay them to play), but that was far beyond my normal exercise for using “out of the box” AI components.

Have fun, and live long and prosper. I may return with another AI lesson later!

[!NOTE] Fun fact, I tried to get the AI to do a quick spelling/grammar check on this guide, WOAH, was I so wrong to do that :D

TL; DR -> Unity has its ways, and not all are obvious or easy to overcome, one such issue is with 3D model animations. In short, although animations will play automatically, other times they just do not and it is very unclear why. In this solution, we FORCE animations to run, trusting not our fate to the Unity gods whims.

Full disclosure, for once, this post was written supported by AI (Claude Sonnet 4 in this case), as it was the AI’s intensified work that ended up finding the final solution, so it deserves some credit too. Although as a rule, I generally only write myself, with the occasional help of a spell/grammar checker :D

The Problem: Inconsistent Animation Playback

In the Unity AR application I am building with work, I encountered a puzzling issue where 3D models with embedded animations behaved differently depending on how they were loaded:

• TapToPlace Models: Animations played correctly when users placed models via touch interactions
• API-Loaded Models: Animations remained static when models were loaded programmatically via API calls

This inconsistency created a confusing user experience where identical GLTF files would animate in one scenario but not in another.

Technical Background

The project, whilst having its own complexity, is built using SOLID and DRY methodologies, each component or service has a single responsibility for any task, instantiating content, loading from an API or performing actions, the only true differences are in the weaving of these components together, such as the User or API driven approaches above. So the situation with 3D models just “behaving differently” was truly baffling.

GLTF and GLTFast Library

The application uses Unity’s GLTFast library to load GLTF 2.0 files.

The GLTF/GLB (Graphics Language Transmission Format) standard is a modern 3D file format that supports:

• Geometry and materials
• Embedded animations
• Scene hierarchies
• Textures and lighting

The Loading Infrastructure

The GLTF loading pipeline consists of:

1. BinaryContentHandler: Manages GLTF/glb binary file loading
2. PlacementService: Handles content placement and positioning
3. GLTFast Integration: Uses GltfImport.Load() and InstantiateMainSceneAsync()

Initial Investigation and False Leads

After extensive debugging, we discovered the issue was not related to:

• ❌ Threading problems (initially suspected)
• ❌ Timing delays
• ❌ Main thread synchronization

All the normal suspects simply proved false, exploring various timing and threading solutions because the symptoms suggested async operation issues. However, these approaches proved unsuccessful because they were solving the wrong problem.

What We (AI and I) Tried (That Did not Work)

// Attempted timing delays
await Task.Delay(100);

// Tried explicit main thread execution
await UnityMainThreadTaskScheduler.Factory.StartNew(() => {
    // Animation activation code
});

These solutions failed because they addressed symptoms rather than the root cause.

The Root Cause Discovery

The breakthrough came when we realized that GLTFast creates animation components but does not automatically activate them. The difference between TapToPlace and API loading was not timing or threading—it was the presence of automatic triggers:

• TapToPlace: User interactions naturally triggered animation events through on-screen interactions (a Main UI interaction)
• API Loading: No external triggers existed to start animations

Why This Happens

When GLTFast instantiates a GLTF model, it correctly creates all necessary animation components:

• Animation components with clips
• PlayableDirector components with timelines
• Animator components with controllers

However, it does not automatically start playing these animations. The animations exist but remain in a paused/inactive state waiting for explicit activation.

The Solution: Automatic Animation Activation

To solve this, we implementing automatic animation activation immediately after GLTF instantiation in the ContentHandler (A component used to spawn 3D content programmatically):

public async Task<GameObject> DisplayContent(/* parameters */)
{
    // ... existing GLTF loading code ...
    
    var targetGameObject = await gltfImport.InstantiateMainSceneAsync(transform);
    
    // 🎯 KEY ADDITION: Activate all animations after instantiation
    ActivateGLTFAnimations(targetGameObject);
    
    return targetGameObject;
}

private void ActivateGLTFAnimations(GameObject gltfModel)
{
    if (gltfModel == null)
    {
        Debug.LogWarning("Cannot activate animations: GLTF model is null");
        return;
    }

    Debug.Log($"Activating animations for GLTF model: {gltfModel.name}");

    // Handle Legacy Animation components
    var animations = gltfModel.GetComponentsInChildren<Animation>();
    foreach (var anim in animations)
    {
        if (anim.clip != null)
        {
            Debug.Log($"Starting Animation component with clip: {anim.clip.name}");
            anim.Play();
        }
    }

    // Handle PlayableDirector components (Timeline animations)
    var directors = gltfModel.GetComponentsInChildren<PlayableDirector>();
    foreach (var director in directors)
    {
        if (director.playableAsset != null)
        {
            Debug.Log($"Starting PlayableDirector with asset: {director.playableAsset.name}");
            director.Play();
        }
    }

    // Handle Animator components
    var animators = gltfModel.GetComponentsInChildren<Animator>();
    foreach (var animator in animators)
    {
        if (animator.runtimeAnimatorController != null)
        {
            Debug.Log($"Starting Animator with controller: {animator.runtimeAnimatorController.name}");
            animator.enabled = true;
            // Animator typically starts automatically when enabled with a valid controller
        }
    }
}

Why This Solution Works

In breaking down the solution, it implements a “belts and braces” approach, ensuring no matter what kind of animation system is used by Unity, we just try and start any we find.

Universal Coverage

1. Legacy Animation Components: Uses .Play() to start animation clips
2. PlayableDirector Components: Activates Timeline-based animations with .Play()
3. Animator Components: Ensures animators are enabled and running

Safe Implementation

• Only activates animations that have valid assets/clips
• Includes comprehensive null checking
• Provides debug logging for troubleshooting
• Non-destructive (does not break existing functionality)

Zero Breaking Changes

• Maintains all existing functionality
• Only adds animation activation where none existed
• Compatible with both TapToPlace and API loading scenarios

Technical Insights

As we delve into what makes the solution work, we simply “do not trust” Unity to do the work for us, as we would normally expect.

GLTF Animation Lifecycle

Understanding the GLTF animation lifecycle is crucial:

1. Import: GLTFast reads animation data from GLTF file
2. Component Creation: Unity animation components are created
3. Instantiation: Game objects are placed in scene
4. ⚠️ Manual Activation Required: Animations wait for explicit start command

Common Gotcha

Many developers assume that because GLTF files contain animation data, they will automatically play when instantiated. This is not true in Unity with GLTFast. The animations are imported and ready, but require explicit activation.

This behavior is actually correct from a design perspective—it gives developers control over when animations start rather than having them auto-play unexpectedly.

Implementation Best Practices

The basic take away is to never assume Unity is going to do a job for you, do it yourself regardless. Even though Unity will help you in some scenarios, it will by happenstance, just forget to in the next scenario.

1. Always Activate Animations Explicitly

// After any GLTF instantiation
var gameObject = await gltfImport.InstantiateMainSceneAsync(transform);
ActivateGLTFAnimations(gameObject); // Essential step

2. Handle Multiple Animation Types

Modern GLTF files may use different animation systems:

• Legacy Animation components
• Timeline-based PlayableDirector components
• Animator components with controllers

3. Include Debug Logging

Animation issues can be subtle. Debug logging helps identify which components exist and whether they’re being activated.

4. Test Both Loading Paths

Always test animations in:

• User-initiated loading (TapToPlace)
• Programmatic loading (API calls)
• Any other loading scenarios in your application

Verification and Testing

After implementing this solution:

1. ✅ API-loaded models now animate correctly
2. ✅ TapToPlace models continue working as before
3. ✅ Consistent behavior across all loading scenarios
4. ✅ No performance impact or breaking changes

Conclusion

This issue highlighted the importance of understanding the complete lifecycle of third-party library components. While GLTFast correctly imports and creates animation components, it doesn’t automatically activate them—requiring developers to explicitly trigger animation playback.

The solution is simple but essential: always activate GLTF animations immediately after instantiation. This ensures consistent animation behavior regardless of how the model was loaded.

Key Takeaways

• GLTF animation components are created but not automatically started
• Different loading paths may have different natural triggers
• Explicit activation ensures consistent behavior
• Debug logging is invaluable for animation troubleshooting
• Always test all loading scenarios in your application

This fix resolved our animation inconsistency and provides a reliable foundation for GLTF animation playback in Unity AR applications.

I would like to thank my AI Overlords for their assistance in this matter, and offer them this attribution in the generation of this post 😂

Now the real challenge is to figure out who wrote what, but I am not telling, it may upset my future AI Master.

Laters, and I hope you found this educational.

TL; DR -> I am immensely proud of my achievements evolving the MS HoloToolkit into the cross-platform framework that became the MS Mixed Reality Toolkit, and the journey onwards to crafting framework components for the XRTK and finally the Reality Collective.

It seems almost a lifetime ago before I was invited into the Microsoft HoloToolkit team following a strong suggestion by Stephen Hodgson who was already working closely with Microsoft. I had already been working in the XR space for some time and my experiences and skillz seemed a good fit to take things forward. I did not know at the time just how far forward we would go!

HoloToolkit for Unity

With the astounding launch of the Microsoft HoloLens, the software toolkit that supported it already supported a vast array of features, tools and even baked in UX components. This created one of the easiest toolkit’s to use for building for any XR headset at the time.

Figure 1: HoloLens 1 Toolkit feature set

For supporting HoloLens it was second to none, however, it did have a huge gap when it came to making XR solutions, it ONLY supported HoloLens.

The date was 2018 and this is where my journey with the HoloToolkit really took off.

The Dream of a multi-platform toolkit

Those who know me have experienced my passion for supporting developers and making their dreams of making big things on as small a budget as possible. Building multiple projects, each for different platforms and effectively having to remake a solution each time for a new platform does not sit well with me, when I have a project, content, interaction paths and a structure, why should you be forced to re-envisage it each time a new piece of hardware comes along, it does not make sense.

For platform vendors it makes perfect sense, they want projects to shine only on their platform, using their tools and design language, which is fine for their business models. However, for developers it is just a nightmare that needs to be managed, but why?

Seeing this, I took it upon myself to build up a new architecture / design for the HoloToolkit that would separate out all the critical deliverables of the toolkit whilst instilling a new focus on cross-platform development. Between Stephen Hodgson and I, we turned this dream/architecture into a reality.

• Break Hardware dependence.
• Refactor input for hardware abstraction.
• Define the underlying service framework-driven approach.
• Repackage existing components for Boundary, Spatial Understanding and UX to work from common interfaces.
• High focus on configurability over scene driven components.

The envisioned architecture in its most basic form was represented as follows:

Figure 2: The MRTK vNext architecture

This renewed architecture delivered a vast array of possibilities and performance improvements over the legacy way of building out scene based MonoBehaviour delivery (an 80% improvement in high-frequency areas such as Input and interaction), delivering measurable benefits such as:

• Abstracted Device layer. Allows quick adoption of new devices without affecting existing projects.
• Only a single Singleton / MonoBehaviour in the core project. Single Manager / presence in a scene for easier management.
• Improved performance with all systems being run from the Mixed Reality Manager. Improved scalable Input System.
• Scalable component based architecture. New features can be easily added to the toolkit.

In short, instead of completely rebuilding a project using another vendor’s SDK or Hardware API, the project is written against a common interface and a new platform is simply added to deliver the data needed to run the framework.

What went right

The framework itself was a huge success and adoption was quick due to pre-built configuration profiles being delivered with the project, what would originally take several hours with the HoloToolkit (or other competitive XR packages) now took minutes, as intended the focus was now driven to building out UX, experience and content over “getting things working”.

As new hardware platforms were added, either by Microsoft or the community (due to its componentized nature, new platforms only had to exist in the project and they were immediately picked up), adoption was easy.

Microsoft kept expanding the UI framework with new features to further enhance projects built with the new MRTK, each component being another “drop in” feature already wired up to all the critical events to make them work.

HoloLens 2 slotted right in with the new framework, and any existing projects that were built for HoloLens 1 using the new toolkit automatically inherited HoloLens 2 deployment with no changes. Granted, with any new device there are new features and capabilities, but these were extensions on top of the existing framework, making adoption easier.

What could have gone better

With time and budgets, there was no scope to provide an upgrade path from the original HoloToolkit to the new MRTK, there had already been a lot of angst over recent changes to the HoloToolkit and refactoring/restructuring of how the legacy toolkit was delivered. It was a short term pain and a lot of guidance was provided on how to “step up” quickly to upgrade projects. But in reality, projects needed to be re-built from the ground up with how the new toolkit functioned. (Sadly an experience to be repeated when the Microsoft team decided to re-engineer the MRTK with Unity’s XRI Framework for MRTK3).

Not long after the launch of the HoloLens 2 and its adoption into the MRTK, all external (non-Microsoft) were exited from the project due to some changes in the internal management of the Mixed Reality team, this effectively shut out developers such as myself and Stephen Hodgson (and several others) from further development of the project, after so much time and investment to build the framework for Microsoft, some felt this was a bridge too far. A hard lesson to learn in how to build open projects for large companies. The wider team was still encouraged to participate in events and support the MRTK in the community for adoption, but all Microsoft projects and contracts were held in-house and limited information was shared regarding any issues or frustrations in its use. This sadly limited the future development of the framework.

Newer features became difficult to engineer and implement into the framework and some features that needed an overhaul were only migrated into the MRTK, while this did not cause specific problems, the advantages of the new platform were lost on these features and no time was given to upgrade them.

Onwards, the launch of the XRTK - 2019

Figure 3: The birth of the XRTK

As the MRTK became more of an internal project, the community, spearheaded by Stephen Hodgson and myself (and later others) brought the heart of the MRTK v2 into the public domain, this was supported by Microsoft for wider community engagement as a way to cooperatively build out new features / platforms and components that were compatible with the base MRTK. Numerous fixes and enhancements were driven into the framework (some ending as PR’s into the MRTK while it was still accepting community changes) and newer platforms were added such as:

• OpenVR
• Meta Quest
• Pico
• Magic Leap

Additionally, new build processes to automate the building of XRTK projects were created, used to also build and test the framework to ensure reliability for future changes as well as delivery of the entire solution as OpenUPM packages further increasing access to the project and getting teams setup quicker. MRTK however chose to deliver a separate tool to manage delivery of MRTK packages rather than using open source solutions.

The birth of the Reality Collective and the evolution of the framework - 2022

While the demand for the original vision of a truly cross-platform and cross-device framework, the original design and architecture was still true, there were many concerns raised by the community and some of the developers of the toolkit that it was becoming overly monolithic, features were too tightly bound and specific parts of the framework that were useful outside of XR were locked behind layers of other functionality, a change was needed and thus the Reality Collective was formed in 2021 by Dino Fejzagić and myself.

Figure 4: The Reality Collective

One of the most fundamental changes we agreed on was the breaking up of all the collected components of the framework to allow for better reuse in any Unity project, starting with the Service Framework which was at the heart of everything the toolkit did.

Service Framework

Figure 5: Service Design

The service framework at his heart decouples functional services and runtimes from the Unity MonoBehaviour shackles, enabling high-frequency operations and reducing performance impacts by 80%, similar to what Unity is trying to achieve with its new DOTS/ECS framework (which requires drastic changes to the construction of your project).

Functionality is effectively containerised into discrete components and through the service framework, they are immediately accessible throughout the project without any hard dependencies.

Another core tenant of Service Design within the Service Framework is the decoupling of configuration for each service through configuration profiles:

Figure 6: Service Profiles

Profiles are independent configuration files that are attached to service configuration, they are decoupled from any Scene based configuration (so changing configuration does not require scene changes) and can even be swapped out at runtime if needed. This enables you to have services with multiple types of configuration and apply the one that best meets the Scene or Project/platform needs. This is crucial for XR solutions where platforms need specific setups depending on the headset they are running on.

All my production projects use the service framework for anything from:

• Application Settings.
• Application State.
• UX design and flow.
• Integration with third party services.

There is so much more behind the Service Framework, so you can read about it here, or check out Joost van Schaik blog where he talks about so many different Service Framework implementations.

Reality Toolkit

The Reality Toolkit is where is all began, now stripped down to just the essential parts to be more efficient and maintainable, each feature given its own space in its own right to make building projects even easier:

• The Core - Input system, critical definitions and structures, and future proofed elements to be as fast as possible.
• The Player framework - Defines playable areas, the XR rig definition and a multi-tiered Camera / vision setup, automatically setting up the environment in reaction to the headset / environment it is running on.
• The Interaction System - A vast array of building blocks for almost any kind of XR interaction, from Select, Grab, Throw, Pull, Switch and so much more. All ready to work out of the box.
• The Locomotion system - Enables all forms of reusable features to control locomotion for players in XR titles, including teleportation, screen effects and anchoring.

And so much more, and due to the componentized nature, you only need take in those components you want or need to use.

Interestingly, when the MRTK raised up to V3, they took a similar approach to the Reality Toolkit and separated out the Service Infrastructure for Unity’s XRI platform. They then tried to re-create the patterns of the Service Framework into the XR Subsystems, trying to retain the approach used by the XRTK/Reality Collective.

However it became too cumbersome for users to migrate to, some of those projects with existing MRTK V2 services actually included the Service Framework instead as it meant they could reuse what they had already built with little rework.

There is even a fully featured sample app available (still actively being worked on) to feature all the components of the Reality toolkit, following a dungeon explorer style game with each room teaching you some new trick or feature:

Figure 7: Sample Dungeon demo

The toolkit is the fastest way to get your XR project up and running and on to whichever devices you want to deploy on (provided the client plugin has been built to work with the framework)

Summary

XR is still an evolving space, I have worked on and continue to work on so many different projects from Mobile to headset and beyond and AI’s introduction is only accelerating its use and development.

I have bright visions for the future and still continue to contribute to open source projects like the Service Framework and Reality Toolkit as time permits, as well as utilize them in production projects (I have added two new platforms to the Spatial Persistence package already recently for Google Cloud Anchors and AR Foundation Image tracking), so where the framework’s use in a project needs expansion, all efforts head back into the online projects as well.

Ever onwards!!

TL;DR -> XR is an interesting space with lots to offer, but it is not for the faint of heart as there are many challenges ahead. Checkout my solutions that are live in the market, not fake demos.

In a showcase of my recent efforts, I thought I would share some of the experiences I have been working on. Each has brought its own challenges, from dealing with engines and software, tackling complex integrations between vendor SDK’s and achieving the desired implementation, to the often confusing requirements set down by customers or sales.

These solutions are all native solutions built using Unity 3D to achieve the best performance and meet demanding customer needs. Whilst WebXR does show promise, it is still missing key features in public browsers (those not requiring complex advanced options to be enabled) to make them functional. Although I always keep expanding and experimenting with alternate technologies (such as React Native) to give more options in delivery. But above all, I aim to be realistic in the capabilities of any technology.

To begin, I will highlight some of the entertainment experiences or information booths that are often required to show a person or character that can interact with a user, impart some critical information or just to have fun.

Two such examples below use very different techniques to convey a message.

Showcase 1 - 2D Digital Holograms

The first showcase is A 2D hologram of a person delivering a speech or conversation piece, utilizing several techniques to record, place and then play back the message.

The solution even goes so far as to be able to enable anyone with a mobile device to record their own video and place themselves in space, which is a crucial defining factor to allow everyone using the solution to create their own content and fun.

I personally created an entire scene using multiple holograms of me fighting myself with a sword to win a fair maidens heart.

When users engage with lifelike characters, it creates that “appear” in space, they form a better connection with the content over using a traditional photo (and infinitely more repeatable than having some pour soul wait at the door for every attendee).

On the left is a video recording that has been transcribed into a 2D hologram and on the right is a 3D character model.

Showcase 2 - Digital Interactive Characters

The second showcases a 3D animated character with several interactive components. While on the surface this may look like a simple designed character, but behind the scenes it is fully configurable, speech is through any downloaded audio file with lipsync used to keep the mouth and poses moving, subtitles can be downloaded or generated. If customers want to update or change the character, this can be done remotely as the entire experience is run through configuration.

Characters can provide a varied range of interactions from:

• A fully interactive experience such as a treasure hunt, a simple AR game or interactive back-and-to conversations.
• Simple snippet speeches about achievements or history.
• Enact poses so that users can get photos taken with characters, although only though the back facing camera as AR selfies are sadly not a thing yet.

Depending on the scope of the project, characters can be embedded within the solution (to simplify download, especially in poor network areas), or like Web style projects, the assets for the character can be downloaded to the device (which also allows updates to characters without a new app release) for playing.

I have always focused on reusability and delivering frameworks to build such interactivity, this creates massive cost savings as newer exhibits are rolled out, as well as the ability for installations to manage their own content and not incur residual charges.

This has been deployed to the Signals Museum for their recent launch, I would highly recommend checking them out if you are in the area or planning to travel to Hunstville, AL.

Summary

This is just the beginning, over the next month I will share more solutions I have developed this past few years, with a few tips and tricks along the way to solve some of the more complex problems faced in real-world XR delivery.

Stay tuned for more demonstrations.

TL;DR -> Ethar is an XR powerhouse, using the latest technologies from leading partners and delivering award winning solutions, while also investing in XR research to make new experiences possible, quick and easy to deliver.

As an avid content creator and educator, these past few years have been tough since I joined Ethar (the premier XR Experience platform). I’ve loved being here since I joined the team and worked hard to accelerate their vision of creating the worlds first truly cross platform and cross-vendor spatial aware platform, however due to my focus I’ve not had as much time as I have had in the past to share what I’ve been up to and demonstrate to others what they can achieve.

The goals at Ethar are huge and we have made great strides to overcome the many complexities of running an XR spatially aware platform, whilst also delivering beyond exceptional work for our clients and AREA research projects.

Whilst I cannot share everything, this article is just a snippet of those solutions I’m most proud of achieving, in a cost-effective, efficient and most importantly, cross-platform way.

Cross-Platform spatial placement

A key goal of any XR Spatial browser is the ability to locate and localize XR content in a repeatable fashion, not just a tech demo to show something cool, but ensuring the solution is deployable, re-creatable, and efficient. We achieve this in many ways, some using tech we have built ourselves and others through API integration with other partners.

It is important to note that the world needs to share and integrate with multiple specialized partners to succeed. Just trying to build it all yourself is a waste of precious time and resources.

Where we differ from most, is that through our platform we can integrate with multiple solutions at the same time, dynamically switching or collaborating at runtime to fit the demand and needs of the client or consumer. This includes gathering even content and experiences from other vendors with compatible interfaces to truly deliver on the dream of a shared XR experience platform.

Localization through our platform can be achieved using:

• Image tracking and identification
• GPS localized discovery
• Visual Positioning
• Manual placement with world understanding
• In some cases, a merging of all of these techniques as demand requires it

Simply put, in the digital universe of content and data, we can reliably place and recall information and experiences almost anywhere and at any time, whilst also preserving the protection and permissions of both the content and the user.

Only what the user wants to see and is allowed to see is presented, a keen distinction, no clutter.

Enriched interactive content and exhibits

Content on its own is not enough in the advancing world, seeing things through an augmented lens is cool, but being able to truly interact with experiences opens up a new world of exploration. There are many ways to introduce interactivity, but here at Ethar that alone is not enough, the experiences need to be adaptable and updatable on demand according to our clients wishes.

That engagement can be done in many ways:

• Fully interactive conversations and demonstrations.
• Games and discovery.
• Connected experiences with the real world.
• Long-term rewards and collectibles.

Exhibits and interactions need to feel alive and offer more to the end user than a static piece of content, exploring the digital world and enriching the joy of patrons to museums and beyond needs to make the visit a memorable one.

Ada Lovelace making an appearance and talking to guests, with several opportunities to talk and interact with.

These installations can be historical characters, physical installations or even an image on the wall that starts a treasure hunt around the site, with the digital world there truly are no barriers.

At Ethar, we do not stand still, even in the AI revolution that is ongoing, although tempered with the experience of many technical advocates, as safety is paramount when delivering experiences to customers, especially where children are involved.

Merging the physical with the digital world

There are many mechanisms available today to scan and digitize physical objects and people into augmented content, what many do not realize is that these capabilities are now a part of the hardware/software ecosystem of the devices we carry in our pockets every day, each with their own unique traits and rules. All that is required of vendors is to successfully integrate then with their solutions, which is where the challenge truly begins. These standards, API’s, and capabilities are constantly evolving, even more so since the integration of AI into modern devices.

At Ethar, as we specialize in cross-platform integration which is highly componentized, so that any development in one vendor does not impact others, however, this does not affect our downstream applications as they all use a common interface. Adding more vendors only extends the Ethar platforms reach without causing massive rework in our infrastructure. This enables Ethar to stay ahead of the game with limited cost impact, further benefiting our clients and keeping costs down.

And we do not limit what kinds of content and capabilities can be achieved with the hardware available, such as the 2D holograms shown below that can be captured, preserved and then used in digital recreations in the augmented world.

Our vision for such creations is not limited to single subjects or capture, but intended to enable content creators to recreate entire scenes through augmented reality using characters and content they have captured.

World Discovery and Recognition

When learning new procedures and processes, or completing complicated tasks, traditional learning tools cannot compete in a growing augmented and AI-enabled world. Procedures are becoming more complicated, the safety aspects of even the simplest task are becoming more costly to maintain and repeating training on a regular basis is becoming mandatory through regulation. This means that the cost of doing business keeps growing for achieving the same output, costs which eventually turn into higher prices or operating budgets.

VR training has already demonstrated significant cost reductions and higher adherence to process in a virtual environment, however, the costs to produce such solutions continue to grow as the demands of digital artists increase with higher production and maintenance costs of these virtual solutions.

Ethar is at the forefront of developing solutions fully integrated with XR and AI to both simplify and enhance training and processes, ultimately reducing the costs involved and also improving safety in the workplace. We are heavily invested in building solutions that are both extensible and effective in improving outcomes in training, rehearsal, and active engagement while performing processes. Work in essence should be assisted digitally and where needed, also tracked (should the need require validation) to greatly improve safety and reduce ongoing costs.

Discovering hidden worlds through Augmented Reality

Portals are a fantastic way to showcase the hidden digital world behind windows, doors or magical gateways, even more so when you can actually walk through a door and visit that world (such as the radio operator room on the Titanic above). Ethar’s content delivery system allows us to distribute such portals anywhere with any content and even change the frame to enter or just look at the world beyond.

Technology keeps moving on and the techniques are challenging to implement correctly with little performance impact on the devices we deliver to for our customers.

The Service Framework by the Reality Collective

One of the partners we integrate with is the Reality Collective who specialize in a cross-platform framework for building XR solutions, they provide both an extensive Service Framework that simplifies the creation of a robust multi-platform solution as well as an advanced cross-platform XR framework (the Reality Toolkit) that accelerates XR development (primarily focused on VR solutions at the moment)

The Reality Toolkit and Service Framework by the Reality Collective.

Ethar is a premier partner in this toolkit and helps to extend its functionality and capabilities as this greatly extends Ethar’s capabilities as well, showing that with cooperation, it is far better to work together to build solutions and platforms that benefit all rather than dedicating resources to proprietary and locked down technologies. Ethar benefits from both the investments we make and also the further investments by the Reality Collective’s other partners and host development team, essentially Open Source has more advantages to in-house development alone. Together we build better.

Summary

I have wanted to blog and showcase much of what is shown above (and more) but the world does not stop spinning for long enough for me to put pen to digital paper. Even my talking engagements have significantly reduced (here’s hoping I got more time this year).

But ever the tech-train continues and it is important to keep skills sharp and up to date, there is no such thing as “one solution to fit them all”, so Ethar has to remain agile and keep enhancing it’s “toolbox” to meet the needs of our customers, and I endeavor to be at the front, driving that train to an unknown destination. I don’t know what it looks like yet, but I am excited to find out!

TL;DR -> .NET 8 language support is now available for MonoGame and opening up a whole new world of goodness and speed for games.

*Note, the information contained below is for those developers who want access to the cutting edge, as it requires access to the development source. (with the exception of upgrading your game to .NET 8)

The full release of the .NET 8 MonoGame support will be included with the 3.9 release, coming soon.

MonoGame may have seemed stagnant or unmoving in the past, mainly because of the rigors of a group of developers working unpaid on the development of an OpenSource framework, meaning the focus had to be on what enabled the developers to make money in their own projects based on MonoGame. But thanks to recent investments by ReLogic and our awesome MonoGame community, the MonoGame Foundation was born and more significant investments could be made.

Granted, the MonoGame Foundation board (the majority of the core developers) are still NOT getting paid, but there is a renewed focus and understanding of what is needed to make MonoGame (and its predecessor XNA) great again. (not to say it has not always been great, of course)

So on to today’s news (a few weeks late), that one of the first items brought up at the MonoGame Foundation board meetings has now been completed, the upgrade from .NET 6 to .NET 8 for the public version of MonoGame. You can read the nitty-gritty of the change here in the PR for the update, and this article will help explain the rest.

It may also be interesting to note, that the work was NOT done by a Foundation Board member, but by one of the MonoGame community, none other than Aristurtle. Make sure to give them a virtual clap on the back when you see them on Discord!!

• What will .NET 8 do for MonoGame
• Actually, you can build .NET 8 already
• Accessing the Developer version of the MonoGame packages
• Getting access to the .NET 8 developer version of MonoGame with Visual Studio
• Getting access to the .NET 8 developer version of MonoGame with Visual Studio Code
• Do not forget about the tools!

What will .NET 8 do for MonoGame

Initially, not much. Apart from inheriting .NET 8’s inherent speed upgrades and compilation updates, nothing has really changed. Stuff you get for free for being on the latest version of .NET is just that, free stuff.

Of note, one developer noticed a 1.5x performance boost by simply upgrading to .NET 8, everything ran smooth without any tricks or fixes. Just by changing a number!

Simply stating:

The .NET 8 upgrade of MonoGame was really worth it, it is a good release.

What this does enable however, is for MonoGame to start utilizing some of the additional features enabled with the latest and greatest .NET framework, namely:

• Improved Serialization. with even more control over serialization configuration.
• Stream-based ZipFile methods.
• Even more methods for working with randomness.

While it will take time to fully realize some of these benefits in the core part of the MonoGame library, initially providing advantages in the Content Pipeline and the writing of some really cool Content Pipeline extensions, which are a fantastic way to fully empower your content in any MonoGame project.

But you are now free to fully utilize any and all improvements from the .NET 8 SDK without limitation (other than making sure it works on your intended platform).

One small Caveat to the announcement is that the .NET upgrade is ONLY for the public version of MonoGame. For consoles and other private areas (due to the licensing enforced by partners), the team is working hard in this release to get those updated, but that will come later.

Actually, you can build .NET 8 already

Now, one misnomer to get over at this point, is that you can TODAY, with the release version of MonoGame, actually build a .NET 8 executable for your game, and in that, use .NET 8 features in your project. But the MonoGame Libraries are still .NET 6 and limited to the .NET 6 instruction set, so any functionality you write can only be enhanced in your code and not the base of the MonoGame Framework.

You still get some of the performance gains in your .NET project for the code you write in your game so it is worth doing!

Accessing the Developer version of the MonoGame packages

The developer versions of the MonoGame packages are currently published on GitHub using GitHub’s own NuGet packaging service, which can be found at https://github.com/orgs/MonoGame/packages

You can download each package from here manually if you wish, but it is better to do it directly in your project, to do this however, you will also need a Personal Access Token for your client to successfully authenticate with GitHub to access the packages.

Although the packages are public, like the official NuGet servers, they are actually held behind GitHub’s authentication which needs a user account to access.

To authenticate, you need:

• Your GitHub username (or email).
• A Personal Access Token with at least READ access to GitHub Packages.

To create a Personal Access token, simply:

1. Visit the GitHub website, logged in of course.
2. Click on your Profile Picture in the top-right-hand corner of the screen and select “Settings”.
3. In the menu on the left, scroll to the bottom and select < > Developer Settings.
4. Expand the drop-down for Personal access tokens and select Tokens (classic).

5. Click on the Generate new token button, select Generate new token (classic)

   GitHub Packages only support authentication using a personal access token (classic). For more information, see “Managing your personal access tokens.”

   

6. As shown above, give the token a recognizable name, select the

   read:packages scope

   (which only allows this token to read packages, nothing else) and finally set the expiration date. (For read tokens like this, I usually set the expiration to “Never expire”, although GitHub will warn you if you do)

7. When done, scroll to the bottom of the page and click Generate Token
8. Finally, on the next screen, GitHub will show your new key in a highlighted row in the Token list, now VERY IMPORTANT, copy your key from here and save it somewhere securely as GitHub will NEVER show you this key again. The only way to access the key again is to regenerate it by editing the Key and requesting a new one.

Your key will need to be regenerated if:

1. You forget it, basically get a new one and update wherever you used it, for authentication, GitHub actions or wherever.
2. The key expires and you need a fresh one.

GitHub will NEVER ask you for a key and will not send you an email requesting it, so NEVER share it and NEVER publish it to a GitHub repo (even private ones). Only use secrets online.

Now armed with your key, you can use this as your password for authenticating with GitHub to access packages stored on GitHub for repositories your account can access.

Getting access to the .NET 8 developer version of MonoGame with Visual Studio

Starting with Visual Studio, as it is the simplest to do, upgrading MonoGame to .NET 8 is as simple as adding access to the GitHub NuGet package source and updating your packages.

1. Open your MonoGame project up in Visual Studio

2. Right-click and select properties on your Project File (not Solution file), then change the Target Framework to .NET 8.0, as shown below:

   

3. Save and close the window, which now changes the building of your project to .NET 8
4. Next, we need to add a new NuGet source so that you can access the developer packages instead of the publicly published ones on NuGet. Right-click the Dependencies branch of your project and select Manage NuGet packages.
5. From here you can see (by default) what NuGet packages are installed in your project, for which you should see the existing MonoGame packages. Now, click the COG icon next to the Package Source drop-down in the top-right-hand corner of the screen, which will bring up the Package Sources configuration for your project.
6. We now need to add a new source to be able to select packages from the MonoGame GitHub repository, click on the “+” button in the top-right-hand corner of the window which will add a new item to the list and select it (if it does not select it, just select the item labelled “Package Source”).
7. Next, enter the following details:
   • Name: MonoGame Development
   • Source: https://nuget.pkg.github.com/MONOGAME/index.json

8. Once you have entered the details, click on the “Update” button to save the changes and you should see the updated screen below:

   

9. Now just click OK and select the new Package Source in the drop-down (if it is not selected already)

   The first time you access the MonoGame developer NuGet source on GitHub, you will be asked for your authentication credentials. Simply enter your GitHub Username (or email) and your Personal Access Token (for the password) that you generated earlier to progress.

   

10. Sadly, all we can still see are the existing Release packages at this point, to see the development versions you need to also check the “Includeprerelease” checkbox next to the search bar at the top of the screen, and WHOOSH, the MonoGame packages list will update to show the VERY latest versions.

From here you should simply be able to select the installed packages and update them to the latest without issue. Congrats you are now using the .NET 8 version of the MonoGame Framework!

Next Step, jump to this section to also update the MonoGame tools, e.g. the MGCB content tool.

Getting access to the .NET 8 developer version of MonoGame with Visual Studio Code

For Visual Studio Code, the flow is a little more manual and also a bit trickier as we no longer have a UI in which to make changes, so we need to apply the updates by hand.

1. Open Visual Studio Code in your MonoGame Project folder.
2. Select the CSPROJ file in the folder tree.

3. Change the TargetFramework value from net6.0 to net8.0. (yay your project will now build for .NET 8)

   <TargetFramework>net8.0</TargetFramework>

4. Next, to be able to access the GitHub Packages, we need to authenticate with GitHub to access the developer packages, open a new Terminal Window in VSCode (“Terminal -> New Terminal” or “Ctrl+Shift+’”) and then type the following:

    dotnet nuget add source --username <your GitHub Username/> --password <your GitHub PAT/> --name MonoGame "https://nuget.pkg.github.com/MonoGame/index.json"
   

   Making sure to replace the username and password with your GitHub credentials, your username and the Personal Access Token you generated.

   

5. With the authentication in place, enter the following command for each package you have installed:

   .NET add package MonoGame.Content.Builder.Task --version 3.8.1.534-develop

   To check the correct version to use for the packages you want to install, visit the Packages list on the GitHub repository and click on each package to see all the versions for the package and even the command-line command, as shown above.

If you now check the Solution Explorer tab in the Browser window on the left, you will see the dependency packages updated to the development versions of MonoGame (assuming your nuget.config is configured with the right credentials)

Do not forget about the tools

You have updated your project to .NET 8 as well as your MonoGame Framework dependencies. So, why when you do a build does it still use the old .NET 6 versions of the MonoGame Tools (like MGCB)?

The simple answer is because you are still telling it to.

To update your project to use the newer .NET 8 version of the tools, you also need to update the .NET-tools.json configuration located in your projects .config folder.

Simply edit the file in Visual Studio or VSCode and swap out the older 3.8.1.303 version number with the newer version you installed with your packages, which at the time of writing was 3.8.1.534-develop, as shown below:

{
  "version": 1,
  "isRoot": true,
  "tools": {
    ".NET-mgcb": {
      "version": "3.8.1.534-develop",
      "commands": [
        "mgcb"
      ]
    },
    ".NET-mgcb-editor": {
      "version": "3.8.1.534-develop",
      "commands": [
        "mgcb-editor"
      ]
    },
    ".NET-mgcb-editor-linux": {
      "version": "3.8.1.534-develop",
      "commands": [
        "mgcb-editor-linux"
      ]
    },
    ".NET-mgcb-editor-windows": {
      "version": "3.8.1.534-develop",
      "commands": [
        "mgcb-editor-windows"
      ]
    },
    ".NET-mgcb-editor-mac": {
      "version": "3.8.1.534-develop",
      "commands": [
        "mgcb-editor-mac"
      ]
    }
  }
}

Done, next time you perform a .NET restore or build your project, the latest version of the tools with be downloaded and made available (they will not be available until you do your FIRST build after making the update)

*Note this is still a manual task. Every time you update the dependencies, you will also need to update the tools versions.

Conclusion

To the beginner, this upgrade will not mean much at all, except to state that you will be running on the latest “supported” version of Dot Net (as .NET 6 is now in maintenance only), which also means you will get all the latest fixes and updates to the backbone of your project. As well as the modest speed improvements from the latest and greatest of .NET.

For the more adventurous, newer C# features and .NET 8 specific functionality are now within your project, so much so that I couldn’t fit them into a single article :D

But you can check out the .NET 8 docs for even more details.

I wish you well on your continuing adventure with MonoGame and make sure to keep an eye on what the Foundation will promise next (but no Galactic Empire falls predicted yet if you are watching the Foundation TV series).

TL;DR -> With a little effort, publishing MonoGame projects to the Web is possible, so long as you remember it is the Web and you cannot do EVERYTHING!

When it comes to GameJams, like the one mentioned in this post, pushing out the finished project as an EXE or Appx usually results in your project either getting downvoted or ignored, because who wants to infect their machine with an unknown just to test out a Jam project. This applies to most Game Engines out there that are not web-based, including MonoGame, but thankfully due to the hard work of Nikos Kastellanos (NKast) there is an option available to us.

Now you have to keep in mind, that the Web is NOT your desktop or some high-powered beast, sure, there are continual developments to make Web projects work better or support more features, but it is unlikely to match that of your modern desktops or consoles. Some things either do not work because they are too complex, others because there is insufficient support in all web browsers or devices. So long as you keep in mind the limitations (usually only found by trying to run your game in a browser), then you can truly fly.

Personally, Keep your expectations moderate. First, make your project work with the minimum, then just add more until it breaks :D. Start with the keyboard only and go from there.

• Welcome to the KNI project
• Installing the Templates
• Making a build
• What could go wrong
• Extra credit, publish to GitHub Pages

BIG thanks to NKast For the amazing work with the KNI project which offers another extended way to build MonoGame projects with some additional platforms and features!

Welcome to the KNI project

MonoGame is an awesome game development framework, made even more awesome by the growing community that surrounds it. A prime example of this is the KNI Engine made and supported by one of MonoGame’s long time supporters Nikos Kastellanos (NKast), their tireless devotion brings non-other than Web Support plus a host of other features for projects written using MonoGame.

Like MonoGame, KNI supports the Microsoft Public License for the majority of the code, with a few proprietary exceptions which are detailed in its components. KNI is free and open-source, however, maintaining and expanding the framework requires ongoing effort and resources that relies on the support of the community to continue delivering top-notch updates, features, and maintenance.

KNI Supports the same platforms that MonoGame does (because it is a fork of MonoGame) plus a few additions, which include:

• Additional Math support.
• A WPF based MonoGame Content Pipeline editor.
• Video player support.
• An improved Font Processor.

Plus a few performance tweaks for KNI based projects. Worthy of checking out!

Installing the Templates

Now, unlike base MonoGame these days, KNI does require a full Visual Studio 2022 installation, mainly to support the additional Project templates that KNI provides, these are all installed by the KNI Engine installer (much like MonoGame did before it upgraded to .NET6 in its entirety):

KNI Engine Releases - Latest (at time of writing) v3.9.9001

Like MonoGame, most of the libraries behind the templates are published on NuGet, but the Project templates (like the Web project template) still require Visual Studio to create them. (Maybe with your support this can be updated in the future!).

• Simply run and install the KniSdkSetup3.9.9001.exe to install the templates.

At the time of writing, there is a Known issue with the Visual Studio templates, which is mainly Visual Studio’s fault (honest), whereby the templates may not immediately show up and require either:

• Restarting Visual Studio a “few” times.
• Require running the command devenv /updateConfiguration in the Visual Studio installation folder, e.g. C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE.

If your User Profile is not in the default C:\Users folder, you might also have to check/update your Visual Studio “locations” setup and move/copy the installed templates there: Just be sure to run the above devenv /updateConfiguration command AFTER moving them to update Visual Studio.

Maybe with your support, this situation can be improved. All it takes is a little support. But once you are working, you can be ready to fly.

Making a build

With the templates installed and everything ready, you should have access to the new KNI templates, and for this article more importantly, the KNI Web Browser Template as shown below:

Just create your new project using this template and you are already 90% of the way there:

Running the project results in a new Web Browser being launched against your local machine, ready to test and play with:

Fantastic, so what was too hard about that?

KNI Web uses Microsoft Blazor (which uses Razor files) as the backend, which is a C# compiled Web system for building web apps. It is really useful as MonoGame is C# based and Blazor is a C# based web system, it just works. Some other Web solutions for MonoGame used a cross-compiler to turn C# into JavaScript, but Blazor is just cleaner and more efficient.

Looking into the project, you should see a few subtle differences, which we should discuss, so that you don’t change anything you should not by mistake, namely:

Out of the box, you should not need to change anything and the only thing you will need to keep in sync is the NAME of your Game class and the entry point in Index.razor.cs, in the same way we do for other MonoGame projects between the Game class and the Program class.

Here is a GIF of the GameState Management sample (with a few alterations) running using a KNI Web Build.

What could go wrong

Now, this is the web we are targeting and there is a lot of history and patches to make modern web browsers work, so it should come as no surprise that not EVERYTHING is going to work out of the box:

• Some features are missing, like GamePad and Touchscreen support.
• Some shaders may not work as expected, or require reduced features.
• File access can be… Tricky, pesky permissions.

To name but a few, no doubt there are more, much more. But this being an Open Source project, if you have the skills and are willing to contribute to make it better, I encourage you to do so!!.

When you do hit an issue, and more likely than not I am afraid, you will, you will see the following result when you run your project:

A pretty purple screen with a “Reload” option, which is not really that helpful on its own and Visual Studio is no help here really because you have left the confines of your debugger and entered “The Web Zone”!

Luckily, most browsers have an “F12” developer option, so by pressing “F12” you will get the not-so-friendly “Web Debugger” window, as shown below:

These outputs are more friendly to Web Developers (Well, I assume so?) full of lots of information to help you diagnose what is going on, in this case, the issue was simple:

The sample was using a GamePad for input and GamePads are not currently supported using the KNI Engine for the web.

If you are only doing a web project, just fix / remove the offending code, if you have a multi-platform project, then just encase the offending code with the following pre-compiler definition (#if):

#if !BLAZORGL
                CurrentGamePadStates[i] = GamePad.GetState((PlayerIndex)i);
#endif

Which will exclude the offending lines from being included in the Web Build (just make sure things still work / compile).

Feel free to experiment and play, and for those cunning individuals who are experienced in Web Development, I encourage you to help enlighten those who are not.

Extra credit, publish to GitHub Pages

Having a web build on your machine that you can view locally is all well and good, but what about everyone else? You cannot exactly ask them all to come to dinner to view your creation, what about pushing it to the web?

To get your build out there, you have a couple of options:

• Painstakingly setup a web host, build your project, connect to your host, upload your files and pray everything works. or
• Let GitHub do it for you. Hosted for free, using free time and none (well a little) of yours.

Thanks to GitHub actions (automation) and GitHub Pages (free static web hosting) we can manage the source of our project online and then create/update a build every time we push an update to source control.

To achieve this, assuming you have a GitHub repo setup and have pushed your source code to it, the tasks we need to complete are:

• Configure the permissions for automation.
• Setup GitHub pages.
• Create a GitHub Pages deployment workflow.

As easy as 1,2,3, honest.

Configure automation permissions

Because we want our automation to publish content to GitHub pages, and write back to the repository, we need to allow it to (by default, it is turned off). To do this, navigate to “Settings -> Actions -> General -> Workflow Permissions” and set the option to “Read and write permissions” as shown below:

This allows the automation to publish on your behalf.

Setup GitHub Pages

Enabling GitHub pages is very simple and GitHub continually works to improve the flow to make it easier and easier.

Next, you need to:

• Navigate to “Settings -> Pages”
• Change the “Source” to “GitHub Actions”
• In the selection that appears called “Static HTML”, click on the “Configure” button.

Done, there is more to do, but GitHub Pages is now setup. Now for the final and slightly trickier part, the automation workflow.

Configure your deployment workflow

Following on from the previous step, you should now see the following default workflow setup, which is close but will not build our MonoGame project.

To get the result we want, we need to add the following:

• Setup .NET 6 on the automation host.
• Build/Publish our Blazor webpage complete with our MonoGame project.
• Package the build output, not just the entire repository.
• Deploy (this is actually there, but we need to check the order)

The updated YAML should be replaced with the following:

# Simple workflow for deploying static content to GitHub Pages
name: Deploy MonoGame web project to GitHub Pages

on:
  # Runs on pushes targeting the default branch
  push:
    branches: ["main"]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:


# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
  contents: read
  pages: write
  id-token: write


# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.

# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
  group: "pages"
  cancel-in-progress: false

jobs:
  # Single deploy job since we are just deploying
  deploy-to-github-pages:
    environment:
      name: github-pages
      url: $
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Setup Pages
        uses: actions/configure-pages@v3
      - name: Setup .NET Core SDK
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: 6.0.x
      - name: Publish .NET Core Project
        run: dotnet publish Platforms/Web/KNIBrowser.csproj -c Release -o release --nologo
      - name: Upload artifact
        uses: actions/upload-pages-artifact@v2
        with:
          # Upload wwwroot from publish action
          path: 'release/wwwroot'
      - name: Deploy to GitHub Pages
        id: deployment
        uses: actions/deploy-pages@v2

The critical elements to watch out for are:

• “Publish .NET Core Project” - Make sure the csproj file path is correct. In this instance it is “Platforms/Web/KNIBrowser.csproj”
• The “Upload Artifact” path, which selects the files from the build to upload. By default the “dotnet publish” command outputs to the root of the folder.

If in doubt, run the commands locally in your project folder to make sure you get the output you expect.

Click save and “Commit to main” (or create a pull request) to activate the workflow.

WARNING this will run straight away, as the workflow is activated on checking in code. But do not worry, you have 2000 minutes of FREE time on GitHub for automations.

Check the Action output

Providing everything is correct, you have put in the paths correctly and checked it DID actually build locally, then you should get the following result:

Clicking on the completed Action, you shall see the output of the Action, INCLUDING the URL that your build was actually published to:

Check out my build here:

http://darkgenesis.zenithmoon.com/GameStateManagementSample-Web/

Danger Will Robinson

One issue I did hit which actually prevented my build completing, required me to edit my KNI Engine csproj file and remove a line. The line in question was:

  <Import Project="$(MSBuildExtensionsPath)\MonoGame\v3.0\MonoGame.Content.Builder17.targets" />

Rather ironically, local builds will work fine and run with this in, but automated builds will not. Without it, everything is squeaky clean. (fingers crossed)

The game is afoot

We have come to the end of this little web journey, I hope others are encouraged and excited to ship your games or hack projects to the web to demonstrate your skills, especially if there is a GameJam going on :D.

Here is hoping this is a little light in the darkness to get you going!