Use DOTNET_STARTUP_HOOKS and Harmony to hotpatch UBT and UAT, and add support for Kubernetes-based UBA
Currently UET intercepts progress information by pretending to be XGE via OpenGE, which allows us to write better progress information to the terminal. Since Unreal Engine 5.4 though, Epic provides the Unreal Build Accelerator which is a replacement to XGE (and OpenGE). Not only do we want to be able to leverage UBA from UET (and distribute builds), but we need to plan for a future where XGE support is removed from the Unreal tooling since UBA exists and is recommended by Epic.
Because we can no longer pretend to be xgConsole
and execute the graph ourselves, we instead need to hotpatch UBT so we can get at the internals of ActionGraph
and ImmediateActionQueue
, and forward all of the data we care about back to the main UET process.
-
Experiment with hotpatching Kubernetes-based UBA support into UBT, so we can use it regardless of whether https://github.com/EpicGames/UnrealEngine/pull/11919 gets merged into the engine. This will also validate that our approach for DOTNET_STARTUP_HOOKS
and using Harmony 2 will work. -
Hotpatch configuration reading to turn on UBA in engines that support it, and to automatically set Kubernetes UBA information from environment variables. -
Hotpatch ImmediateActionQueue
constructor to retrieve total task information. -
Hotpatch ImmediateActionQueue.SetActionState
to determine when an action's status has changed. -
Hotpatch ImmediateActionQueue.TryStartOneActionInternal
with a transpiler patch to determine when actions start and when actions error. -
Hotpatch ImmediateActionQueue.RunTillDone
so we know when the queue is complete. -
Test our hotpatching works on Unreal Engine 5.2 and 5.3. We don't need the UBA stuff to work, but we do need the ImmediateActionQueue
to work so we can get progress. -
Add Redpoint.Uet.Patching.Loader
project, which will replaceRedpoint.Uet.OpenGE
. -
Get Redpoint.Uet.Patching.Loader.csproj
to bundle the output ofRedpoint.Uet.Patching.Runtime
and it's dependencies into an embedded resource. -
Get Redpoint.Uet.Patching.Loader
to extract the bundle at runtime. -
Get Redpoint.Uet.Patching.Loader
to setDOTNET_STARTUP_HOOKS
to point at the extractedRedpoint.Uet.Patching.Loader.dll
file. -
Define a gRPC protocol between Redpoint.Uet.Patching.Runtime
andRedpoint.Uet.Patching.Loader
for passing job progress. -
Get Redpoint.Uet.Patching.Runtime
to pass the information collected from hotpatches back to the loader over gRPC, and then emit logging information similar toLogInterceptingDispatcher
. -
Once this is working, delete all of the (now obsolete) OpenGE code.
We should then also replace the Redpoint.Uet.BuildPipeline
patching code with Harmony, and avoid recompiling DLLs. This should make engine builds start much faster, and allow us to turn on UEFS-unmount-after-build to ensure cleaner build environments (currently we keep engines mounted because the .NET rebuild is too slow).
-
Make sure we apply patches as a prefix to the assembly's entry point, instead of doing it immediately on startup, since non-startup assemblies aren't loaded at that point and thus Harmony won't be able to patch them yet. -
For AutomationTool
, patch inBUILD_GRAPH_PROJECT_ROOT
support. -
For AutomationTool
, patch inBUILD_GRAPH_ALLOW_MUTATION
support. -
For AutomationTool
, patch script file loading so that scripts can be referenced outside the engine directory. -
For AutomationTool
, patchbRemote
to exclude BuildGraph targets that we know won't work with it. -
For AutomationTool
, patch outrobocopy
-based file copies since it's unreliable. -
For EpicGames.BuildGraph
, patchSwitch
handling so that it isn't broken with unexpected fallthroughs happening (this is fixed on modern engines, but broken for 5.0). -
For UnrealBuildTool
, patchSingleInstanceMutex
so thatuebp_UATMutexNoWait
can be used to skip it. -
For EpicGames.Build
, patchWritableEngineDirectory
so that includes the process ID in theUnrealEngine
folder name. -
For UnrealBuildTool
, patchUEDeployAndroid
to fix directory name handling when paths include.uet
for Gradle execution. -
Remove the old patching code. -
Enable UEFS engine dismount.