Skip to content

Game Development: Rendering, Physics and Collision, Animation, and Audio

Part 2 of 5 of the Video Game Development reference (plugin video-game-development), covering §4–§7. Sibling skills: game-engines-loop-and-architecture (§0–§3), game-ai-networking-and-tools (§8–§10), game-performance-feel-and-shipping (§11–§15), game-development-reference (§16–§20). Section numbers are shared across the set; a reference written as §N → skill points into that sibling skill.

Currency: Verified August 2026. See §17 → game-development-reference for the currency snapshot and what goes stale first.

How to read this. Reference, not tutorial. Sections are independent. Three markers: - [DURABLE] — real-time systems reality, human perception, or a lesson the industry has relearned every console generation. Does not expire. - [VERSIONED] — specific to an engine version, API, platform, or market condition. Verify before relying on it. - [CONTESTED] — practitioners genuinely disagree, usually because the right answer depends on team size and genre.

⚠️ GOTCHA boxes mark the mistakes that ship broken games, blow the frame budget, or destroy a studio's schedule.

The three framings that organize everything below: 1. A game is a soft-real-time simulation with a hard deadline, 60 times a second. 16.67 ms at 60 fps, 8.33 ms at 120. Everything — rendering, physics, AI, audio, networking, streaming, GC — shares that budget. Consistency beats peak framerate; a stable 30 feels better than a 60 that stutters. 2. Games are content pipelines with a game attached. On any team above about five people, the tools, build system, and iteration loop determine your output more than your engine code does. Ten-minute iteration times will cost you the game. 3. Scope is the thing that kills projects, not technology. The overwhelming majority of unfinished games died of ambition, not of a hard technical problem. §14 → game-performance-feel-and-shipping is the section most people need and skip.

§4. Rendering

4.1 The pipeline

scene → CULLING (frustum, occlusion) → sorting/batching → draw submission
  → vertex/mesh shaders → rasterize → fragment shaders → depth/stencil
    → post-processing (TAA, bloom, tonemap, color grade) → UI → present

Forward (shade each fragment as drawn — good for MSAA, transparency, mobile, and tile-based GPUs), Deferred (write a G-buffer, then shade — many lights cheaply, but transparency and MSAA are awkward and bandwidth is high), Forward+/Clustered (light culling into tiles or clusters, then forward-shade — the mainstream modern answer), and Visibility buffer (store triangle IDs, shade once per pixel — what Nanite-style systems do).

4.2 Graphics APIs

[VERSIONED]

API Platforms Notes
Direct3D 12 Windows, Xbox DXR ray tracing, VRS, mesh shaders, DirectStorage, work graphs. DX12 Ultimate is the feature bundle
Vulkan Windows, Linux, Android, macOS via MoltenVK Open, portable, explicit. Current spec is 1.4; Vulkan Roadmap 2026 requires 1.4 and targets mid-to-high-end hardware shipping in 2026 or shortly after, adding baseline requirements like hostImageCopy
Metal Apple only The only first-class path on Apple platforms
WebGPU Browsers, and increasingly native The modern web target; also a decent portable abstraction
OpenGL / WebGL Legacy Still relevant for compatibility floors
Console APIs NDA'd GNM/AGC, and the Xbox D3D12 variant

[DURABLE] Use your engine's abstraction unless you have a specific reason not to. Writing directly against D3D12/Vulkan is a large, ongoing commitment; both are explicit APIs where you manage memory, synchronization, descriptors, and pipeline state.

[CONTESTED] Whether the explicit APIs are still the right shape. Sebastian Aaltonen's widely-discussed argument: DX12, Vulkan, and Metal are now ten years old and were designed for GPUs that are thirteen years old, from an era before bindless resources were widely supported — with the result that a new low-level remapping layer has grown beneath engines' RHIs, re-assuming the complexity the old drivers used to handle. There is real disagreement about whether the next step is simpler high-level APIs, further explicitness, or GPU-driven pipelines making the question moot.

Modern GPU-driven rendering is where the field is going: mesh shaders (replacing the vertex/geometry pipeline with amplification+mesh stages over meshlets), GPU-driven culling and draw generation, and work graphs (the GPU enqueuing its own work, shipping in D3D12 and available in Vulkan via VK_AMDX_shader_enqueue, not yet standardized across vendors).

⚠️ GOTCHA — mesh shaders are not a universal replacement. Mobile GPUs are tile-based renderers that bin individual triangles to small tiles; meshlets are too coarse-grained for that, and binning them to tiny tiles causes significant geometry overshading. There is no clear convergence path — you still need the vertex-shader path. Any "just switch everything to mesh shaders" plan is a desktop-only plan.

4.3 The things that actually cost you

  • Draw calls / CPU submission — batch, instance, and use indirect draws. Historically the #1 CPU bottleneck.
  • Overdraw — shading pixels that get covered. Depth pre-pass, front-to-back sorting.
  • Bandwidth — texture reads and G-buffer traffic. Compress everything (BC1–BC7 on desktop, ASTC on mobile), mip properly, and use the smallest formats that look right.
  • Shader complexity and register pressure — high register use kills occupancy.
  • State changes and pipeline switches.
  • ⚠️ Shader compilation stutter — the defining PC technical problem of this generation. Compile and cache PSOs ahead of time, at load or install; do not compile on first use during gameplay. Vulkan's SPIR-V helps by moving parsing offline, but driver-side pipeline compilation still happens.

4.4 Ray tracing and upscaling

Hardware RT (DXR/Vulkan RT, both organizing geometry into acceleration structures) is now mainstream for reflections, GI, and shadows — usually as an option on top of a raster path, rarely as the only path. Upscaling (DLSS, FSR, XeSS, MetalFX) and frame generation are now assumed in performance budgets rather than treated as a bonus, which has quietly changed what "runs at 4K60" means. [DURABLE] Budget for the native resolution you actually render at, and treat upscaling as a quality lever, not a substitute for optimization.

§5. Physics and Collision

5.1 Use a physics engine

Havok, PhysX, Jolt, Box2D, Bullet, Rapier. [VERSIONED] Jolt became Godot 4.6's default physics engine (January 2026), closing much of Godot's previous gap against Unity and Unreal on 3D physics fidelity — a good example of how quickly this layer moves.

[DURABLE] Most games do not want realistic physics. They want controllable physics that feels good. Character controllers are usually kinematic (you move them; you resolve collisions manually) rather than dynamic rigid bodies, because rigid-body player characters feel floaty, get stuck, and are hard to tune. This surprises people every time.

5.2 Collision detection

Broad phase (which pairs might collide — spatial hash, BVH, sweep-and-prune, grid, octree) then narrow phase (do they actually — SAT, GJK/EPA, sphere/AABB/capsule tests). [DURABLE] The broad phase is where the algorithmic win is; narrow-phase micro-optimization matters far less than not testing 10,000 irrelevant pairs.

Continuous collision detection (CCD) for fast objects, or your bullets pass through walls. Sub-stepping for stability. Layers and masks so things only collide with what they should.

⚠️ GOTCHA — the classic physics bugs, all of which you will hit: tunneling (fast object, thin wall — needs CCD or raycast movement); jitter (conflicting constraints, or a too-large timestep); objects gaining energy (integration error — use a semi-implicit Euler or better); the "sticky wall" (missing collision-margin handling); and framerate dependence (§2.1 → game-engines-loop-and-architecture — physics must be on a fixed timestep).

§6. Animation

  • Skeletal animation: bones, skinning (linear blend or dual-quaternion), the bind pose, and the eternal problem of the elbow that collapses.
  • Blend trees / blend spaces — parameter-driven blending (speed, direction) is how locomotion actually works.
  • State machines and layers — the standard authoring model; upper body and lower body on separate layers.
  • Root motion vs. in-place — [CONTESTED] root motion looks better and fights your character controller; in-place is controllable and can foot-slide. Most action games use in-place with foot IK; most third-person cinematic games use root motion.
  • IK — foot placement on slopes, look-at, hand placement on ledges. Two-bone IK covers most of it; FABRIK and CCD for chains.
  • Animation compression matters enormously for memory in large games.
  • Procedural — additive layers, ragdolls, physical animation blends, and increasingly ML-based motion matching (Motion Matching replaced hand-built state machines in a number of AAA locomotion systems).

[DURABLE] Animation is where "game feel" (§12 → game-performance-feel-and-shipping) mostly lives. Attack-cancel windows, animation-driven hitboxes, blend-out times, and root-motion authority are gameplay design decisions expressed as animation data — not art polish applied afterward.

§7. Audio

[DURABLE] Audio is the most-cut and highest-return-per-dollar discipline in games. Players don't consciously notice good audio and immediately feel its absence.

  • Middleware: Wwise and FMOD are the standards, and both are worth the integration cost on any team with a dedicated audio person. Engine-native audio is fine for small projects.
  • Concepts: buses and mixing, ducking (dip the music under dialogue), DSP (reverb, EQ, compression), occlusion and obstruction, HRTF/spatial audio, attenuation curves, and voice limiting (⚠️ 200 simultaneous gunshots will clip and eat your CPU — cap and prioritize).
  • Variation is what prevents fatigue: multiple samples per event, randomized pitch and volume, round-robin. A single footstep sample played 10,000 times is a defining amateur tell.
  • Streaming vs. in-memory: music and dialogue stream; short SFX stay resident.
  • Loudness: normalize to a platform target; console certification (§13.3 → game-performance-feel-and-shipping) has loudness requirements you can fail on.

Plugin: video-game-development · View SKILL.md on GitHub