Leaderboard

Software Rendering in MTA:SA Lua

Dear MTA community, I have been spending my last 8 weeks on mathematical problems. One of them is the 3D Frustum-Plane intersection that is used by GPUs to draw triangles onto your screen. If you want to learn more about this please consider reading this thread. Promotional Video: https://www.youtube.com/watch?v=RQy3Q4Xe110 Prerequisites This tutorial is aimed at people who are capable of scientific thinking and are willing to playfully learn with Lua code. To execute steps in this tutorial minimal knowledge of Linear Algebra and Lua is required. Required MTA Resource: https://github.com/quiret/mta_lua_3d_math Description of the math Imagine that we have got a frustum and a plane in a 3D room described by coordinates plus their boundaries. By intersecting both you obtain all coordinates on a screen along with their depth values. Now think about how your vision works. You see distant objects smaller than closer ones. You rotate your eyes to angles of vision. If we were to put this concept into terms of math we could say: the plane of vision is bigger in the distance than in close proximity. The frustum is a seamless row of vision planes starting from the pyramid tip to the bottom. How to use the MTA Resource Just download the GitHub repository into a folder of your MTA Resources, name it "math_3d_nonlin" and start it. You can execute the following commands for quick testing: send_bbuf: draws a simple depth test draw_model: draws the DFF file "gfriend.dff" Now we have got the basics out of the way. Time to start coding. Please create a new "_math_test.Lua" script file in the resource and include it server-side at the bottom of meta.xml. Tutorial: software rendering a plane on screen Open your _math_test.Lua and include the following code: local viewFrustum = createViewFrustum( createVector(0, 0, 0), -- position createVector(10, 0, 0), -- right createVector(0, 0, 10), -- up createVector(0, 20, 0) -- front ); local plane = createPlane( createVector(-3, 10, -3), createVector(6, 0, 0), createVector(0, 0, 6) ); local function task_draw_scene(thread) local bbuf = create_backbuffer(640, 480, 255, 255, 0, 50); local dbuf = createDepthBuffer(640, 480, 1); local time_start = getTickCount(); do local gotToDraw, numDrawn, numSkipped = draw_plane_on_bbuf(viewFrustum, bbuf, dbuf, plane, true); if ( gotToDraw ) then outputDebugString( "drawn " .. numDrawn .. " pixels (skipped " .. numSkipped .. ")" ); end end local time_end = getTickCount(); local ms_diff = ( time_end - time_start ); outputDebugString( "render time: " .. ms_diff .. "ms" ); taskUpdate( 1, "creating backbuffer color composition string" ); local bbuf_width_ushort = num_to_ushort_bytes( bbuf.width ); local bbuf_height_ushort = num_to_ushort_bytes( bbuf.height ); local pixels_str = table.concat(bbuf.items); local bbuf_string = pixels_str .. ( bbuf_width_ushort .. bbuf_height_ushort ); taskUpdate( false, "sending backbuffer to clients (render time: " .. ms_diff .. "ms)" ); local players = getElementsByType("player"); for m,n in ipairs(players) do triggerClientEvent(n, "onServerTransmitImage", root, bbuf_string); end outputDebugString("sent backbuffer to clients"); end addCommandHandler( "testdraw", function(player) spawnTask(task_draw_scene); end ); Result: Try executing the "testdraw" command. At the top of file you see the definition of our frustum cone as well as a plane. By calling the function "draw_plane_on_bbuf" we put color information into bbuf for exactly the pixels that make up the rectangle. If you change the plane definition to... local plane = createPlane( createVector(-2, 10, -4), createVector(6, 0, 3), createVector(-2, 0, 6) ); you instead get this image: Try changing around the coordinates of frustum and plane to obtain different pictures! Tutorial: software rendering a triangle on screen Take the same code as in the tutorial above but change line 19 to: local gotToDraw, numDrawn, numSkipped = draw_plane_on_bbuf(viewFrustum, bbuf, dbuf, plane, true, "tri"); This way we have changed the primitive type to triangle (rectangle is the default). Try executing the "testdraw" command again to inspect the new result! Tutorial: drawing a DFF file onto screen Instead of writing triangle definitions by hand we can take them from a DFF file instead. DFF files are storage of triangle and vertex information along with 3D rotation and translation information. By extacting the triangles from the DFF file we can put them into our algorithm to software-render them! Here is a related excerpt from math_server.Lua: local modelToDraw = false; do local modelFile = fileOpen("gfriend.dff"); if (modelFile) then modelToDraw = rwReadClump(modelFile); fileClose(modelFile); end end local function task_draw_model(thread) local bbuf = create_backbuffer(640, 480, 255, 255, 0, 50); local dbuf = createDepthBuffer(640, 480, 1); local time_start = getTickCount(); local num_triangles_drawn = 0; if (modelToDraw) then -- Setup the camera. local geom = modelToDraw.geomlist[1]; local mt = geom.morphTargets[1]; local centerSphere = mt.sphere; local camPos = viewFrustum.getPos(); camPos.setX(centerSphere.x); camPos.setY(centerSphere.y - 3.8); camPos.setZ(centerSphere.z); local camFront = viewFrustum.getFront(); camFront.setX(0); camFront.setY(5 + centerSphere.r * 2); camFront.setZ(0); local camRight = viewFrustum.getRight(); camRight.setX(centerSphere.r * 2); camRight.setY(0); camRight.getZ(0); local camUp = viewFrustum.getUp(); camUp.setX(0); camUp.setY(0); camUp.setZ(centerSphere.r * 2); local triPlane = createPlane( createVector(0, 0, 0), createVector(0, 0, 0), createVector(0, 0, 0) ); local vertices = modelToDraw.geomlist[1].morphTargets[1].vertices; local triangles = modelToDraw.geomlist[1].triangles; local tpos = triPlane.getPos(); local tu = triPlane.getU(); local tv = triPlane.getV(); for m,n in ipairs(triangles) do taskUpdate( m / #triangles, "drawing triangle #" .. m ); local vert1 = vertices[n.vertex1 + 1]; local vert2 = vertices[n.vertex2 + 1]; local vert3 = vertices[n.vertex3 + 1]; tpos.setX(vert1.x); tpos.setY(vert1.y); tpos.setZ(vert1.z); tu.setX(vert2.x - vert1.x); tu.setY(vert2.y - vert1.y); tu.setZ(vert2.z - vert1.z); tv.setX(vert3.x - vert1.x); tv.setY(vert3.y - vert1.y); tv.setZ(vert3.z - vert1.z); local gotToDraw, numDrawn, numSkipped = draw_plane_on_bbuf(viewFrustum, bbuf, dbuf, triPlane, false, "tri"); if (gotToDraw) and (numDrawn >= 1) then num_triangles_drawn = num_triangles_drawn + 1; end end end local time_end = getTickCount(); local ms_diff = ( time_end - time_start ); (...) end The code first loads a DFF file called "gfriend.dff" and stores it inside the "modelToDraw" variable. Once you execute the "draw_model" command the code looks up the first geometry in the DFF file and fetches all triangles associated with it. The rendering camera is set up to point at the middle of the model. Then all triangles are drawn one-by-one. https://twitter.com/rplgn/status/1230650912345067520 Try swapping the DFF file for another one, like biker.dff, and examine the results! Maybe extract a different DFF file from GTA:SA and replace gfriend.dff with that one. External references: math calculation on paper example: https://imgur.com/gallery/rLvln3X German thread on mta-sa.org: https://www.mta-sa.org/thread/38693-3d-frustum-ebene-schneidung-in-Lua/ Do you have any questions related to the math or the implementation? Do not shy away from asking! I want to provide you with as much insight as I can.