Sounds like you have onboard graphics. The memory for onboard graphics gets taken out of your system memory so if you're running applications that use up a lot of memory (e.g. MTA and GTA at the same time), you'll get less video memory left over for the graphics.
Solution: buy more RAM or better still, buy a graphics card. Trust me, you'll really notice a difference.
Edit: Btw, if you want a solution that doesn't involve spending money, try closing all unnecessary applications when running MTA (e.g. stuff in the system tray).