Help with data management

[Prever77]

Hi. Long story short.

I made year or two ago my own package of scripts for role play server, which i never started. 3 months ago i decided to rewrite everything. As i looked at it and said "what a mess. Who wrote this?" meanwhile i've gained experience in C and good habits, so choice to rewrite everything was obvious for me at that moment.

So i did basic optimisation. What can be stored and should be in bools is now stored in bools. not 1 and 20s lol. etc etc

But for now i hitted the wall. Which is named "to buffer. Or not to". By that i mean. My scripts are connected to outside database as built in database is.. well not so fast and secure. And i didn't used to do that mechanism in original code. But i think it's rational to store very often pulled data from database localy on server memory. And update tables in parts to database like every 30seconds. Also i want to use buffer data and database to validate what comes from client side. Im storing vehicles on map, their custom data etc etc.

And here's my main questions.

1.is it worth of effiord to make buffer if i plan 80-200 players on my server.

2.is validating client data with buffer and database worth concept

3.what are your coding advices for me?. Im Asking experienced ones that ruled servers with large player base. 

--[[btw i know that if server has 3 players buffer is not needed. Thanks, awesome advice.--]]

 

[IIYAMA]

17 hours ago, Prever77 said:

1.is it worth of effiord to make buffer if i plan 80-200 players on my server.

Buffers can be useful.
Because you can only pick a server with a good CPU once. Sometimes you can upgrade the CPU, but there is a limit.

• While with ram, it is easier to upgrade.
• MTA Lua processes are afaik still is single threaded.

But you need to invest a lot of time.

17 hours ago, Prever77 said:

2.is validating client data with buffer and database worth concept

I can be.

• An example db resource I made a while a go, not very efficient storage (text/string based), but easy to use.
  Not sure why I build the types text based, I would have build it differently now a days.
  • Also recommended to write automatic tests for your creation, like I did with mine in sub folder \dev\*.
• You can storage md5 hash validation strings for validation, if you are interested in that.
  Smaller hash storage [binary]: using the UNHEX(hash) function (convert back with HEX(binary).
  Example resource that uses hash validation for client screenshots.

 

17 hours ago, Prever77 said:

3.what are your coding advices for me?. Im Asking experienced ones that ruled servers with large player base. 

I am not one of them. But I do work often met databases.

I have put my recommendation in a spoiler if you are still interested:

Spoiler

While buffers can be useful.

• They can be also create bugs if not created correctly.
• Require lots of testing.
• But they do in fact make it easier to handle Async requests. Keeping your db from blocking you Lua processes.
  -> Using synced requests makes things easier, but could also create lag on your server.

There is 1 alternative to consider and that is to keep the active data in memory. While you do write to the db, you do not fetch data that you already know and keeping it around. This is a concept you could consider a buffer, the only difference is that you pre-load all the data you might need. And moving the 'truth' from the db to the memory when initialising. 

So when a player joins. You get his health, armor, money, house name,  name etc. etc. And put it in a table that represents his exact model in the db.
When values change, you only write to the db.

This keeps the data fast and persistent. But with the downside that if there is a bug in your code, your db becomes desynced with your memory. And when the player reconnects the 'truth' of the db leads again. It is important to mirror in Lua the database column limitations, which should solve most of those issues.

 

Last recommendation:

• Add rate limits for requests done by a client. Your db is one of your server it's weaknesses. (concept of a passive rate limit)

 

 

[Prever77]

Ty so much for reply.

Yes i had same thing in head about how buffer should look like. Changing only values that has changed etc. I didn't precised it well. By sending tables to database every 30s. i was thinking about mechanism that also stacks changed values to certain amount. Like old table and new table concept. Lets say im checking every 30s if cars changed their position. I compare old table with new. And values that has changed i move to some sort of stack. And stack is let say sent every 4 cycles of checking car positions. Before that stack is sent to database he's checked for latest values and reduced in size. 

By validating what comes from client i was thinking about putting clever anticheat trigger mechanism. Like while checking for changes. It's checking if that change was possible with my scripts for cars. Is car moving too fast or something. If it's. Then anticheat is triggered. and let me end here about anticheat mechanism. But ty also for mentioning your script. I'll take a look on it.

Also by outside database i was thinking about LAMP thing in LANetwork on the other physical device. As i wanna only let one device to talk with the world. Just to reduce risk. Im sharing same network so i need to keep packs small so i have transfer reserved for server talking to trustfull world. Not exact desciption on purpouse here.

Im having physical hardware so CPU and RAM are not problem. Problem is rational use of power. As i want to run my server independent. And made it most watt per player ratio effective. And keep it cheap. As im not planing earn money from server.

Sorry for not mentioning details, but i didn't knew how to describe it all earlier and now too. as english is not my great side. Also ty for yours recommendations. Especialy about last one i forgot about that nice feature 

EDIT ADD

Isn't it better for me to just make mta module that handle this stuff with multi core rather than single threaded lua? TheoreticaIy it should save me cycles on lua. And let them to be used more productive on bare bone server mechanics and scripting. And if so. what are good sources to begin with mta modules.

 

 

Edited 1 hour ago by Prever77