Best practice to synchronize server/client caches

[Sorata_Kanda]

Hello everyone,

as you probably know, some people suggest that it makes sense to use element data without having them synchronized.

With that being said, I try to script a resource which uses OOP and in order to e. g. make changes through GUI, I need to push objects through events to client, then modify specific attributes that have been changed through GUI, and afterwards send that object back to the server and save it in a table.

I'm wondering if you should have both server and client caches or just have a server-sided cache and push the affected objects through events.

What is an efficient way to handle caches?

Thanks in advance!

[IIYAMA]

Server should be in control at all times.
So yes caches on serverside to reduce impact on the database. But keep in mind that if a query fails, it is important that the data is reset.

 

In case of more users are editing the same data. > No caches recommended on clientside. (until the instructions from clientside are perfect)

 

While there are changes made, it is recommended that the client his GUI is disabled until the server is finished. Else the data might not be mirrored correctly, that being said it is not impossible.

 

You can use caches clientside, but it is important that the identifiers remain unique at all cost. The server is in control of the identifiers. Else there is a VERY HIGH risk of data corruption.

 

Clientside should only send retrace able instructions of the modifications and not the modified data. Else there is a risk of VERSION corruption. Instructions also allows you to mirror without freezing the GUI.

 

Each instruction should be validated on both serverside as well as clientside. In case of a failed data-update/query clientside should be restored to it's previous data-structure. It is VERY important to block new instructions and freeze the GUI until both sides are mirrored again.

 

It is not that easy after all

 

And use or build a tool like this:

https://gitlab.com/IIYAMA12/mta-communication-enchantment/tree/master/

Edited March 19, 2019 by IIYAMA

[Sorata_Kanda]

13 hours ago, IIYAMA said:

Clientside should only send retrace able instructions of the modifications and not the modified data. Else there is a risk of VERSION corruption. Instructions also allows you to mirror without freezing the GUI.

So it should be better to just use one server-sided cache and i. e. if I unlock a door of a house through GUI, I'm better off using TriggerServerEvent?

As I use OOP for creating house objects, I'm worried about how to display house details properly in a GUI if the house object isn't exisiting on the client-side and element data is async.

 

13 hours ago, IIYAMA said:

While there are changes made, it is recommended that the client his GUI is disabled until the server is finished. Else the data might not be mirrored correctly, that being said it is not impossible.

Is there something like a keyword (synchronized in Java) in Lua? Or do I have use a boolean to set the event execution to be blocked until the previous execution is finished?

[IIYAMA]

2 hours ago, Sorata_Kanda said:

So it should be better to just use one server-sided cache and i. e. if I unlock a door of a house through GUI, I'm better off using TriggerServerEvent?

As I use OOP for creating house objects, I'm worried about how to display house details properly in a GUI if the house object isn't exisiting on the client-side and element data is async.

 

Is there something like a keyword (synchronized in Java) in Lua? Or do I have use a boolean to set the event execution to be blocked until the previous execution is finished?

 

1.

Yea, this will also prioritise the request over the network.

 

2.

The entity you will be using to update the main data is a table. MTA objects/elements are secondary.

Ofcourse this doesn't mean you can't use an object to fake the updates before they are actually applied. (Data reduction)

 

3.

A boolean or a custom function in between. The default MTA functions do not have such a feature without creating errors.

[Sorata_Kanda]

Let's imagine this:

I go into a pickup and want to have a GUI to appear.

Following implementation:

-- Is it good to pass in the element in order to keep the reference to the house object?
local pickup = --Pickup element
local owner = -- Owner string
local rent = -- random int

addEventHandler('onPickupHit', resourceRoot, function()
	triggerClientEvent(source, 'showGUI', pickup, owner, rent) -- Notice that pickup is used as source element
end)

My problem is that somehow I need to keep the reference as I want to e.g. enter the house or lock it. That means I need to trigger a server event, passing that element again. I'm not sure if this is reliable or not. Possible implementation

--- Client - Used in GUI
triggerServerEvent('enterHouse', localPlayer, source) -- source equals pickup that was set as source element in the implementation above

--- Server

-- A table where each house object that was instantiated is being kept track with pickup elements as a reference
houses = {
  pickup1 = House(...),
  pickup2 = House(...)
}

-- Table keeping track of booleans for each event in order to realize "synchronized" state (like Java)
blocked_exec = {
  "lockHouse" = false
}

addEventHandler('lockHouse', resourceRoot, function(pickupElement)
  while (blocked_exec["lockHouse"]) do end -- In case this event is currently handled by someone else, keep in loop until finished
    
  blocked_exec["lockHouse"] = true -- Block execution until finished (possible concurrency problem/race condition?)
  local houseObject = houses[pickupElement]
  houseObject:setLocked(not houseObject:isLocked())
    
  blocked_exec["lockHouse"] = false 
end)

Got any feedback on this concept?

Edited March 22, 2019 by Sorata_Kanda

[IIYAMA]

 

@Sorata_Kanda

while (blocked_exec["lockHouse"]) do end

This will kill your server.

The code will not stop at the current server/client frame, but freeze all frames that come after it and keep doing endlessly looping as fast as it can. Luckily Lua is smart enough to abort it after a few milliseconds.

 

 

 

If we translate this to a possible working version. (I AM NOT saying that it is a good method, AS IT IS NOT, uses a lot of CPU as you are literally rendering on serverside)

local function waitWithLocking (pickupElement)
	if blocked_exec["lockHouse"] then
		callNextFrame(waitWithLocking, pickupElement)
	else
		blocked_exec["lockHouse"] = true -- Block execution until finished (possible concurrency problem/race condition?)
		
		local houseObject = houses[pickupElement]
		houseObject:setLocked(not houseObject:isLocked())

		blocked_exec["lockHouse"] = false
	end
end

addEventHandler('lockHouse', resourceRoot, function(pickupElement)
	callNextFrame(waitWithLocking, pickupElement) -- In case this event is currently handled by someone else, keep in loop until finished
end)

 

callNextFrame

Spoiler

    --[[
    	-- callNextFrame function
    ]]
    do
    	local tableRemove = table.remove

    	local serverSide = triggerClientEvent and true or false
    	
    	local nextFrameCalls = {}
    	
    	local serverSideTimer
    	

    	local processing = false
    	
    	local function process ()
    		
    		--[[ 
    			Do an empty check at the beginning of the function, this will make sure to make an extra run in case of heavy work load. 
    			If the timer is killed or the addEventHandler is removed, then this has to be re-attached again every frame. This is not very healthy...
    		]]
    		
    		if #nextFrameCalls == 0 then
    			if serverSide then
    				if serverSideTimer then
    					
    					if isTimer(serverSideTimer) then
    						killTimer(serverSideTimer)
    					end
    					serverSideTimer = nil
    					
    				end
    			else
    				removeEventHandler("onClientRender", root, process)
    			end
    			
    			processing = false
    			return
    		end
    		
    		
    		-- In case of calling the function callNextFrame within the process, the loop type `repeat until` is required.
    		repeat
    			local item = nextFrameCalls[1]
    			item.callback(unpack(item.content))
    			tableRemove(nextFrameCalls, 1)
    		until #nextFrameCalls == 0

    	end
    	
    	
    	
    	function callNextFrame (callback, ...)
    		if type(callback) == "function" then
    			local newIndex = #nextFrameCalls + 1
    			nextFrameCalls[newIndex] = {callback = callback, content = {...}}
    			if not processing then
    				if serverSide then
    					serverSideTimer = setTimer(process, 50, 0)
    				else
    					addEventHandler("onClientRender", root, process)
    				end
    				processing = true
    			end
    			return true
    		end
    		return false
    	end
    end

 

 

 

The thing you better can do, is sending a message back to clientside and reset the change. + WARNING.

 

Or a just as risky, keep a table ready with all updates. When the user releases it's edit, update all changes at once. The release of the edit should also be triggered or when the player leaves. And it might also be handy to release it when the resource is going to stop.

 

And the most enchanted method, update all clients that are editing as well instead of blocking!

 

 

Edited March 22, 2019 by IIYAMA

[Sorata_Kanda]

I'm not sure if it's really worth the resources to block events from being called multiple times simultaneously as I don't think that there's a high chance 25 people are locking/unlocking the same house at the same time.

I thought about having the serversided cache updated mainly, e. g. someone unlocks house -> set "locked" to false in house object in server table and in a certain interval, the changed table is pushed to MySQL database.

[IIYAMA]

2 hours ago, Sorata_Kanda said:

I'm not sure if it's really worth the resources to block events from being called multiple times simultaneously as I don't think that there's a high chance 25 people are locking/unlocking the same house at the same time.

I thought about having the serversided cache updated mainly, e. g. someone unlocks house -> set "locked" to false in house object in server table and in a certain interval, the changed table is pushed to MySQL database. 

You can also keep track of people that are allowed to edit the data.

And only open a UI for 1 person at the time, per house. + only let that person push updates on serverside.

[savour]

well, the way I prefer is tables and check in advance, like in your example, if the house is locked, it will be sent whenever the pickup is streamed and the layout stuff will be handled client-side in that exact moment without any delay, but if the house lock stats changed, the server will loop through the players in it's radius to let there client's know through events.