Questions about fetchRemote

Bilal135 · December 16, 2019

I'm trying to learn fetchRemote. Never used it before. For test purposes, I uploaded a file to 000webhost, and ran the code below,

server:

local vehicle_data 
fetchRemote('https://nightwalkersnw.000webhostapp.com/public_html/sultan.dff', function(err, data) 
 if (err) then print(err) return end 
 outputChatBox("Download started", root)
 vehicle_data = data 
end) 
  
addEventHandler('onPlayerJoin', root, function() 
 triggerLatentClientEvent(source, 'downloadVehicle', resourceRoot, vehicle_data) 
end)

client:

addEventHandler ( "downloadVehicle" , resourceRoot , 
    function (vehicle_data) 
        engineReplaceModel ( engineLoadDFF ( vehicle_data ) , 560 ) 
        outputChatBox("Download complete")
end)

Debugscript just outputs 'INFO: ERROR'. What could be the cause? Also, what if I wanted to download multiple vehicle files (dff and txd) without actually having to re-write this code for every file. I suppose making a table with all the links and looping through it?

IIYAMA · December 16, 2019

38 minutes ago, Lynch said:

Debugscript just outputs 'INFO: ERROR'. What could be the cause?

Did you check the link with a (000webhost not logged in) browser?

38 minutes ago, Lynch said:

I suppose making a table with all the links and looping through it?

Yes, you can do that. But don't make the queue too long.

Edited December 16, 2019 by IIYAMA

vicisdev · December 16, 2019

I see some possible problems there.

- The version of fetchRemote

- Callback args

- File binary

If you'll look at the wiki fetchRemote has tones of versions and maybe none of those are one of you are using. I used recently fetchRemote with the following sintax:

fetchRemote(string url, table requestOptions, function callback, table callbackArgs)

Maybe you can try it as well with some headers definitions.

The first argument of the callback function seems to be always the data and not the request information. Particulary with the fetchRemote above, the arguments are responseData, responseInfo and callbackArgs.

If you're not sure that it's downloading successfully, then you could try to create a file, write the response data in it and load the dff. Also, get the response code from the responseInfo to check if everything done ok.

When you download a file you receive its binary through the response data and maybe the file binary it's not exactly the dff information that's needed to load a model into the game. Let's say I have a txt file with "I'm a txt file" wrote in it. If I try to request this file I'll receive its binary not the "I'm a txt file".

Good luck

Edited December 16, 2019 by vicisdev

Bilal135 · December 16, 2019

Tried this time with another link. Download did start, and it outputted some weird stuff in debug, but the texture was not loaded in game. Is there any problem with the trigger?

-- server

local vehicle_data 
fetchRemote('http://gofile.io/?c=f6vZuA', function(err, data)
    if (err) then print(err) end
        outputChatBox("Download started", root)
        vehicle_data = data
end) 
  
addEventHandler('onPlayerJoin', root, function() 
    triggerLatentClientEvent(source, 'downloadVehicle', resourceRoot, vehicle_data) 
end) 

-- client

addEventHandler ( "downloadVehicle" , resourceRoot, 
    function (vehicle_data) 
        engineImportTXD(engineLoadTXD(vehicle_data), 560) 
        outputChatBox("Download complete") -- this check failed
end)

Tried creating a file. Did not work.

-- client

addEventHandler ( "downloadVehicle" , resourceRoot, 
    function (vehicle_data) 
        local file = fileCreate("sultan.txd")
        fileWrite(file, vehicle_data)
        engineImportTXD(engineLoadTXD(file), 560) 
        outputChatBox("Download complete")
end)

Edited December 16, 2019 by Lynch

vicisdev · December 16, 2019

1 hour ago, Lynch said:

Oh, okay. It seems you're getting the HTML page instead of the file. You need a direct download link like this one from your sultan.txd

Bilal135 · December 16, 2019

I replaced the link with a direct link. Errors in debug script disappeared, but the texture still did not load. Also, how do I approach fetchRemote with tables?

local links = {

    [560] = {txd = "https://cdn-20.anonfile.com/n9r7yfG3nd/e52c4b4d-1576530500/sultan.txd", 
    dff = "https://cdn-07.anonfile.com/99u8y2G1n9/35d1852b-1576531060/sultan.dff"}

}

local vehicle_data 
    fetchRemote('https://cdn-20.anonfile.com/n9r7yfG3nd/e52c4b4d-1576530500/sultan.txd', function(err, data)
    if (err) then print(err) end
        outputChatBox("Download started", root)
        vehicle_data = data
end) 
  
addEventHandler('onPlayerJoin', root, function() 
    triggerLatentClientEvent(source, 'downloadVehicle', resourceRoot, vehicle_data) 
end)

addEventHandler ( "downloadVehicle" , resourceRoot, 
    function (vehicle_data) 
        engineImportTXD(engineLoadTXD(vehicle_data), 560) 
        outputChatBox("Download complete")
end)

IIYAMA · December 16, 2019

1 hour ago, Lynch said:

Also, how do I approach fetchRemote with tables?

Just some inspiration, normally I do not write so much code on the forum, untested.

-- INPUT
local links = {
	{
		model = 560,
		txd = "https://cdn-20.anonfile.com/n9r7yfG3nd/e52c4b4d-1576530500/sultan.txd", 
		dff = "https://cdn-07.anonfile.com/99u8y2G1n9/35d1852b-1576531060/sultan.dff"
	}
}

-- OUTPUT
local modelDataList = {}


-- 
local fileTypeLoadOrder = {
	"col",
	"txd",
	"dff"
}

do
	-- This is a temporary function. Only used for the initial state.
	local fileRequest = function (err, data, linkIndex, fileType) 
		if (err) then print(err) end
		
		outputChatBox("Download started", root)
		
		local link = links[linkIndex]
		local model = link.model
		
		-- make a new container if not exist 
		local modelData = modelDataList[model]
		if not modelData then
			modelData = {
				fileCountRemaining = link.fileCount
			}
			modelDataList[model] = modelData
		end
		
		-- save the data
		modelData.model = model
		modelData[fileType] = data
		
		modelData.fileCountRemaining = modelData.fileCountRemaining - 1
		
		-- clean up
		if modelData.fileCountRemaining == 0 then
			modelData.fileCountRemaining = nil
			modelData.ready = true
		end
	end
	
	
	-- Loop through all the links
	for i=1, #links do
		local link = links[i]
		
		-- Keep track of the amount of files that need to be downloaded
		local fileCount = 0
		
		-- Loop through all the file types in order.
		for j=1, #fileTypeLoadOrder do
			local fileType = fileTypeLoadOrder[j]
			local url = link[fileType]
			if url then -- If file type has an URL, then try to download it.
				fetchRemote(url, fileRequest, "", false, i, fileType)
				fileCount = fileCount + 1 -- + file
			end
		end
		link.fileCount = fileCount
	end
end

vicisdev · December 17, 2019

The IIYAMA's logic works perfectly, but I needed to make some tweaks. For some reason in his code the argument data is always 0. When I changed the fetchRemote implementation to the one mentioned above, it works ok.

Another one is that, I don't know why, but triggerLatentClientEvent isn't called when passing modelDataList as an argument. Switched it to triggerClientEvent and it works.

Bilal135 · December 17, 2019

On 16/12/2019 at 17:16, vicisdev said:

The IIYAMA's logic works perfectly, but I needed to make some tweaks. For some reason in his code the argument data is always 0. When I changed the fetchRemote implementation to the one mentioned above, it works ok.

Another one is that, I don't know why, but triggerLatentClientEvent isn't called when passing modelDataList as an argument. Switched it to triggerClientEvent and it works.

If you could show me your new implementation, that would be of great help.

Would this be the right way to receive and load the data in client side?

addEvent("downloadVehicle", true)
addEventHandler ( "downloadVehicle" , resourceRoot, 
    function (modelDataList)
        for k, v in ipairs(modelDataList) do
            local model = k
            outputChatBox(""..model.."")
            local txd = v.txd
            local dff = v.dff
        engineImportTXD(engineLoadTXD(txd), model) 
        engineReplaceModel(engineLoadDFF(dff), model)
        setTimer(function()
        outputChatBox("Download complete")
        end, 10000, 1)
        end
end)

Also, it seems that the script would download the files every time the player joins. Is it possible to make the player download the files only once?

IIYAMA · December 17, 2019

13 hours ago, vicisdev said:

I don't know why, but triggerLatentClientEvent isn't called when passing modelDataList as an argument.

It probably it takes too long to transfer the data without changing the bandwidth settings. The triggerClientEvent is very aggressive and will be high prioritised as well as blocking everything else untill it is finished.

Edited December 17, 2019 by IIYAMA

vicisdev · December 17, 2019

6 hours ago, Lynch said:

If you could show me your new implementation, that would be of great help

I changed the fetchRemote at line 66

fetchRemote(url, {}, fileRequest, {i, fileType})

And the fileRequest parameters to match the fetchRemote

local fileRequest = function (data, err, linkIndex, fileType) 
  if not err.success then print(err.statusCode) end
  ...

Have you tried downloadFile()? It seems to do exactly what you need.
I made an alternative code using this function. It's not good, it's a concept... but let me explain how it works.

To use downloadFile() it's needed to store your file paths in the meta.xml with download attribute set to false. In order to let you create only one file, I made a trick using the meta.xml as a config file as well. I don't know the implications of it - IIYAMA can tell us - but this way you're going to be able to loop through the meta.xml and get the <file> tags. The meta.xml will look like this:

<file src="sultan.txd" download="false" modelid="560"/>
<file src="sultan.dff" download="false" modelid="560"/>

<!-- In this case the files are in the resource root folder -->

I added a extra attribute to store the modelid, so you can know for what model the file is related.
It's needed to put the <file> tags in the proper load oder (col, txd, dff).

So the code wil loop this meta file and store the informations to the fileModelList table then it will iterate this table and download all entries. The table will look like this:

local fileModelList = {
  ["sultan.txd"] = {
    model = "560",
    fileType = "txd"
  },
  ["sultan.dff"] = {
    model = "560",
    fileType = "dff"
  }
}

Client:

local fileModelList = {}

local downloadFiles = function()
  local metaNode = getResourceConfig("meta.xml")

  for _, v in pairs(xmlNodeGetChildren(metaNode)) do
    if xmlNodeGetName(v) == "file" then
      local fileName = xmlNodeGetAttribute(v, "src")
      local modelId = xmlNodeGetAttribute(v, "modelid")
      local fileType = gettok(fileName, 2, "\.")

      setTimer(downloadFile, 500, 1, fileName)
      -- In the localhost the download is so fast that one event isn't triggered

      -- downloadFile(fileName)

      fileModelList[fileName] = {
        model = modelId,
        fileType = fileType
      }
    end
  end
end

addEventHandler("onClientFileDownloadComplete", resourceRoot, function(fileName, success)
    if not success then return end -- Later you should do something to handle failed downloads

    local model = fileModelList[fileName].model
    local type = fileModelList[fileName].fileType

    if type == "txd" then
      engineImportTXD(engineLoadTXD(fileName), model)
    end

    if type == "dff" then
      engineReplaceModel(engineLoadDFF(fileName), model)
    end

    fileModelList[fileName] = nil
  end)

addEvent("downloadFiles", true)
addEventHandler("downloadFiles", resourceRoot, downloadFiles)

Server:

addEventHandler("onPlayerJoin", root, function()
    -- triggerLatentClientEvent(source, "downloadFiles",  10^7, true, resourceRoot)
    -- Even 10 MBytes didn't work
    
    triggerClientEvent(source, "downloadFiles", resourceRoot)
end)

You will need to mess with the triggerLatentClientEvent()

downloadFile() is good if you need to postpone the file downloads, but if you're going to download everything onPlayerJoin wouldn't it be better to let the server handle it?

23 minutes ago, IIYAMA said:

It probably it takes too long to transfer the data without changing the bandwidth settings. The triggerClientEvent is very aggressive and will be high prioritised as well as blocking everything else untill it finished

He's downloading 5 MB files (or even more). For his case, what's the correct value to put in the bandwidth parameter? I set it to 10^7 and didn't works lol

Edited December 17, 2019 by vicisdev

Bilal135 · December 17, 2019

Messed around with triggerLatentClientEvent and triggerClientEvent but both didn't work. Added the files in the resource and changed the meta.xml just like you said, but the models still do not load. Tried debugging this way,

addEventHandler("onClientFileDownloadComplete", resourceRoot, function(fileName, success)
    if not success then return outputChatBox("Download failed") end -- Later you should do something to handle failed downloads

    local model = fileModelList[fileName].model
    local type = fileModelList[fileName].fileType

    if type == "txd" then
        outputChatBox("Loading txd")
      engineImportTXD(engineLoadTXD(fileName), model)
    end

    if type == "dff" then
        outputChatBox("Loading dff")
      engineReplaceModel(engineLoadDFF(fileName), model)
    end

    fileModelList[fileName] = nil
  end)

addEvent("downloadFiles", true)
addEventHandler("downloadFiles", resourceRoot, downloadFiles)

There's no output so it's not being triggered. Where could the problem lie?

Edited December 17, 2019 by Lynch

IIYAMA · December 17, 2019

2 hours ago, vicisdev said:

For his case, what's the correct value to put in the bandwidth parameter? I set it to 10^7 and didn't works lol

You can check the status of latent events. Also you might have bugged it (happend to me once), so restarting the client and the server.

downloadFile doesn't work, as the files have to be initiated before the resource start. Without you can't even start the resource.

1 hour ago, Lynch said:

There's no output so it's not being triggered. Where could the problem lie?

Show serverside. Also keep in mind that you can't send files if they are not downloaded by the server yet.

onPlayerJoin can't be used for players that are already in the server.

Bilal135 · December 17, 2019

This is the server side,

-- INPUT
local links = {
	{
		model = 560,
		txd = "http://drive.google.com/uc?export=download&id=1XIlsWiphME11AXW94cQGtOGJl4hMEX9V", 
		dff = "http://drive.google.com/uc?export=download&id=1exS-vr_kYp-tnD4xiCs8qW53zEAqT_S3"
	}
}

-- OUTPUT
local modelDataList = {}

local fileModelList = {
	["sultan.txd"] = {
	  model = "560",
	  fileType = "txd"
	},
	["sultan.dff"] = {
	  model = "560",
	  fileType = "dff"
	}
  }
-- 
local fileTypeLoadOrder = {
	"col",
	"txd",
	"dff"
}

do
	-- This is a temporary function. Only used for the initial state.
		local fileRequest = function (data, err, linkIndex, fileType) 
			if not err.success then print(err.statusCode) end
		
		outputChatBox("Download started", root)
		
		local link = links[linkIndex]
		local model = link.model
		
		-- make a new container if not exist 
		local modelData = modelDataList[model]
		if not modelData then
			modelData = {
				fileCountRemaining = link.fileCount
			}
			modelDataList[model] = modelData
		end
		
		-- save the data
		modelData.model = model
		modelData[fileType] = data
		
		modelData.fileCountRemaining = modelData.fileCountRemaining - 1
		
		-- clean up
		if modelData.fileCountRemaining == 0 then
			modelData.fileCountRemaining = nil
			modelData.ready = true
		end
	end
	
	
	-- Loop through all the links
	for i=1, #links do
		local link = links[i]
		
		-- Keep track of the amount of files that need to be downloaded
		local fileCount = 0
		
		-- Loop through all the file types in order.
		for j=1, #fileTypeLoadOrder do
			local fileType = fileTypeLoadOrder[j]
			local url = link[fileType]
			if url then -- If file type has an URL, then try to download it.
				fetchRemote(url, {}, fileRequest, {i, fileType})
				fileCount = fileCount + 1 -- + file
			end
		end
		link.fileCount = fileCount
	end
end


addEventHandler("onPlayerJoin", root, function()
    --triggerLatentClientEvent(source, "downloadFiles",  10^7, true, resourceRoot)
	--Even 10 MBytes didn't work
	setTimer(function()
	triggerClientEvent(source, "downloadFiles", resourceRoot)
	outputChatBox("Download triggered", root)
	end, 10000, 1)
end)

Edited December 17, 2019 by Lynch

IIYAMA · December 17, 2019

1 hour ago, Lynch said:

triggerClientEvent(source, "downloadFiles", resourceRoot)

You are sending nothing. Maybe it is a good idea to send what you want to send.

And is 10 seconds delay enough?

Edited December 17, 2019 by IIYAMA

vicisdev · December 17, 2019

Just now, IIYAMA said:

downloadFile doesn't work, as the files have to be initiated before the resource start. Without you can't even start the resource.

Yes, of course. I forgot to mention that. It's needed to have the files in the resource's folder so you can tell to the meta.xml where it is.

Just now, Lynch said:

This is the server side

Just now, IIYAMA said:

You are sending nothing. Maybe it is a good idea to send what you want to send.

Okay. We are conflicting ideas here lol. The last approach I sent it's not complementary of IIYAMA's approach - you can't put them together.

Edited December 17, 2019 by vicisdev

Bilal135 · December 17, 2019

I apologize I got a little confused between the two different implementations.

Continuing @IIYAMA's code with @vicisdev's server side implementation, this is the final code. It still does not work tho. No errors in debug. Should I try restarting the server?

EDIT: Actually there's an error. Attempt to concatenate global 'model', a nil value. Working on it.

local links = {
	{
		model = 560,
		txd = "https://drive.google.com/uc?export=download&id=1XIlsWiphME11AXW94cQGtOGJl4hMEX9V", 
		dff = "https://drive.google.com/uc?export=download&id=1exS-vr_kYp-tnD4xiCs8qW53zEAqT_S3"
	}
}

-- OUTPUT
local modelDataList = {}


-- 
local fileTypeLoadOrder = {
	"col",
	"txd",
	"dff"
}

do
	-- This is a temporary function. Only used for the initial state.
		local fileRequest = function (data, err, linkIndex, fileType) 
			if not err.success then print(err.statusCode) end
		
		outputChatBox("Download started", root)
		
		local link = links[linkIndex]
		local model = link.model
		
		-- make a new container if not exist 
		local modelData = modelDataList[model]
		if not modelData then
			modelData = {
				fileCountRemaining = link.fileCount
			}
			modelDataList[model] = modelData
		end
		
		-- save the data
		modelData.model = model
		modelData[fileType] = data
		
		modelData.fileCountRemaining = modelData.fileCountRemaining - 1
		
		-- clean up
		if modelData.fileCountRemaining == 0 then
			modelData.fileCountRemaining = nil
			modelData.ready = true
		end
	end
	
	
	-- Loop through all the links
	for i=1, #links do
		local link = links[i]
		
		-- Keep track of the amount of files that need to be downloaded
		local fileCount = 0
		
		-- Loop through all the file types in order.
		for j=1, #fileTypeLoadOrder do
			local fileType = fileTypeLoadOrder[j]
			local url = link[fileType]
			if url then -- If file type has an URL, then try to download it.
				fetchRemote(url, {}, fileRequest, {i, fileType})
				fileCount = fileCount + 1 -- + file
			end
		end
		link.fileCount = fileCount
	end
end

addEventHandler('onPlayerJoin', root, function()
	triggerLatentClientEvent(source, 'downloadVehicle', 10^7, true, resourceRoot, model, data) 
end)

addEvent("downloadVehicle", true)
addEventHandler ( "downloadVehicle" , resourceRoot, 
    function (model, data)
        if data == "txd" then
            engineImportTXD(engineLoadTXD(txd), model) 
        elseif data == "dff" then
            engineReplaceModel(engineLoadDFF(dff), model)
        end
    end
)

Edited December 17, 2019 by Lynch

IIYAMA · December 17, 2019

2 minutes ago, Lynch said:

triggerLatentClientEvent(source, 'downloadVehicle', 10^7, true, resourceRoot, model, data)

Model and data? Where is that defined?

The modelDataList from line 10 is what you have to send, when the server is finished with getting the data.

Bilal135 · December 17, 2019

Something like this then?

addEvent("downloadVehicle", true)
addEventHandler ( "downloadVehicle" , resourceRoot, 
    function (modelDataList)
        for k, v in pairs(modelDataList) do
            local model = modelDataList.model
            local data = modelDataList[fileType]
            if data == "txd" then
                outputChatBox("loading txd")
                engineImportTXD(engineLoadTXD(data), model) 
            end 
            if data == "dff" then
                outputChatBox("loading dff")
                engineReplaceModel(engineLoadDFF(data), model)
            end
        end
    end
)

Edited December 17, 2019 by Lynch

IIYAMA · December 17, 2019

1 minute ago, Lynch said:

Something like this then?

getting better

local col = modelDataList.col
if col then
	local col_loaded = engineLoadCOL (col)
	if col_loaded then
		engineReplaceCOL (col_loaded, model )
	end
end

local txd = modelDataList.txd
if txd then
  	local txd_loaded = engineLoadTXD(txd)
  	if txd_loaded then
		engineImportTXD(txd_loaded, model)
	end
end

local dff = modelDataList.dff
if dff then
	local dff_loaded = engineLoadDFF(dff)
	if dff_loaded then
		engineReplaceModel(dff_loaded, model)
	end
end

Bilal135 · December 17, 2019

Errors are gone but none of the test outputs appear upon reconnecting.

addEvent("downloadVehicle", true)
addEventHandler ( "downloadVehicle" , resourceRoot, 
    function (modelDataList)
        for k, v in pairs(modelDataList) do

            local col = modelDataList.col
            if col then
                local col_loaded = engineLoadCOL (col)
                if col_loaded then
                    engineReplaceCOL (col_loaded, model )
                    outputChatBox("replacing col")
                end
            end
            
            local txd = modelDataList.txd
            if txd then
                  local txd_loaded = engineLoadTXD(txd)
                  if txd_loaded then
                    engineImportTXD(txd_loaded, model)
                    outputChatBox("replacing txd")
                end
            end
            
            local dff = modelDataList.dff
            if dff then
                local dff_loaded = engineLoadDFF(dff)
                if dff_loaded then
                    engineReplaceModel(dff_loaded, model)
                    outputChatBox("replacing dff")
                end
            end

        end
    end
)

IIYAMA · December 17, 2019

Just now, Lynch said:

Errors are gone but none of the test outputs appear upon reconnecting.

Did you debug the data?

Bilal135 · December 17, 2019

I did. Turns out 'k' returns the value of model. While modelDataList is replaced by v. It works. I can see the new car model in game.

That being said, next question is, is the file downloaded every time the player joins even if he has downloaded it before on his previous visits to the server? Because I wouldn't want player to wait 30 minutes before all files are downloaded every time he joins the server. EDIT: I think I've found the answer. It only downloads once which is very good.

2. Is it possible to show how much % of the file has been downloaded through onClientRender and dxDraw functions?

Edited December 17, 2019 by Lynch

IIYAMA · December 17, 2019

9 minutes ago, Lynch said:

Because I wouldn't want player to wait 30 minutes before all files are downloaded every time he joins the server.

Yes that is happening.

If you do not want that to happen, you have to do a lot more work.

The first part is file management. "Which files does the client have downloaded?" You can use this resource for keeping track of the file locations and statuses.

https://community.multitheftauto.com/index.php?p=resources&s=details&id=16187

And for writing the files:

https://wiki.multitheftauto.com/wiki/FileCreate

1. Client

[resource:xmldata] These are the files I have on my computer! > triggerServerEvent

2. Server

These files are available to download.

Filter out the files that have already been downloaded. > triggerClientEvent

3.Client

Create new files fileCreate() and update the file management [resource:xmldata].

21 minutes ago, Lynch said:

2. Is it possible to show how much % of the file has been downloaded through onClientRender and dxDraw functions?

Yes, that is very much possible.

triggerLatentClientEvent(player, ...)
local handles = getLatentEventHandles (player)
local handle  = handles[#handles]
local status = getLatentEventStatus(player, handle)

iprint((status.percentComplete or 100) .. "%")

Bilal135 · December 17, 2019

Can you elaborate the process in a step by step a bit more please so its easier to understand?

Questions about fetchRemote

Recommended Posts

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members