JavaScript 函数阻止 Web 套接字和导致同步问题&延迟

发布于 2025-01-09 05:12:16 字数 9475 浏览 0 评论 0原文

我有一个 Web 套接字,每 100 到 200 毫秒从 Web 套接字服务器接收数据(我已经尝试过共享 Web Worker 以及 main.js 文件中的所有内容),

当新的 JSON 数据到达时,我的 main.js 运行filter_json_run_all(json_data) 更新 Tabulator.js & Dygraph.js 表和根据值是否增加或减少而使用一些自定义颜色编码的图表

1) Web 套接字 json 数据(每 100 毫秒或更短)-> 2)运行函数filter_json_run_all(json_data)(需要150到200毫秒)-> 3)重复1& 2 永远

由于filter_json_run_all 导致操作积压,传入 json 数据的时间戳很快就会相对于实际时间延迟(json_time 15:30:12 与实际时间:15:31:30)。

因此,它会导致不同 PC 上的用户根据打开或刷新网站的时间出现 websocket 同步问题。

这只是由长filter_json_run_all()函数引起的,否则如果我所做的只是console.log(json_data),它们将完全同步。

如果有人有任何想法,我将非常感激如何防止这种由运行缓慢的 javascript 引起的传入 JSON websocket 数据的阻塞/积压 函数:)

我尝试使用一个共享的网络工作者,它可以工作,但它不能解决被filter_json_run_all()阻止的main.js中的延迟,我不认为我可以把filter_json_run_all(),因为所有的图表&表对象在 main 和 main 中定义。当我点击表格手动更新值时,我也有回调(双向网络套接字)

如果您有任何想法或提示,我将非常感激:)

worker.js:

    const connectedPorts = [];

    // Create socket instance.
 var socket = new WebSocket(
    'ws://'
    + 'ip:port'
    + '/ws/'
);
 
 // Send initial package on open.
 socket.addEventListener('open', () => {
   const package = JSON.stringify({
     "time": 123456,
     "channel": "futures.tickers",
     "event": "subscribe",
     "payload": ["BTC_USD", "ETH_USD"]
   });

   
 
   socket.send(package);
 });
 
 // Send data from socket to all open tabs.
 socket.addEventListener('message', ({ data }) => {
   const package = JSON.parse(data);
   connectedPorts.forEach(port => port.postMessage(package));
 });
 
 /**
  * When a new thread is connected to the shared worker,
  * start listening for messages from the new thread.
  */
 self.addEventListener('connect', ({ ports }) => {
   const port = ports[0];
 
   // Add this new port to the list of connected ports.
   connectedPorts.push(port);
 
   /**
    * Receive data from main thread and determine which
    * actions it should take based on the received data.
    */
   port.addEventListener('message', ({ data }) => {
     const { action, value } = data;
 
     // Send message to socket.
     if (action === 'send') {
       socket.send(JSON.stringify(value));
 
     // Remove port from connected ports list.
     } else if (action === 'unload') {
       const index = connectedPorts.indexOf(port);
       connectedPorts.splice(index, 1);
     }
   });

< strong>Main.js 这只是 filter_json_run_all 的一部分,它会持续大约 6 或 7 个制表符和Dygraph 对象。我想给出一些使用 SetTimeout() 等调用的操作的想法

    function filter_json_run_all(json_str){
                const startTime = performance.now();
                const data_in_array = json_str //JSON.parse(json_str.data);
    
                // if ('DATETIME' in data_in_array){
    
    
                //     var milliseconds = (new Date()).getTime() - Date.parse(data_in_array['DATETIME']);
                //     console.log("milliseconds: " + milliseconds);
                // }
            
                if (summary in data_in_array){
                    
                    if("DATETIME" in data_in_array){
                        var time_str = data_in_array["DATETIME"];
                        element_time.innerHTML = time_str;
                    }
    
                    // summary Data
                    const summary_array = data_in_array[summary];
                    var old_sum_arr_krw = [];
                    var old_sum_arr_irn = [];
                    var old_sum_arr_ntn = [];
                    var old_sum_arr_ccn = [];
                    var old_sum_arr_ihn = [];
                    var old_sum_arr_ppn = [];
    
    
    
    
                    var filtered_array_krw_summary = filterByProperty_summary(summary_array, "KWN")
                    old_sum_arr_krw.unshift(Table_summary_krw.getData());
                    Table_summary_krw.replaceData(filtered_array_krw_summary);
                    //Colour table
                    color_table(filtered_array_krw_summary, old_sum_arr_krw, Table_summary_krw);
    
                    var filtered_array_irn_summary = filterByProperty_summary(summary_array, "IRN")
                    old_sum_arr_irn.unshift(Table_summary_inr.getData());
                    Table_summary_inr.replaceData(filtered_array_irn_summary);
                    //Colour table
                    color_table(filtered_array_irn_summary, old_sum_arr_irn, Table_summary_inr);
    
                    var filtered_array_ntn_summary = filterByProperty_summary(summary_array, "NTN")
                    old_sum_arr_ntn.unshift(Table_summary_twd.getData());
                    Table_summary_twd.replaceData(filtered_array_ntn_summary);
                    //Colour table
                    color_table(filtered_array_ntn_summary, old_sum_arr_ntn, Table_summary_twd);
    
              // remove formatting on fwds curves
                setTimeout(() => {g_fwd_curve_krw.updateOptions({
                'file': dataFwdKRW,
                'labels': ['Time', 'Bid', 'Ask'],
                strokeWidth: 1,
                }); }, 200);
                setTimeout(() => {g_fwd_curve_inr.updateOptions({
                'file': dataFwdINR,
                'labels': ['Time', 'Bid', 'Ask'],
                strokeWidth: 1,
                }); }, 200);

// remove_colors //([askTable_krw, askTable_inr, askTable_twd, askTable_cny, askTable_idr, askTable_php])              
                setTimeout(() => {  askTable_krw.getRows().forEach(function (item, index) {
                    row = item.getCells();
                    row.forEach(function (value_tmp){value_tmp.getElement().style.backgroundColor = '';}
                )}); }, 200);
                setTimeout(() => {  askTable_inr.getRows().forEach(function (item, index) {
                    row = item.getCells();
                    row.forEach(function (value_tmp){value_tmp.getElement().style.backgroundColor = '';}
                )}); }, 200);

color_table 函数

function color_table(new_arr, old_array, table_obj){

            // If length is not equal
            if(new_arr.length!=old_array[0].length)
                    console.log("Diff length");
                else
                {
                    
                // Comparing each element of array
                    for(var i=0;i<new_arr.length;i++)

                        //iterate old dict dict
                        for (const [key, value] of Object.entries(old_array[0][i])) {
                            
                            if(value == new_arr[i][key])
                                {}
                            else{
                                // console.log("Different element");
                                if(key!="TENOR")
                                // console.log(table_obj)
                                    table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'yellow';
                                if(key!="TIME")
                                    if(value < new_arr[i][key])
                                        //green going up
                                        //text_to_speech(new_arr[i]['CCY'] + ' ' +new_arr[i]['TENOR']+ ' getting bid')
                                        table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'Chartreuse';
                                if(key!="TIME")
                                    if(value > new_arr[i][key])
                                        //red going down
                                        table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'Crimson';
                            }
                                                                                 
                            }
                }
        }

潜在的软糖/解决方案,谢谢 Aaron :):

 function limiter(fn, wait){
            let isCalled = false,
                calls = [];

            let caller = function(){
                if (calls.length && !isCalled){
                    isCalled = true;
                    if (calls.length >2){
                        calls.splice(0,calls.length-1)
                        //remove zero' upto n-1 function calls from array/ queue
                    }
                    calls.shift().call();
                    setTimeout(function(){
                        isCalled = false;
                        caller();
                    }, wait);
                }
            };
            return function(){
                calls.push(fn.bind(this, ...arguments));
                // let args = Array.prototype.slice.call(arguments);
                // calls.push(fn.bind.apply(fn, [this].concat(args)));

                caller();
            };
        }

这是然后定义的作为网络工作人员调用的常量:

const filter_json_run_allLimited = limiter(data => { filter_json_run_all(data); }, 300); // 300ms for examples

当新的网络套接字数据到达时,网络工作人员调用有限功能:

// Event to listen for incoming data from the worker and update the DOM.
        webSocketWorker.port.addEventListener('message', ({ data }) => {
           // Limited function 
            filter_json_run_allLimited(data);
            
           
           
        });

如果有人知道交易视图或实时高性能等网站数据流站点可实现低延迟可视化更新了,请大家评论,在下面回复:)

I have a web socket that receives data from a web socket server every 100 to 200ms, ( I have tried both with a shared web worker as well as all in the main.js file),

When new JSON data arrives my main.js runs filter_json_run_all(json_data) which updates Tabulator.js & Dygraph.js Tables & Graphs with some custom color coding based on if values are increasing or decreasing

1) web socket json data ( every 100ms or less) -> 2) run function filter_json_run_all(json_data) (takes 150 to 200ms) -> 3) repeat 1 & 2 forever

Quickly the timestamp of the incoming json data gets delayed versus the actual time (json_time 15:30:12 vs actual time: 15:31:30) since the filter_json_run_all is causing a backlog in operations.

So it causes users on different PC's to have websocket sync issues, based on when they opened or refreshed the website.

This is only caused by the long filter_json_run_all() function, otherwise if all I did was console.log(json_data) they would be perfectly in sync.

Please I would be very very grateful if anyone has any ideas how I can prevent this sort of blocking / backlog of incoming JSON websocket data caused by a slow running javascript
function :)

I tried using a shared web worker which works but it doesn't get around the delay in main.js blocked by filter_json_run_all(), I dont thing I can put filter_json_run_all() since all the graph & table objects are defined in main & also I have callbacks for when I click on a table to update a value manually (Bi directional web socket)

If you have any ideas or tips at all I will be very grateful :)

worker.js:

    const connectedPorts = [];

    // Create socket instance.
 var socket = new WebSocket(
    'ws://'
    + 'ip:port'
    + '/ws/'
);
 
 // Send initial package on open.
 socket.addEventListener('open', () => {
   const package = JSON.stringify({
     "time": 123456,
     "channel": "futures.tickers",
     "event": "subscribe",
     "payload": ["BTC_USD", "ETH_USD"]
   });

   
 
   socket.send(package);
 });
 
 // Send data from socket to all open tabs.
 socket.addEventListener('message', ({ data }) => {
   const package = JSON.parse(data);
   connectedPorts.forEach(port => port.postMessage(package));
 });
 
 /**
  * When a new thread is connected to the shared worker,
  * start listening for messages from the new thread.
  */
 self.addEventListener('connect', ({ ports }) => {
   const port = ports[0];
 
   // Add this new port to the list of connected ports.
   connectedPorts.push(port);
 
   /**
    * Receive data from main thread and determine which
    * actions it should take based on the received data.
    */
   port.addEventListener('message', ({ data }) => {
     const { action, value } = data;
 
     // Send message to socket.
     if (action === 'send') {
       socket.send(JSON.stringify(value));
 
     // Remove port from connected ports list.
     } else if (action === 'unload') {
       const index = connectedPorts.indexOf(port);
       connectedPorts.splice(index, 1);
     }
   });

Main.js This is only part of filter_json_run_all which continues on for about 6 or 7 Tabulator & Dygraph objects. I wante to give an idea of some of the operations called with SetTimeout() etc

    function filter_json_run_all(json_str){
                const startTime = performance.now();
                const data_in_array = json_str //JSON.parse(json_str.data);
    
                // if ('DATETIME' in data_in_array){
    
    
                //     var milliseconds = (new Date()).getTime() - Date.parse(data_in_array['DATETIME']);
                //     console.log("milliseconds: " + milliseconds);
                // }
            
                if (summary in data_in_array){
                    
                    if("DATETIME" in data_in_array){
                        var time_str = data_in_array["DATETIME"];
                        element_time.innerHTML = time_str;
                    }
    
                    // summary Data
                    const summary_array = data_in_array[summary];
                    var old_sum_arr_krw = [];
                    var old_sum_arr_irn = [];
                    var old_sum_arr_ntn = [];
                    var old_sum_arr_ccn = [];
                    var old_sum_arr_ihn = [];
                    var old_sum_arr_ppn = [];
    
    
    
    
                    var filtered_array_krw_summary = filterByProperty_summary(summary_array, "KWN")
                    old_sum_arr_krw.unshift(Table_summary_krw.getData());
                    Table_summary_krw.replaceData(filtered_array_krw_summary);
                    //Colour table
                    color_table(filtered_array_krw_summary, old_sum_arr_krw, Table_summary_krw);
    
                    var filtered_array_irn_summary = filterByProperty_summary(summary_array, "IRN")
                    old_sum_arr_irn.unshift(Table_summary_inr.getData());
                    Table_summary_inr.replaceData(filtered_array_irn_summary);
                    //Colour table
                    color_table(filtered_array_irn_summary, old_sum_arr_irn, Table_summary_inr);
    
                    var filtered_array_ntn_summary = filterByProperty_summary(summary_array, "NTN")
                    old_sum_arr_ntn.unshift(Table_summary_twd.getData());
                    Table_summary_twd.replaceData(filtered_array_ntn_summary);
                    //Colour table
                    color_table(filtered_array_ntn_summary, old_sum_arr_ntn, Table_summary_twd);
    
              // remove formatting on fwds curves
                setTimeout(() => {g_fwd_curve_krw.updateOptions({
                'file': dataFwdKRW,
                'labels': ['Time', 'Bid', 'Ask'],
                strokeWidth: 1,
                }); }, 200);
                setTimeout(() => {g_fwd_curve_inr.updateOptions({
                'file': dataFwdINR,
                'labels': ['Time', 'Bid', 'Ask'],
                strokeWidth: 1,
                }); }, 200);

// remove_colors //([askTable_krw, askTable_inr, askTable_twd, askTable_cny, askTable_idr, askTable_php])              
                setTimeout(() => {  askTable_krw.getRows().forEach(function (item, index) {
                    row = item.getCells();
                    row.forEach(function (value_tmp){value_tmp.getElement().style.backgroundColor = '';}
                )}); }, 200);
                setTimeout(() => {  askTable_inr.getRows().forEach(function (item, index) {
                    row = item.getCells();
                    row.forEach(function (value_tmp){value_tmp.getElement().style.backgroundColor = '';}
                )}); }, 200);

color_table Function

function color_table(new_arr, old_array, table_obj){

            // If length is not equal
            if(new_arr.length!=old_array[0].length)
                    console.log("Diff length");
                else
                {
                    
                // Comparing each element of array
                    for(var i=0;i<new_arr.length;i++)

                        //iterate old dict dict
                        for (const [key, value] of Object.entries(old_array[0][i])) {
                            
                            if(value == new_arr[i][key])
                                {}
                            else{
                                // console.log("Different element");
                                if(key!="TENOR")
                                // console.log(table_obj)
                                    table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'yellow';
                                if(key!="TIME")
                                    if(value < new_arr[i][key])
                                        //green going up
                                        //text_to_speech(new_arr[i]['CCY'] + ' ' +new_arr[i]['TENOR']+ ' getting bid')
                                        table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'Chartreuse';
                                if(key!="TIME")
                                    if(value > new_arr[i][key])
                                        //red going down
                                        table_obj.getRows()[i].getCell(key).getElement().style.backgroundColor = 'Crimson';
                            }
                                                                                 
                            }
                }
        }

Potential fudge / solution, thanks Aaron :):

 function limiter(fn, wait){
            let isCalled = false,
                calls = [];

            let caller = function(){
                if (calls.length && !isCalled){
                    isCalled = true;
                    if (calls.length >2){
                        calls.splice(0,calls.length-1)
                        //remove zero' upto n-1 function calls from array/ queue
                    }
                    calls.shift().call();
                    setTimeout(function(){
                        isCalled = false;
                        caller();
                    }, wait);
                }
            };
            return function(){
                calls.push(fn.bind(this, ...arguments));
                // let args = Array.prototype.slice.call(arguments);
                // calls.push(fn.bind.apply(fn, [this].concat(args)));

                caller();
            };
        }

This is then defined as a constant for a web worker to call:

const filter_json_run_allLimited = limiter(data => { filter_json_run_all(data); }, 300); // 300ms for examples

Web worker calls the limited function when new web socket data arrives:

// Event to listen for incoming data from the worker and update the DOM.
        webSocketWorker.port.addEventListener('message', ({ data }) => {
           // Limited function 
            filter_json_run_allLimited(data);
            
           
           
        });

Please if anyone knows how websites like tradingview or real time high performance data streaming sites allow for low latency visualisation updates, please may you comment, reply below :)

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

倥絔 2025-01-16 05:12:16

在不知道 color_table 中发生了什么的情况下,我不愿意尝试真正回答这个问题。根据您所描述的行为,我的预感是 filter_json_run_all 被迫等待拥塞的 DOM 操作/渲染管道,因为正在更新 HTML 以实现颜色- 更新的表格元素的编码。

我发现您已经采取了一些措施来防止某些 DOM 操作阻止该函数的执行(通过 setTimeout)。如果 color_table 尚未采用类似的策略,那么这将是我首先要重点关注的重构以疏通此处的内容。

可能还值得将已处理事件的这些 DOM 更新放入一个简单的队列中,这样,如果浏览器行为缓慢会导致渲染积压,则实际负责调用挂起的 DOM 操作的函数可以选择跳过过时的渲染操作,以保持 UI 可接受的敏捷性。

编辑:一个基本的排队系统可能涉及以下组件:

  1. 队列本身(这可以是一个简单的数组,它只需要可供下面的两个组件访问)。
  2. 队列附加程序,在 filter_json_run_all 期间运行,只需将对象添加到队列末尾,代表您计划使用 color_table 或 setTimeout` 回调之一完成的每个 DOM 操作作业。这些对象应该包含要执行的操作(即:未调用的函数定义)以及该操作的参数(即:传递给每个函数的参数)。
  3. 队列运行器按自己的时间间隔运行,并从队列前面调用挂起的 DOM 操作任务,并在运行时将其删除。由于此操作可以访问队列中的所有对象,因此它还可以采取措施来优化/组合类似的操作,以最大程度地减少在执行后续代码之前要求浏览器执行的重绘量。例如,如果您有多个 color_table 操作多次为同一单元格着色,则只需使用来自最后一个 color_table 项的数据执行此操作一次即可。涉及该单元的队列。此外,您可以通过在 requestAnimationFrame 回调,这将确保仅当浏览器准备就绪时才发生计划的回流/重绘,从性能角度来看比 DOM 操作更可取通过setTimeout/setInterval排队。

I'm reticent to take a stab at answering this for real without knowing what's going on in color_table. My hunch, based on the behavior you're describing is that filter_json_run_all is being forced to wait on a congested DOM manipulation/render pipeline as HTML is being updated to achieve the color-coding for your updated table elements.

I see you're already taking some measures to prevent some of these DOM manipulations from blocking this function's execution (via setTimeout). If color_table isn't already employing a similar strategy, that'd be the first thing I'd focus on refactoring to unclog things here.

It might also be worth throwing these DOM updates for processed events into a simple queue, so that if slow browser behavior creates a rendering backlog, the function actually responsible for invoking pending DOM manipulations can elect to skip outdated render operations to keep the UI acceptably snappy.

Edit: a basic queueing system might involve the following components:

  1. The queue, itself (this can be a simple array, it just needs to be accessible to both of the components below).
  2. A queue appender, which runs during filter_json_run_all, simply adding objects to the end of the queue representing each DOM manipulation job you plan to complete using color_table or one of your setTimeout` callbacks. These objects should contain the operation to performed (i.e: the function definition, uninvoked), and the parameters for that operation (i.e: the arguments you're passing into each function).
  3. A queue runner, which runs on its own interval, and invokes pending DOM manipulation tasks from the front of the queue, removing them as it goes. Since this operation has access to all of the objects in the queue, it can also take steps to optimize/combine similar operations to minimize the amount of repainting it's asking the browser to do before subsequent code can be executed. For example, if you've got several color_table operations that coloring the same cell multiple times, you can simply perform this operation once with the data from the last color_table item in the queue involving that cell. Additionally, you can further optimize your interaction with the DOM by invoking the aggregated DOM manipulation operations, themselves, inside a requestAnimationFrame callback, which will ensure that scheduled reflows/repaints happen only when the browser is ready, and is preferable from a performance perspective to DOM manipulation queueing via setTimeout/setInterval.
~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文