Page 1 of 1

Experimental Rithmic adapter build

Posted: Tue Jul 07, 2020 2:36 pm
by Svyatoslav
Hi everyone,
Some of you have reported performance issues when using Rithmic data. Specifically I'm speaking about the issue where you see "Data latency spike detected" warning. So, we made certain (somewhat risky - might cause deadlocks and/or memory leaks that we'll need to catch before it's production ready) optimizations to our code interfacing with Rithmic adapter. Basically for most frequent update types we copy the data as soon as we can and let R API code keep running. We do not know if this ever was a bottleneck, so before we start deploying it I'd like to gather some feedback, and that's where I need your help.

So, if you are willing to try, please make sure not to use it for trading. It was NOT through QA, and one of failure modes I'm concerned about is if it freezes on some order-related update. You will be proceeding at your own risk, so ideally just try it with paper trading.

The steps to install this version are below:
1. Make sure you are running Windows 10. This adapter is built with Win10 Rithmic libraries and I don't know if those will work on older systems at all, and if those do - if performance will be affected. If things work out - we will apply same changes to MacOS and Linux versions.
2. Download this experimental build: https://drive.google.com/file/d/1_qYqfK ... sp=sharing
3. Install it. It's not signed, so Windows might complain.
4. Start bookmap.
5. If it freezes (it should not, but it might) - kill it and try starting again. There might be concurrency issues in the startup logic, that's just an early prototype, main purpose is to understand the lags issue better.
6. After Bookmap starts and you connect it to Rithmic the following log file should appear "C:\Bookmap\Logs\rithmic-engine-log-bm.txt". If it did - you are now running the new experimental adapter.

Let me know your feedback.
If it does not help (you still see significant lag warnings, like 3+ seconds of lag) - please send me (here or via support - keep in mind that those file contain your login, so maybe don't send it publicly) "C:\Bookmap\Logs\rithmic-engine-log-bm.txt", "C:\Bookmap\Logs\rithmic-engine-log.000" and corresponding bookmap -common log ("C:\Bookmap\Logs\log_*-common.txt"). First one contains some data that should allow to differentiate "Network or R API too slow" from "Bookmap code too slow" (keep in mind that file is wiped every time bookmap starts, so send it after closing Bookmap but before starting it again).
If it does help - please be mindful of confirmation bias. Ideally go back to "standard" version, see if things get worse, go back to experimental again, repeat a few times.

Thanks,
Svyatoslav

Re: Experimental Rithmic adapter build

Posted: Mon Jul 13, 2020 2:55 pm
by dap
Today I had cpu to almost 100% for an hour [task manager affinity didn't work] which caused latencies (at max 3 seconds) and a slow PC.
Then I closed r trader and cpu returned to 10/20%, I suppose it was rtrader that slowed everything down, even the PC (all rtrader icons were green during high load cpu).

In last 2 weeks everything was fine with almost no latencies (and using the regular build).
Last friday I used the experimental build, no delays, everything normal. 

Re: Experimental Rithmic adapter build

Posted: Mon Jul 13, 2020 2:59 pm
by Svyatoslav
About R Trader... What is the CPU in that machine? I was under impression R Trader uses up to 2 threads, so not sure how it could do that...

Re: Experimental Rithmic adapter build

Posted: Mon Jul 13, 2020 3:15 pm
by dap
Svyatoslav wrote:
Mon Jul 13, 2020 2:59 pm
About R Trader... What is the CPU in that machine? I was under impression R Trader uses up to 2 threads, so not sure how it could do that...

I have ryzen 1700x 

Yes during the previous tests I've seen 2 thread with 100% cpu form task manager.
Today I had 90/100% only in cpu0 so I tried to set the affinity, to switch rtrader in the latter cores, but in that case nothing changed/moved (normally the 100% will switch to other cores via the affinity). 

So maybe could be something else, but shutting down rtrader released almost all cpu usage