Experimental Rithmic adapter build
Posted: Tue Jul 07, 2020 2:36 pm
Hi everyone,
Some of you have reported performance issues when using Rithmic data. Specifically I'm speaking about the issue where you see "Data latency spike detected" warning. So, we made certain (somewhat risky - might cause deadlocks and/or memory leaks that we'll need to catch before it's production ready) optimizations to our code interfacing with Rithmic adapter. Basically for most frequent update types we copy the data as soon as we can and let R API code keep running. We do not know if this ever was a bottleneck, so before we start deploying it I'd like to gather some feedback, and that's where I need your help.
So, if you are willing to try, please make sure not to use it for trading. It was NOT through QA, and one of failure modes I'm concerned about is if it freezes on some order-related update. You will be proceeding at your own risk, so ideally just try it with paper trading.
The steps to install this version are below:
1. Make sure you are running Windows 10. This adapter is built with Win10 Rithmic libraries and I don't know if those will work on older systems at all, and if those do - if performance will be affected. If things work out - we will apply same changes to MacOS and Linux versions.
2. Download this experimental build: https://drive.google.com/file/d/1_qYqfK ... sp=sharing
3. Install it. It's not signed, so Windows might complain.
4. Start bookmap.
5. If it freezes (it should not, but it might) - kill it and try starting again. There might be concurrency issues in the startup logic, that's just an early prototype, main purpose is to understand the lags issue better.
6. After Bookmap starts and you connect it to Rithmic the following log file should appear "C:\Bookmap\Logs\rithmic-engine-log-bm.txt". If it did - you are now running the new experimental adapter.
Let me know your feedback.
If it does not help (you still see significant lag warnings, like 3+ seconds of lag) - please send me (here or via support - keep in mind that those file contain your login, so maybe don't send it publicly) "C:\Bookmap\Logs\rithmic-engine-log-bm.txt", "C:\Bookmap\Logs\rithmic-engine-log.000" and corresponding bookmap -common log ("C:\Bookmap\Logs\log_*-common.txt"). First one contains some data that should allow to differentiate "Network or R API too slow" from "Bookmap code too slow" (keep in mind that file is wiped every time bookmap starts, so send it after closing Bookmap but before starting it again).
If it does help - please be mindful of confirmation bias. Ideally go back to "standard" version, see if things get worse, go back to experimental again, repeat a few times.
Thanks,
Svyatoslav
Some of you have reported performance issues when using Rithmic data. Specifically I'm speaking about the issue where you see "Data latency spike detected" warning. So, we made certain (somewhat risky - might cause deadlocks and/or memory leaks that we'll need to catch before it's production ready) optimizations to our code interfacing with Rithmic adapter. Basically for most frequent update types we copy the data as soon as we can and let R API code keep running. We do not know if this ever was a bottleneck, so before we start deploying it I'd like to gather some feedback, and that's where I need your help.
So, if you are willing to try, please make sure not to use it for trading. It was NOT through QA, and one of failure modes I'm concerned about is if it freezes on some order-related update. You will be proceeding at your own risk, so ideally just try it with paper trading.
The steps to install this version are below:
1. Make sure you are running Windows 10. This adapter is built with Win10 Rithmic libraries and I don't know if those will work on older systems at all, and if those do - if performance will be affected. If things work out - we will apply same changes to MacOS and Linux versions.
2. Download this experimental build: https://drive.google.com/file/d/1_qYqfK ... sp=sharing
3. Install it. It's not signed, so Windows might complain.
4. Start bookmap.
5. If it freezes (it should not, but it might) - kill it and try starting again. There might be concurrency issues in the startup logic, that's just an early prototype, main purpose is to understand the lags issue better.
6. After Bookmap starts and you connect it to Rithmic the following log file should appear "C:\Bookmap\Logs\rithmic-engine-log-bm.txt". If it did - you are now running the new experimental adapter.
Let me know your feedback.
If it does not help (you still see significant lag warnings, like 3+ seconds of lag) - please send me (here or via support - keep in mind that those file contain your login, so maybe don't send it publicly) "C:\Bookmap\Logs\rithmic-engine-log-bm.txt", "C:\Bookmap\Logs\rithmic-engine-log.000" and corresponding bookmap -common log ("C:\Bookmap\Logs\log_*-common.txt"). First one contains some data that should allow to differentiate "Network or R API too slow" from "Bookmap code too slow" (keep in mind that file is wiped every time bookmap starts, so send it after closing Bookmap but before starting it again).
If it does help - please be mindful of confirmation bias. Ideally go back to "standard" version, see if things get worse, go back to experimental again, repeat a few times.
Thanks,
Svyatoslav