CFDs are complex instruments and come with a high risk of losing money rapidly due to leverage. 64 % of retail investor accounts lose money when trading CFDs with this provider. You should consider whether you understand how CFDs work and whether you can afford to take the high risk of losing your money.

(SOLVED) DWX ZeroMQ Connector v2.0.1 - Help Please

Hello, I cannot seem to get the ZMQ EA to perform basic tasks reliably or at all.

_zmq.DWX_MTX_GET_ALL_OPEN_TRADES()
responds with
“Resource timeout… please try again.”

If I subscribe:
_zmq.DWX_MTX_SUBSCRIBE_MARKETDATA(_symbol=‘EURUSD’)

it will tell me the Kernel subscribed but _zmq._Market_Data_DB.keys()
will respond with dict_keys([]), meaning there is no data for EURUSD.

I’ll get “KeyError: 'EURUSD” when I _zmq._Market_Data_DB[‘EURUSD’] eventhough it saids it is subscribed.

I am wondering whether I have this EA installed correctly or it is bugged.
When I go into the dependencies, I see a ton of red flags (please see attached picture). I followed the instructions on the site to a T, dropping the Zml and Mql folders into Include as-is, putting libsodium.dll and libzmq.dll into the Libaries folder, and also installed pyzmq via pip install pyzmq in command line. The EA is in the experts folder and I did a %run for the .py file before running the commands in the Ipython shell.

I am at a loss now as to how to make this work. I need this ZMQ EA to come to a place where I can trust it to work so I can continue developing my algorithm. The idea of binding Python to MT4 is extremely promising.

Thank you so much for your help and for your time.

2 Likes

I also want to add that even when MT4 is not running, if I run _zmq = DWX_ZeroMQ_Connector(), it will still return
[INIT] Ready to send commands to METATRADER (PUSH): 32768
[INIT] Listening for responses from METATRADER (PULL): 32769
[INIT] Listening for market data from METATRADER (SUB): 32770

so my best guess is that it is actually not successfully connecting to the EA, so my question is how we verify there is actually a connection to the EA and ensure that connection is stable?

Thank you

2 Likes

Hi @doublingpenny , welcome to the community!
I think @integracore2 should be able to help you.

3 Likes

Hi @doublingpenny,

Welcome to the Darwinex Community :slightly_smiling_face:

  1. Could you confirm that you are indeed using the latest code via the following link:
  1. The outputs on your Python console as you’ve shared above, are simply print() statements after the script connects to the PUSH, PULL and SUB ports passed via the init() initialization method.

I encourage you to read the code to gain a functional understanding of all the moving parts in both the Python and MQL4 scripts.

  1. Could you please open an issue on GitHub via the link above, so it can be addressed and appropriately referenced there? Thanks!

  2. Simply put, Resource Timeout in this situation implies that the client script (Python) requested something via the ports, but didn’t get anything and timed out.

i.e. the EA isn’t successfully bound to those ports at the other end and didn’t receive nor process the client’s requests.

  1. The errors you see in the EA dependencies are likely the issue, best thing here is to post a detailed log of all the errors you see in that tab, over at GitHub via an issue ticket.

If you are an existing Darwinex client, my colleague @miguelrDarwinex may also be able to assist you with resolving the above.

Hope this helps! :+1:

2 Likes

@CavaliereVerde thank you for the reference

@integracore2 thank you for joining this thread. I should be using the latest code from that github as I downloaded it 3 days ago (unless there was a change I’m unaware of).

I am entirely new to Python and MQL4 the code in the script is a bit beyond my understanding. However, I did follow the playlist at the Darwinex youtube to a T and the intructions on the Github precisely. Is there more that needs to be done to the code in order to get the ZMQ EA to properly bind?

As for the issues with the dependencies, like I said, I installed pyzmq and placed all the Dingmaotu dependency files in the MQL folder. Is there more that I need to do that is not mentioned on the instructions page that maybe perhaps they are just a given assumption for someone who is more experienced?

I see things like ADVAPI32.dll and WS2_32 or MSVCP140.dll that were not included in those files. Am I supposed to retrieve these?

I submitted an issue on Github per your request.

Thank you so much. Really excited to get this up and running.

Hi @doublingpenny,

Thank you for sharing more regarding your situation.

Please note that we cannot provide support for Python or MQL as programming languages themselves.

Therefore, if you are new to Python and MQL, incorporating the project into your specific algorithmic trading environment will require some additional work on your part (i.e. enough Python experience to integrate the Bridge into your environment -> it is assumed that users of the Bridge are self-sufficient in Python).

This is noted on the GitHub README here as well:

Quote from the page: IMPORTANT NOTE: Any code provided and/or referenced in this repository is NOT meant to be used “as-is”. Users must treat all code as educational content that requires modification / incorporation into existing works, as per their individual requirements.

Issues with dependencies hence require some work on the part of users too.

For example,

On the README above, links to the dependency projects (mql-zmq, mql-lib, libsodium) are provided.

As these dependencies are beyond our control, any requirements/updates to the same require users to adapt to those changes by following the dependency projects.

In the specific case of mql-zmq for instance, the dependency project’s README states the following:
The DLLs require that you have the latest Visual C++ runtime (2015) .

Link: https://github.com/dingmaotu/mql-zmq

In your screenshot above, there are dependency errors under VCRUNTIME140.dll, implying an issue with Visual C++ runtime libraries (a dependency for the mql-zmq dependency)…

We have drafted as detailed a set of steps as possible in our project README, but cannot cover all the dependencies as they are independent projects on their own that programmers need to account for / follow / keep up to speed with when considering using the DWX ZeroMQ Connector.

As an example of what programmers with experience would do in the situation above (the VCRUNTIME140.dll errors):

  1. Dependencies are showing VCRUNTIME140.dll exception handler errors… what on earth are those?

Step 1 -> Google “vcruntime140.dll”

  1. This will lead to results where other programmers have asked a similar question via a similar search query.

  2. Navigating the various resources in the search results will lead you to the same answer that the authors of the mql-zmq project have stated on their particular README:

VC++ Runtime Libraries are required to be downloaded and installed.

This response is meant to encourage you to bypass the hurdle presented by being new to Python and MQL development, so please take it as friendly advice from someone who has been where you are (and still is from time to time!) :slight_smile:

Another piece of advice:

When requesting assistance, replace “it is not working” with “I am getting some errors under Dependencies, specifically to do with calls to VCRUNTIME140.dll. Here’s a log of the errors.”

Not only is this a very specific way to explain the situation to people you’re requesting help from, it also helps you as a programmer to understand your errors and explore solutions better :+1:

Hope the above helps, and you’ll certainly be getting a response on GitHub too, no worries.

We’ll need to reproduce the errors you’re getting by simulating an environment without VC++ Runtime libraries installed. So kindly wait until next week for this.

Hope the above proves helpful! :slight_smile:

And please do keep the questions coming - it is THE best way to learn anything, particularly in the Darwinex Community… you will not find the level of community support you have here at Darwinex, anywhere else :muscle:

1 Like

@integracore2 thank you for you kind response and advice. I will try to phrase my questions and roadblocks in a better way that will elicit a better response.

While i may be new to MQL and Python, I am agressively learning and trying to make sense of the code and hope to arrive at a place of self-sufficiency rather than depence on others. I’ll look into the VC++ Runtime Libraries and then report my results back here!

Thank you

1 Like

after reviewing the Github instructions, I installed the VC++ Runtime 2010 libraries because my being a Wine user, and I used the libraries from the VC2010 folder.

I was able to successfully get a JSON response from issuing the _zmq.DWX_MTX_GET_ALL_OPEN_TRADES() and _zmq.DWX_MTX_CLOSE_ALL_TRADES() commands.

However, any adjustment to settings in the EA settings resulted in more Resource Timeout or no response at all.

I was only able to receive responses again after rebooting my computer but any adjustments to the EA setting resulted in the same. Simply restarting the MT4 did not correct the issue.

So it appears now the EA will respond to commands after a fresh restart if no adjustments are made to the EA after initialization.

1 Like

Hi @doublingpenny,

Great to hear of your progress!

Bear in mind that the project has not been tested on emulated Windows environments, e.g. via Wine, VMWare etc. This is mentioned in the GitHub readme as well.

This is something on the roadmap for evaluation, but at the present time there isn’t any data on performance in emulated environments.

Hope this helps :+1:

2 Likes

Hi @integracore2,

I read into your advice per your last post and decided to run the ZMQ EA through Windows to eliminate any issues of compatibility.

The only changes in the setup now is I used the dependencies from the MT4 folder instead of the VC2010, and I installed Visual C++ 2015 instead of 2010(since I am no longer using Wine).

So I’d like to report back my results:

  1. just as before, the the same red flags are appearing the in dependencies when the EA is loaded to a chart. I am wondering what else needs to be installed that has been overlooked.
  2. the EA will disconnect with Jupyter Notebook if any adjustments are made to the inputs. Now the connection can be re-established if the MT4 terminal is restarted instead of the having to reboot the machine.
  3. When subscribed to a Symbol, there will be occasional lapses in tick data when invoking _zmq._Market_Data_DB[‘EURUSD’] as I am watching the market movement.

Through this I made the observation that the Market Watch time would be 7/8/2019 00:17, whereas the data coming into the Jupyter Notebook cell would be 21:17 7/7/2019, while the MT4 terminal journal would be 14:17 7/7/2019.

Is there a way to reconcile the time difference between these platforms? In a best-practice case, should GMT be the master time reference?

So my only remaining concerns with the EA is reliability of the connection between MT4 and Python, timestamp consistency, and lapses in tick data.

2 Likes

Hi @doublingpenny,

Thank you for sharing more detail!

Here is a session I just ran without issues, so we need to dig deeper in your situation:

_zmq = DWX_ZeroMQ_Connector()
[INIT] Ready to send commands to METATRADER (PUSH): 32768
[INIT] Listening for responses from METATRADER (PULL): 32769
[INIT] Listening for market data from METATRADER (SUB): 32770

_zmq._DWX_MTX_GET_ALL_OPEN_TRADES_()
{'_action': 'OPEN_TRADES', '_trades': {}}

_zmq._DWX_MTX_NEW_TRADE_()
{'_action': 'EXECUTION', '_response': '131', 'response_value': 'invalid trade volume'}

_order = _zmq._generate_default_order_dict()

_order
Out[6]: 
{'_action': 'OPEN',
 '_type': 0,
 '_symbol': 'EURUSD',
 '_price': 0.0,
 '_SL': 500,
 '_TP': 500,
 '_comment': 'DWX_Python_to_MT',
 '_lots': 0.01,
 '_magic': 123456,
 '_ticket': 0}

_order['_lots'] = 0.1

_zmq._DWX_MTX_NEW_TRADE_(_order=_order)
{'_action': 'EXECUTION', '_magic': 123456, '_ticket': 91944657, '_open_time': '2019.07.09 18:23:52', '_open_price': 1.12073, '_sl': 500.0, '_tp': 500.0}

_zmq._DWX_MTX_GET_ALL_OPEN_TRADES_()
{'_action': 'OPEN_TRADES', '_trades': {91944657: {'_magic': 123456, '_symbol': 'EURUSD', '_lots': 0.1, '_type': 0, '_open_price': 1.12073, '_open_time': '2019.07.09 18:23:52', '_SL': 1.11573, '_TP': 1.12573, '_pnl': 0.0, '_comment': 'DWX_Python_to_MT'}}}

zmq.__version__
Out[10]: '18.0.0'

_zmq._DWX_MTX_CLOSE_ALL_TRADES_()
{'_action': 'CLOSE_ALL', '_responses': {91944657: {'_symbol': 'EURUSD', '_magic': 123456, '_close_price': 1.12099, '_close_lots': 0.1, '_response': 'CLOSE_MARKET'}}, '_response_value': 'SUCCESS'}

Operating System:
Windows 10 Professional

Python module “zmq” version:
18.0.0

Screenshot of EA dependencies:
image


Let’s start with OS and module versions, could you please reply with your OS and zmq version? -> in a Python shell, run the following code:

import zmq
print(zmq.__version__)

ZeroMQ handles connections asyncronously - the EA acts as a server and binds to the specified ports.

I recommend you run your tests via a Python or IPython shell first to query the issue.

There is also the possibility of a cell execution in Jupyter timing out between calls to and from MetaTrader.

If you plan on running an automated trading algorithm, I would not run it in a Jupyter notebook and use a cron or batch process to run a local Python script :+1:

p.s. Jupyter was not used during development nor testing for this project for this reason - there isn’t a particular necessity for involving an IPython Notebook to run trading infrastructure.

Hope this helps!

Also, please note that these are not error flags - they are different function calls, coded in red and green by type.

The green icons are calls to functions from MQL programs, and the red icons are calls to functions within DLLs.

If there are errors during importing a function call from a dependency DLL, they are displayed in your Experts tab in MetaTrader with a message that goes something like "unresolved import function call…"

and all this time I thought the dependencies were incomplete because of the red icons!

ok, so I subscribed to market data through Ipython shell and it appears it is coming in consistently now. Thank you for your help!

For posterity, OS: Windows 10 Pro, ZMQ version: 18.0.0

1 Like

Fantastic stuff! :slightly_smiling_face: :clap:

This was a great debugging exercise worthy of a video tutorial on its own, so thank you for consistently providing feedback like you did.

Persistence is the mark of a great programmer, thank you for demonstrating exactly that…

Welcome to Darwinex :muscle:

3 Likes

Thank you for the kind welcome. This was an odyssey in refining a new skillset and I’m glad you were along for th journey :+1:t3:

3 Likes