Millistream Database Application (MDA) version 0.9.43 released

/in /by

The Millistream MDA project team is pleased to announce the release of version 0.9.43 of our open source application for updating SQL servers with data from the Millistream Data Feed.

  • New mod_odbc module for ODBC connectivity to sql servers. On Windows this more or less replaces the old OLE DB module (mod_mssql) since Microsoft seams to premier ODBC over OLE DB these days. On Linux it allows connectivity to MSSQL servers running either on Linux, macOS, Windows or Azure.While both MySQL, MariaDB and PostgreSQL have been fully tested and is supported we would recommend to use our native modules there instead for both performance and stability (ODBC is native for MSSQL but not for other sql servers).This should also allow connectivity to other ODBC compliant sql servers such as IBM D2 and Oracle but this has not been tested and most likely “–check-types” will not work since every sql server seams to define their data types differently even on a unified interface such as ODBC.
  • INSERT first tables will now only retry with an UPDATE if the error from the sql server indicated that it was a key violation.
  • Updates to the trades, pricehistory and adjustedpricehistory tables did not log an error if the INSERT failed and the following UPDATE returned with zero affected rows.
  • String values are now truncated to the length defined in the tabledef so that strings that for whatever reason do not comply with our defines will be accepted by the sql server instead of failing the entire query.
  • News items that have lingered for longer than 3600 seconds without receiving a last block of text and that are kicked from the queue will now be logged with their newsid and date.
  • Added connectionstring support for mod_mysql, the connection string is parsed by mda since the MySQL native driver does not support connection strings.
  • Added support for “sslcrl”, “sslcrlpath”, “connect_timeout” and “write_timeout” in the connectionstring for mod_mysql.
  • In order to maintain a uniform and simple interface we have decied to deprecate the “username”, “password”, “server”, “dbname”, “ssl_xxx” arguments from mda.config and instead only support “connectionstring”. However since these settings are in use by existing users MDA will continue to read and parse them as before, we have just removed them from the documentation and the default mda.config in a new install.
  • The status thread could sometimes be confused during shutdown and log that the sql server disconnected mda.
  • Table/column name comparisons are now case insensitive on “–check” and “–check-types”.
  • On “–upgrade-db” mda would enclose table/column literals with [], “ and ” on mod_mssql, mod_mysql and mod_postgresql respectively but since mda does not do this when performing INSERT/UPDATE and since some sql servers (such as PostgreSQL) will handle literals case sensitive if they are created while being enclosed we have now removed this behaviour from “–upgrade-db”.

As earlier announced, as of v0.9.33 the database schema is developed independently of the MDA and the latest schema including upgrade scripts is available on our ftp server under the documents section  (MDA_schemas_[version].zip). As of this writing the latest database schema is v94 and is available from ftp://ftp.millistream.com/documents/MDA_schema_94.zip

Do note however that the latest schemas are only needed if you need the new columns/tables, they are not mandatory to operate the MDA!

We consider MDA 0.9.43 to be the best version of MDA available and we recommend that users of older versions upgrade as soon as possible.

MDA 0.9.43 is available for download from the following locations:

Windows users upgrade by running the installer for the new version.

Linux users who have added our apt or yum repositories can upgrade by simply issuing:

  • Debian/Ubuntu: “apt update && apt upgrade“
  • Red Hat/CentOS/Amazon Linux AMI: “yum upgrade“

Information on how to add these repositories are available at ftp://millistream.com/binaries/linux/

And as usual, if you have any questions what so ever regarding this release or other things, don’t hesitate to contact us at tech@millistream.com