EventQL v0.4.0 released

2017-01-10 18:00:00 UTC by Paul Asmuth

EventQL 0.4.0 is out and is ready for deployment. This release contains a lot of new features and fixes since 0.3, and is a recommended upgrade for all 0.3 users.

The new year is here and so is the next release of EventQL! New features and fixes include:

• Added a native transport layer (a framed TCP-based protocol). See doc/internals/protocol.txt for the specification

• Added a c client driver library (libevqlclient)

• Added a new table config option that allows to set a finite partition size (called a "partition size hint" in the user facing documentation). A table with a finite partition size will initially have zero partitions but instead add (finite) partitions as it receives inserts for the respective chunks of the keyspace (without splitting the new partition from a larger partition)

• Added the DROP TABLE statement

• Added a thread-local storage and retrieval mechanism for the current session. This also adds a new requirement that all operations must be executed from within a valid database thread.

• Added support for loading configuration files using the '-c' flag and for setting/overring individual configuration parameters using the '-C' flags in the EventQL client (evql) binary

• Added the following new configuration options: server.c2siotimeout, server.c2sidletimeout, server.s2siotimeout, server.s2sidletimeout, server.heartbeatinterval, cluster.allowedhosts, server.httpiotimeout, server.queryprogressratelimit, cluster.allowanonymous, cluster.allowdroptable, server.querymaxconcurrentshards, server.querymaxconcurrentshardsperhost, server.queryfailedshardpolicy, client.timeout, server.diskcapacity, server.loadlimit{soft,hard}, server.partitionsloadinglimit{soft,hard}, server.loadinfopublishinterval, server.noalloc, server.s2spoolmaxconnections, server.s2spoolmaxconnectionsperhost, server.s2spoollingertimeout, cluster.allowcreatedatabase, server.replicationthreadsmax

• Added the evqlslap benchmark and load testing tool

• Added caching for metadata lookups

• Added internal (server-to-server) connection pooling for native TCP connections

• Added a new "user defined partitioning" mode that can be enabled by setting userdefinedpartitioning=true on a table and allows to explicitly specify the partition key for each row

• Added a new monitor thread that publishes disk usage and other load information to the coordination service

• Added new frames to native protocol: ACK, INSERT, REPLINSERT, METAGETFILE{RESULT}, METACREATEFILE, METAPERFORMOP{RESULT}, METADISCOVER{RESULT}, METALISTPARTITIONS{RESULT}, METAFINDPARTITION{RESULT}

• Added the DESCRIBE PARTITIONS table statement to obtain information about the partitions of a table

• Added the USE statement to switch databases

• Added new sql functions: usleep, fnv32

• Added the CREATE DATABASE statement

• Added the CLUSTER SHOW SERVERS statement and the cluster-list command to obtain information about the servers of the current cluster.

• Added the table-import command to the evqlctl util

• Added a basic ruby driver (drivers/ruby)

• Changed the parallel GROUP BY operation to use the new native protocol and to schedule all remote partial group bys from a a single thread using asynchronous I/O (currently via poll()).

• Changed the evql client to use the new native protocol

• Changed the connection acceptor code to read the first byte of an incoming connection and switch protocols (currently HTTP and native) based on this first byte

• Changed the http server code to execute incoming requests from within a database thread (previously this was a generic thread pool)

• Changed the internal cluster auth to require all internal hosts to be whitelisted using the cluster.allowed_hosts option. In standalone mode, the whitelist is defaulted to 0.0.0.0/0

• Changed the HTTP transport to run fully within the database thread (i.e. removed the dependency on an external event loop) and added http timeouts

• Changed the SQL engine to use the new native transport

• Changed the zookeeper backend to explicitly reconnect on session timeouts and not rely on the built-in reconnect mechanism

• Changed the DESCRIBE TABLE statement to return a new "encoding" column that contains information about the column's encoding settings

• Changed the server allocator to assign new partitions to servers using a weighted random scheme, taking into a account the load facotr and the number of available bytes on disk for each server

• Changed all metadata operations to use the native protocol

• Removed the obsolete "sha1_tokens" field from the cluster/server config

• Removed the PCRE dependency

We hope you enjoy working with the new release and please let us know of any issues.

-- The EventQL Team

Tagged with eventql, release