HarperDB 2.2.0 Penny Release (Current Release)

08/24/2020

 

Features/Updates

  • CORE-997 Updated the data format for CSV data loads being sync'd across a cluster to take up less resources
  • CORE-1018 Adds SQL functionality for  BETWEEN statements
  • CORE-1032 Updates permissions to allow regular users (i.e. non-super users) to call the  get_job  operation
  • CORE-1036 On create/drop table we auto create/drop the related transactions environments for the schema.table
  • CORE-1042 Built raw functions to write to a tables transaction log for insert/update/delete operations
  • CORE-1057 Implemented write transaction into lmdb create/update/delete functions
  • CORE-1048 Adds  SEARCH wildcard handling for role permissions standards
  • CORE-1059 Added config setting to disable transaction logging for an instance
  • CORE-1076 Adds permissions filter to describe operations
  • CORE-1043 Change clustering catchup to use the new transaction log
  • CORE-1052 Removed word "master" from source
  • CORE-1061 Added new operation called delete_transactions_before this will tail a transaction log for a specific schema / table
  • CORE-1040 On HarperDB startup make sure all tables have a transaction environment
  • CORE-1055 Added 2 new setting to change the server headersTimeout & keepAliveTimeout from the config file
  • CORE-1044 Created new operation read_transaction_log which will allow a user to get transactions for a table by timestamp, username, or hash_value
  • CORE-1043 Change clustering catchup to use the new transaction log
  • CORE-1089 Added new attribute to system_information for table/transaction log data size in bytes & transaction log record count
  • CORE-1101 Fix to store empty strings rather than considering them null & fix to be able to search on empty strings in SQL/NoSQL.
  • CORE-1054 Updates permissions object to remove delete attribute permission and update table attribute permission key to attribute_permissions
  • CORE-1092 Do not allow the __createdtime__ to be updated
  • CORE-1085 Updates create schema/table & drop schema/table/attribute operations permissions to require super user role and adds integration tests to validate
  • CORE-1071 Updates response messages and status codes from describe_schema and describe_table operations to provide standard language/status code when a schema item is not found
  • CORE-1049 Updates response message for SQL update op with no matching rows
  • CORE-1096 Added tracking of the origin in the transaction log. This origin object stores the node name, timestamp of the transaction from the originating node & the user.

Bug Fixes

  • CORE-1028 Fixes bug for simple SQL SELECT queries not returning aliases and incorrectly returning hash values when not requested in query
  • CORE-1037 Fixed an issue where numbers with leading zero i.e. 00123 are converted to numbers rather than being honored as strings.
  • CORE-1063 Updates permission error response shape to consolidate issues into individual objects per schema/table combo
  • CORE-1098 Fixed an issue where transaction environments were remaining in the global cache after being dropped.
  • CORE-1086 Fixed issue where responses from insert/update were incorrect with skipped records.
  • CORE-1079 Fixes SQL bugs around invalid schema/table and special characters in WHERE clause

 


Past Releases:

HarperDB 2.1.1, Penny Release

Highlights:

  • CORE-1007 Added the ability to perform SQL INSERT & UPDATE with function calls & expressions on values.
  • CORE-1023 Fixed minor bug in final SQL step incorrectly trying to translate ordinals to alias in ORDER BY statement.
  • CORE-1020 Fixed bug allowing 'null' and 'undefined' string values to be passed in as valid hash values.
  • CORE-1006 Added SQL functionality that enables JOIN statements across different schemas.
  • CORE-1005 Implemented JSONata library to handle our JSON document search functionality in SQL, creating the SEARCH_JSON function.
  • CORE-1009 Updated schema validation to allow all printable ASCII characters to be used in schema/table/attribute names, except, forward slashes and backticks. Same rules apply now for hash attribute values.
  • CORE-1003  Fixed handling of ORDER BY statements with function aliases.
  • CORE-1004  Fixed bug related to SELECT*  on JOIN queries with table columns with the same name.
  • CORE-996  Fixed an issue where the transact_to_cluster flag is lost for CSV URL loads, fixed an issue where new attributes created in CSV bulk load do not sync to the cluster.
  • CORE-994 Added new operation "system_information". This operation returns info & metrics for the OS, time, memory, cpu, disk, network.
  • CORE-993  Added new custom date functions for AlaSQL & UTC updates.
  • CORE-991  Changed jobs to spawn a new process which will run the intended job without impacting a main HarperDB process.
  • CORE-992   HTTPS enabled by default.
  • CORE-990  Updated describe_table to add the record count for the table for LMDB data storage.
  • CORE-989  Killed the socket cluster processes prior to HarperDB processes to eliminate a false uptime.
  • CORE-975 Updated time values set by SQL Date Functions to be in epoch format.
  • CORE-974  Added date functions to SQL SELECT column alias functionality.

HarperDB 1.3.1, Alby Release

2/26/2019

Features

  • Clustering connection direction appointment
  • Foundations for threading/multi processing
  • UUID autogen for hash attributes that were not provided 
  • Added cluster status operation

 

Bug Fixes and Enhancements 

  • More logging
  • Clustering communication enhancements
  • Clustering queue ordering by timestamps
  • Cluster re connection enhancements
  • Number of system core(s) detection
  • Node LTS (10.15) compatibility
  • Update/Alter users enhancements
  • General performance enhancements
  • Warning is logged if different versions of harperdb are connected via clustering
  • Fixed need to restart after user creation/alteration
  • Fixed SQL error that occurred on selecting from an empty table

HarperDB 1.3, Alby Release

11/2/2018

Features

  • Upgrade: Upgrade to newest version via command line.
  • SQL Support:  Added IS NULL for SQL parser.
  • Added attribute validation to search operations.

 

Fixes

  • Fixed SELECT calculations, i.e. SELECT 2+2.
  • Fixed select OR not returning expected results.
  • No longer allowing reserved words(https://harperdbhelp.zendesk.com/hc/en-us/articles/115004292473-HarperDB-Reserved-words)for schema and table names.
  • Corrected process interruptions from improper SQL statements.
  • Improved message handling between spawned processes that replace killed processes.
  • Enhanced error handling for updates to tables that do not exist.
  • Fixed error handling for NoSQL responses when get_attributes is provided with invalid attributes.
  • Fixed issue with new columns not being updated properly in update statements.
  • Now validating roles, tables and attributes when creating or updating roles.
  • Fixed an issue where in some cases“undefined” was being returned after dropping a role

HarperDB 1.2, Alby Release

7/10/2018

Features

  • Time to Live: Conserve the resources of your edge device by setting data on devices to live for a specific period of time.
  • Geo: HarperDB has implemented turf.js into its SQL parser to enable geo based analytics.
  • Jobs: CSV Data loads, Exports & Time to Live now all run as back ground jobs.
  • Exports: Perform queries that export into JSON or CSV and save to disk or S3.

 

Fixes

  • Fixed issue where CSV data loads incorrectly report number of records loaded.
  • Added validation to stop BETWEEN operations in SQL.
  • Updated logging to not include internal variables in the logs.
  • Cleaned up add_role response to not include internal variables.
  • Removed old and unused dependencies.
  • Build out further unit tests and integration tests.
  • Fixed https to handle certificates properly.
  • Improved stability of clustering & replication.
  • Corrected issue where Objects and Arrays were not casting properly in SQL SELECT response.
  • Fixed issue where Blob text was not being returned from SQL SELECTs.
  • Fixed error being returned when querying on table with no data, now correctly returns empty array.
  • Improved performance in SQL when searching on exact values.
  • Fixed error when ./harperdb stop is called.
  • Fixed logging issue causing instability in installer.
  • Fixed read_log operation to accept date time.
  • Added permissions checking to export_to_s3.
  • Added ability to run SQL on SELECT without a FROM.
  • Fixed issue where updating a user’s password was not encrypting properly.
  • Fixed user_guide.html to point to readme on git repo.
  • Created option to have HarperDB run as a foreground process.
  • Updated user_info to return the correct role for a user.
  • Fixed issue where HarperDB would not stop if the database root was deleted.
  • Corrected error message on insert if an invalid schema is provided.
  • Added permissions checks for user & role operations.

HarperDB 1.1, Alby Release

4/18/2018

Features

  • Users & Roles:
    • Limit/Assign access to all HarperDB operations
    • Limit/Assign access to schemas, tables & attributes
    • Limit/Assign access to specific SQL operations (INSERT, UPDATE, DELETE, SELECT)
  • Enhanced SQL parser
    • Added extensive ANSI SQL Support.
    • Added Array function, which allows for converting relational data into Object/Hierarchical data: http://www.harperdb.io/blog/sql-queries-complex-objects-array-function
    • Distinct_Array Function: allows for removing duplicates in the Array function.
    • Enhanced SQL Validation: Improved validation around structure of SQL, validating the schema, etc..
    • 10x performance improvement on SQL statements. 
  • Export Function: can now call a NoSQL/SQL search and have it export to CSV or JSON.
  • Added upgrade function to CLI
  • Added ability to perform bulk update from CSV
  • Created landing page for harperdb.  
  • Added CORS support to HarperDB : https://harperdbhelp.zendesk.com/hc/en-us/articles/115002214294-HTTP-access-control-CORS-

Fixes

  • Fixed memory leak in CSV bulk loads
  • Corrected error when attempting to perform a SQL DELETE
  • Added further validation to NoSQL UPDATE to validate schema & table exist
  • Fixed install issue occurring when part of the install path does not exist, the install would silently fail.
  • Fixed issues with replicated data when one of the replicas is down
  • Removed logging of initial user’s credentials during install
  • Can now use reserved words as aliases in SQL
  • Removed user(s) password in results when calling list_users
  • Corrected forwarding of operations to other nodes in a cluster
  • Corrected lag in schema meta-data passing to other nodes in a cluster
  • Drop table & schema now move the table & schema or table to the trash folder under the Database folder for later permanent deletion.
  • Bulk inserts no longer halt the entire operation if n records already exist, instead the return includes the hashes of records that have been skipped. 
  • Added ability to accept EULA from command line
  • Corrected search_by_value not searching on the correct attribute
  • Added ability to increase the timeout of a request by adding SERVER_TIMEOUT_MS to config/settings.js
  • Add error handling resulting from SQL calculations.
  • Standardized error responses as JSON.
  • Corrected internal process generation to not allow more processes than machine has cores.