Release Notes

HarperDB 2.3.1, Penny Release (Current Release)

1/29/2021

A bug in HarperDB 2.3.0 was identified related to manually calling the create_attribute operation. This bug caused secondary indexes to be overwritten by the most recently inserted or updated value for the index, thereby causing a search operation filtered with that index to only return the most recently inserted/updated row. Note, this issue does not affect attributes that are reflexively/automatically created. It only affects attributes created using create_attribute.

To resolve this issue in 2.3.0 or earlier, drop and recreate your table using reflexive attribute creation. In 2.3.1, drop and recreate your table and use either reflexive attribute creation or create_attribute.

To resolve this issue in 2.3.0 or earlier, drop and recreate your table using reflexive attribute creation. In 2.3.1, drop and recreate your table and use either reflexive attribute creation or create_attribute.

HarperDB 2.3.0, Penny Release

12/03/2020

Features/Updates

  • CORE-1191, CORE-1190, CORE-1125, CORE-1157, CORE-1126, CORE-1140, CORE-1134, CORE-1123, CORE-1124, CORE-1122
    Added JWT Authentication option (See documentation for more information)
  • CORE-1128, CORE-1143, CORE-1140, CORE-1129
    Added upsert operation
  • CORE-1187 Added get_configuration operation which allows admins to view their configuration settings.
  • CORE-1175 Added new internal LMDB function to copy an environment for use in future features.
  • CORE-1166 Updated packages to address security vulnerabilities.

Bug Fixes

  • CORE-1195 Modified drop_attribute to drop after data cleanse completes.
  • CORE-1149 Fix SQL bug regarding self joins and updates alasql to 0.6.5 release.
  • CORE-1168 Fix inconsistent invalid schema/table errors.
  • CORE-1162 Fix bug which caused delete_files_beforeto cause tables to grow in size due to an open cursor issue.

HarperDB 2.2.3, Penny Release

11/16/2020

  • CORE-1158 Performance improvements to core delete function and configuration of delete_files_before  to run in batches with a pause into between.

HarperDB 2.2.2, Penny Release

10/27/2020

  • CORE-1154 Allowed transaction logging to be disabled even if clustering is enabled.
  • CORE-1153 Fixed issue where delete_files_before was writing to transaction log.
  • CORE-1152 Fixed issue where no more than 4 HarperDB forks would be created.
  • CORE-1112 Adds handling for system timestamp attributes in permissions.
  • CORE-1131 Adds better handling for checking perms on operations with action value in JSON.
  • CORE-1113 Fixes validation bug checking for super user/cluster user permissions and other permissions.
  • CORE-1135 Adds validation for valid keys in role API operations.
  • CORE-1073 Adds new import_from_s3operation to API.

 

HarperDB 2.2.0, Penny Release

08/24/2020

Features/Updates

  • CORE-997 Updated the data format for CSV data loads being sync'd across a cluster to take up less resources
  • CORE-1018 Adds SQL functionality for  BETWEEN statements
  • CORE-1032 Updates permissions to allow regular users (i.e. non-super users) to call the  get_job  operation
  • CORE-1036 On create/drop table we auto create/drop the related transactions environments for the schema.table
  • CORE-1042 Built raw functions to write to a tables transaction log for insert/update/delete operations
  • CORE-1057 Implemented write transaction into lmdb create/update/delete functions
  • CORE-1048 Adds  SEARCH wildcard handling for role permissions standards
  • CORE-1059 Added config setting to disable transaction logging for an instance
  • CORE-1076 Adds permissions filter to describe operations
  • CORE-1043 Change clustering catchup to use the new transaction log
  • CORE-1052 Removed word "master" from source
  • CORE-1061 Added new operation called delete_transactions_before this will tail a transaction log for a specific schema / table
  • CORE-1040 On HarperDB startup make sure all tables have a transaction environment
  • CORE-1055 Added 2 new setting to change the server headersTimeout & keepAliveTimeout from the config file
  • CORE-1044 Created new operation read_transaction_log which will allow a user to get transactions for a table by timestamp, username, or hash_value
  • CORE-1043 Change clustering catchup to use the new transaction log
  • CORE-1089 Added new attribute to system_information for table/transaction log data size in bytes & transaction log record count
  • CORE-1101 Fix to store empty strings rather than considering them null & fix to be able to search on empty strings in SQL/NoSQL.
  • CORE-1054 Updates permissions object to remove delete attribute permission and update table attribute permission key to attribute_permissions
  • CORE-1092 Do not allow the __createdtime__ to be updated
  • CORE-1085 Updates create schema/table & drop schema/table/attribute operations permissions to require super user role and adds integration tests to validate
  • CORE-1071 Updates response messages and status codes from describe_schema and describe_table operations to provide standard language/status code when a schema item is not found
  • CORE-1049 Updates response message for SQL update op with no matching rows
  • CORE-1096 Added tracking of the origin in the transaction log. This origin object stores the node name, timestamp of the transaction from the originating node & the user.

Bug Fixes

  • CORE-1028 Fixes bug for simple SQL SELECT queries not returning aliases and incorrectly returning hash values when not requested in query
  • CORE-1037 Fixed an issue where numbers with leading zero i.e. 00123 are converted to numbers rather than being honored as strings.
  • CORE-1063 Updates permission error response shape to consolidate issues into individual objects per schema/table combo
  • CORE-1098 Fixed an issue where transaction environments were remaining in the global cache after being dropped.
  • CORE-1086 Fixed issue where responses from insert/update were incorrect with skipped records.
  • CORE-1079 Fixes SQL bugs around invalid schema/table and special characters in WHERE clause

HarperDB 2.1.1, Penny Release

05/22/2020

Highlights:

  • CORE-1007 Added the ability to perform SQL INSERT & UPDATE with function calls & expressions on values.
  • CORE-1023 Fixed minor bug in final SQL step incorrectly trying to translate ordinals to alias in ORDER BY statement.
  • CORE-1020 Fixed bug allowing 'null' and 'undefined' string values to be passed in as valid hash values.
  • CORE-1006 Added SQL functionality that enables JOIN statements across different schemas.
  • CORE-1005 Implemented JSONata library to handle our JSON document search functionality in SQL, creating the SEARCH_JSON function.
  • CORE-1009 Updated schema validation to allow all printable ASCII characters to be used in schema/table/attribute names, except, forward slashes and backticks. Same rules apply now for hash attribute values.
  • CORE-1003  Fixed handling of ORDER BY statements with function aliases.
  • CORE-1004  Fixed bug related to SELECT*  on JOIN queries with table columns with the same name.
  • CORE-996  Fixed an issue where the transact_to_cluster flag is lost for CSV URL loads, fixed an issue where new attributes created in CSV bulk load do not sync to the cluster.
  • CORE-994 Added new operation "system_information". This operation returns info & metrics for the OS, time, memory, cpu, disk, network.
  • CORE-993  Added new custom date functions for AlaSQL & UTC updates.
  • CORE-991  Changed jobs to spawn a new process which will run the intended job without impacting a main HarperDB process.
  • CORE-992   HTTPS enabled by default.
  • CORE-990  Updated describe_table to add the record count for the table for LMDB data storage.
  • CORE-989  Killed the socket cluster processes prior to HarperDB processes to eliminate a false uptime.
  • CORE-975 Updated time values set by SQL Date Functions to be in epoch format.
  • CORE-974  Added date functions to SQL SELECT column alias functionality.

HarperDB 1.3.1, Alby Release

2/26/2019

Features

  • Clustering connection direction appointment
  • Foundations for threading/multi processing
  • UUID autogen for hash attributes that were not provided 
  • Added cluster status operation

 

Bug Fixes and Enhancements 

  • More logging
  • Clustering communication enhancements
  • Clustering queue ordering by timestamps
  • Cluster re connection enhancements
  • Number of system core(s) detection
  • Node LTS (10.15) compatibility
  • Update/Alter users enhancements
  • General performance enhancements
  • Warning is logged if different versions of harperdb are connected via clustering
  • Fixed need to restart after user creation/alteration
  • Fixed SQL error that occurred on selecting from an empty table

HarperDB 1.3.0, Alby Release

11/2/2018

Features

  • Upgrade: Upgrade to newest version via command line.
  • SQL Support:  Added IS NULL for SQL parser.
  • Added attribute validation to search operations.

 

Fixes

  • Fixed SELECT calculations, i.e. SELECT 2+2.
  • Fixed select OR not returning expected results.
  • No longer allowing reserved words(https://harperdbhelp.zendesk.com/hc/en-us/articles/115004292473-HarperDB-Reserved-words)for schema and table names.
  • Corrected process interruptions from improper SQL statements.
  • Improved message handling between spawned processes that replace killed processes.
  • Enhanced error handling for updates to tables that do not exist.
  • Fixed error handling for NoSQL responses when get_attributes is provided with invalid attributes.
  • Fixed issue with new columns not being updated properly in update statements.
  • Now validating roles, tables and attributes when creating or updating roles.
  • Fixed an issue where in some cases“undefined” was being returned after dropping a role

HarperDB 1.2.0, Alby Release

7/10/2018

Features

  • Time to Live: Conserve the resources of your edge device by setting data on devices to live for a specific period of time.
  • Geo: HarperDB has implemented turf.js into its SQL parser to enable geo based analytics.
  • Jobs: CSV Data loads, Exports & Time to Live now all run as back ground jobs.
  • Exports: Perform queries that export into JSON or CSV and save to disk or S3.

 

Fixes

  • Fixed issue where CSV data loads incorrectly report number of records loaded.
  • Added validation to stop BETWEEN operations in SQL.
  • Updated logging to not include internal variables in the logs.
  • Cleaned up add_role response to not include internal variables.
  • Removed old and unused dependencies.
  • Build out further unit tests and integration tests.
  • Fixed https to handle certificates properly.
  • Improved stability of clustering & replication.
  • Corrected issue where Objects and Arrays were not casting properly in SQL SELECT response.
  • Fixed issue where Blob text was not being returned from SQL SELECTs.
  • Fixed error being returned when querying on table with no data, now correctly returns empty array.
  • Improved performance in SQL when searching on exact values.
  • Fixed error when ./harperdb stop is called.
  • Fixed logging issue causing instability in installer.
  • Fixed read_log operation to accept date time.
  • Added permissions checking to export_to_s3.
  • Added ability to run SQL on SELECT without a FROM.
  • Fixed issue where updating a user’s password was not encrypting properly.
  • Fixed user_guide.html to point to readme on git repo.
  • Created option to have HarperDB run as a foreground process.
  • Updated user_info to return the correct role for a user.
  • Fixed issue where HarperDB would not stop if the database root was deleted.
  • Corrected error message on insert if an invalid schema is provided.
  • Added permissions checks for user & role operations.

HarperDB 1.1.0, Alby Release

4/18/2018

Features

  • Users & Roles:
    • Limit/Assign access to all HarperDB operations
    • Limit/Assign access to schemas, tables & attributes
    • Limit/Assign access to specific SQL operations (INSERT, UPDATE, DELETE, SELECT)
  • Enhanced SQL parser
    • Added extensive ANSI SQL Support.
    • Added Array function, which allows for converting relational data into Object/Hierarchical data: http://www.harperdb.io/blog/sql-queries-complex-objects-array-function
    • Distinct_Array Function: allows for removing duplicates in the Array function.
    • Enhanced SQL Validation: Improved validation around structure of SQL, validating the schema, etc..
    • 10x performance improvement on SQL statements. 
  • Export Function: can now call a NoSQL/SQL search and have it export to CSV or JSON.
  • Added upgrade function to CLI
  • Added ability to perform bulk update from CSV
  • Created landing page for harperdb.  
  • Added CORS support to HarperDB : https://harperdbhelp.zendesk.com/hc/en-us/articles/115002214294-HTTP-access-control-CORS-

Fixes

  • Fixed memory leak in CSV bulk loads
  • Corrected error when attempting to perform a SQL DELETE
  • Added further validation to NoSQL UPDATE to validate schema & table exist
  • Fixed install issue occurring when part of the install path does not exist, the install would silently fail.
  • Fixed issues with replicated data when one of the replicas is down
  • Removed logging of initial user’s credentials during install
  • Can now use reserved words as aliases in SQL
  • Removed user(s) password in results when calling list_users
  • Corrected forwarding of operations to other nodes in a cluster
  • Corrected lag in schema meta-data passing to other nodes in a cluster
  • Drop table & schema now move the table & schema or table to the trash folder under the Database folder for later permanent deletion.
  • Bulk inserts no longer halt the entire operation if n records already exist, instead the return includes the hashes of records that have been skipped. 
  • Added ability to accept EULA from command line
  • Corrected search_by_value not searching on the correct attribute
  • Added ability to increase the timeout of a request by adding SERVER_TIMEOUT_MS to config/settings.js
  • Add error handling resulting from SQL calculations.
  • Standardized error responses as JSON.
  • Corrected internal process generation to not allow more processes than machine has cores.