Insights from the HarperDB Product Team
When we say one of our core values here at HarperDB is transparency, we mean it. The goal of this page is to provide insight into the exciting new features and improvements we’re planning for the future of HarperDB. While the priority of these items may change, this effort at transparency is intended to help you understand where HarperDB is heading.
Feedback and feature requests are welcome: visit feedback.harperdb.io to get involved!
We’re the Border Collie of databases: intelligent, resourceful, and hardworking. We’re focused on ensuring that replacing established enterprise databases (or choosing HarperDB for new initiatives) never means leaving industry-standard functionality on the table:
- SQL Subselects: Long-awaited and much-requested, the workhorse of data science is coming to HarperDB’s SQL operation. In the meantime, HarperDB drivers support subselects, so your BI Tools can subquery in peace.
- Backup & Restore: Explicit operations to backup and restore a HarperDB instance, useful for creating clustered replicas, exporting data for analysis or testing, and disaster recovery.
- Data Sharding: Distribute your largest datasets across multiple HarperDB instances with the most efficient distribution algorithm available, affording you unlimited scale at without unnecessary cost.
- Authentication Options: Authentication strategies beyond JWT and Basic Auth. The leading candidate is OAuth support.
HarperDB is dedicated to delivering best-in-class performance. We’re going to have to rerun some benchmarks with our 3.0 release scheduled for late Q1/early Q2 2021:
- Query Engine: HarperDB will be reconfiguring our underlying key-value store to standardize the way we create and manage indexes across all data types, which will result in significant performance improvements for all operations.
- HTTP/S API: We’re replacing Express.js with Fastify as our API layer, resulting in significant performance improvement on every single API interaction.
- Internal Processes: Internal communication is being streamlined, speeding up both internal processes and external operations.
Our HTTP/S API is a great example of extending HarperDB beyond data persistence to achieve that goal. While our core operations are a great first step, we’re going to let our users extend it themselves:
- Custom Functions: Create your own serverless functions using Node.js. This will enable developers to create their own API endpoints with custom logic and direct access to HarperDB functions.
- Triggers: Execute one or more cascading operations when something else happens to a specific schema, table, or record.
- GraphQL: Expose GraphQL as a query option within HarperDB so you can reduce the number of calls required to get exactly the data you need in one shot.
We’re planning substantial improvements and additions to our flexible data-plane architecture:
- Distributed Queries: Provide users with the ability to execute query operations (both SQL and NoSQL) across a cluster via a single HarperDB Instance.
- Improved Pub/Sub Performance and Durability: We’re replacing our existing clustering engine with the fastest greyhound we could find: NATS.io’s new Jetstream platform, delivering durable subscriptions and a 100x throughput boost.
- Offline Studio: We’re going to upgrade the offline, instance-hosted version of HarperDB Studio to match the feature set of the cloud-hosted version available at studio.harperdb.io. We’ll also allow you to disable it entirely for maximum security.
- Clustering Visualization: Our current clustering configuration is done on an instance-by-instance basis. We’re going to add a single, comprehensive interface for managing and monitoring data flow across your network.
- CSV Export: We’re going to let you export and download the results of a specific query (or entire tables) in CSV format directly from the Studio, rather than requiring that you use our current export operation and fetch the file from the filesystem.
- HarperDB Cloud Goes International: Fire up the yacht, we’re going to Monaco 🚤! Well, maybe not, but we do intend to provide users with the ability to deploy hosted HarperDB Cloud both on AWS instances outside of the US, as well as on other cloud providers.
- Kubernetes Deployments: Templates and guidance for automated Kubernetes deployments.
- Optimized Docker Container: Performance optimizations of our existing HarperDB Docker container.
- One-Click DigitalOcean Deployments with Kubernetes: Provide DigitalOcean users the ability to create a one-click distributed database solution using Kubernetes.