Tag: Data Storage


return to all posts

The Future of Big Data is Small Data


The Looming Crisis of Big Data databases at Thing Scale. As a partially reformed software engineer, I’m acutely aware of the balance we all attempt to maintain between what we can achieve today and what we believe we’re going to be able to achieve tomorrow. In theory, our increasingly complex attempts to understand the universe … Continued

Manage High Resolution Data from Rotating Machines – An IIoT Challenge


One of the consistent themes that has surfaced when working with industrial clients is their reliance on high performance rotating machinery. Whether it’s steam turbines delivering electrical power or compressors in mid stream gas fields, these machines are the foundation for most heavy industries. If you consider the combustion engine as a rotating machine as well, we … Continued

Advancing Beyond Store and Forward


As the Internet of Things matures we continue to see design patterns come and go, with some evolving and some disappearing. The time has come- Store and Forward needs to go. This paradigm served as an initial step in shifting to the edge, but it should not survive. In this blog I’ll introduce the concepts of Store and Forward along with an explanation of its inferiority to other, … Continued

What is Polyglot Persistence and Why is it awful?


According to Wikipedia, “Polyglot persistence is the concept of using different data storage technologies to handle different data storage needs within a given software application.”  James Serra, in his blog writes,  “Polyglot Persistence is a fancy term to mean that when storing data, it is best to use multiple data storage technologies, chosen based upon the way data is being used by individual applications … Continued

The Risks of Multi-Tier Database Architecture


Over the last several years multi-tiered database architectures have become increasingly popular. These architectures typically consist of multiple different database products and can be made up of NoSQL products, SQL products, In-Memory products, Object stores etc.  A lot has been written about the benefits and the advantages.   They have become popular patterns due to a few different scenarios.  The … Continued

The Biggest Bottleneck in Application Development


 The other day I was thinking back to the days when I worked in custom application development. I asked myself a question: what was the biggest bottleneck that slowed your projects down? After having a wonderful conversation with myself I began asking my friends the same question. The overwhelming answer has been simple, the database. Systems don’t … Continued

Dynamic Schema: The HarperDB Way


How can you possibly have a dynamic schema? Isn’t the purpose of a schema to provide structure to your data? Well, yes, schemas exist to contain objects and define structure. But what happens if I know my data will continue to evolve, as it tends to do? Should you have to constantly rely on developers … Continued