Dictionary of cloud stuff. (Work in progress…)
See also: Cloud Native (Bookmarks).
(Skip preliminaries — to dictionary.)
Format
- I am considering converting this into an app, adding features; this post format is a quick-and-dirty attempt at releasing “early” — this is a small sample, I’ve already well over 400 entries.
- Not showing timestamps per term, yet.
- Still thinking about RDFa, microformats… (Exposes how Q&D I did data modeling for this. And how hard taxonomies are. ;o)
Classifications
- My attempt at a more (semi-)structured knowledge base.
- Also, to normalize (standardize?) and de-hype terminology, characterizations.
- … turned out to be an impossible task.
- Recommended resources:
- Detailed classifications of many databases done by dbdb — are linked as canonical.
- CNCF SIG Storage whitepaper gives a rudimentary taxonomy.
Dictionary
(Reminder: use Ctrl+F, or “/” on FF. ;o)
Key topics: NoSQL, Kubernetes, continuous, microservices, storage, Hadoop…
-
3Box
Decentralized secure storage. Uses IPFS, OrbitDB. Topics Data model -
Accumulo (Apache Accumulo)
Distributed key-value store, based on BigTable, HDFS, ZooKeeper. Topics Data model Implementation Java Created 2008 -
Aerospike
SSD-optimized distributed key-value database, replicated, secondary indexes. Topics Data model Query custom API Implementation C Stored procedures Lua Created 2009 - Jepsen: Aerospike 3.99.0.3 / 2018: detailed analysis.
- Jepsen: Aerospike / 2015: rigorous analysis, especially of what ACID means, critique of version 3.5.4.
-
Akutan (formerly Beam)
Distributed knowledge graph store — aka RDF or triple store. Topics Data model Implementation Go Created 2019 -
Alluxio (formerly Tachyon)
Virtual distributed file system, used as data orchestration layer, provides multiple data access interfaces, unified mounting name-space, and hierarchical cache, between cloud storage and, eg, Kubernetes workloads, Presto, Spark, Hive, Kubeflow. Multi-cloud. Topics Implementation Java Scale petabyte Created 2013 - CSI implementation to provide POSIX access to Alluxio in containerized environments such as Kubernetes
- Using Alluxio to Optimize and Improve Performance of Kubernetes-Based Deep Learning in the Cloud / 2020-05: whitepaper (PDF), detailed usecase, performance analysis.
- Improving Presto Latencies with Alluxio Data Caching / Rohit Jain, et al, 2020-06: architecture, performance benchmarks.
-
Ambari (Apache Ambari)
Hadoop cluster provisioning, monitoring, management. Topics -
Ansible
Automation, provisioning, and orchestration tool. Agentless — uses SSH. Everything in YAML. 1,300+ integrations. Topics Implementation Python - Module Index (Categories)
- Ansible Galaxy: community contributed roles (tasks) and collections of.
-
Ant (Apache Ant)
Make-like build tool. Topics Implementation Java Created 2000 -
AnyRPC
Multi-protocol remote procedure call C++ library; supports JSON-RPC, XML-RPC, MessagePack RPC. Topics Implementation C++ -
API gateway
Topics -
App Mesh (AWS App Mesh)
Service mesh on AWS. Topics -
ArangoDB (formerly AvocadoDB)
Multi-model distributed database, JSON, in-memory with persistent backend. Joins, either multi-collections transactions for ACID, or single-document transactions for performance. Topics Data model Query AQL, JSON oriented Implementation C++, JavaScript Stored procedures JavaScript (as microservices) Created 2011 - ArangoDB vs Neo4j: detailed comparison; NB: APIs, QL, and clustering.
- ArangoDB Foxx repositories
-
Argo (ArgoCD)
Kubernetes native workflows, events, CI and CD. Topics Implementation Go, TypeScript -
Artifactory
Artifact repository, container image registry. Topics Implementation Ruby -
Ascoltatori
Pub-sub library, multiple brokers/protocols: Redis, MongoDB, MQTT, AMQP, ZeroMQ, *QlobberFSQ, Kafka. Unmaintained? Topics Implementation JavaScript -
Aurora (Amazon Aurora)
AWS managed (by RDS) RDBMS compatible with MySQL and PostgreSQL. Distributed, fault-tolerant, self-healing, low-latency read replicas, point-in-time recovery, continuous backup to S3, and replication across three Availability Zones. Topics Created 2014 Alternatives -
Auto DevOps
GitLab CI template (automatic pipelines) for: build, test, code quality, SAST, dependency scanning, license management, container scanning, review apps, DAST, deployment, and browser performance testing.
Topics - Category Direction – Auto Devops: GL’s plans for Auto DevOps; useful overview.
- GitLab Auto DevOps demo (video) / 2018-08
- Using GitLab Auto DevOps with Kubernetes Through Rancher’s Authorized Cluster Endpoint / 2019-04: example usage.
-
AWS (Amazon Web Services)
The public cloud provider. Created 2006 -
Bazel
Build tool. Topics - Bazel Performance in a CI Environment / Filip Nikolovski, 2020-03: ephemeral containers (GitLab) vs caching, and GIT_CLEAN_FLAGS.
-
BerkeleyDB (BDB)
Embedded database, ACID. Topics Data model Query SQL, PL/SQL Created 1994 -
BigQuery
Query engine, serverless data warehouse, append-only tables, for interactive analysis of datasets over Google Cloud Storage. REST API, CLI tools. Integrated with Google Apps Script (ie Google Docs/Drive). Topics Data model Query SQL, data encoded as CSV or JSON Stored procedures JavaScript Created 2010 - Getting started with BigQuery – Colaboratory: Jupyter notebook on Drive.
-
Brigade
Event-driven scripting for Kubernetes. Topics - Kashti: web UI for Brigade pipelines.
-
Brooklyn (Apache Brooklyn)
Framework for modeling, deploying and managing distributed applications defined using declarative YAML blueprints. Topics Implementation Java Created 2012 -
Buildpacks (Cloud Native Buildpacks, CNB)
Topics Created 2011 - Intro to Cloud Native Buildpacks (video) / Terence Lee & Emily Casey, 2019-11
- Cloud Foundry Buildpacks or Dockerfiles / 2018-03
-
Buildr (Apache Buildr)
Build tool for Java-based applications. Topics Implementation Ruby -
Calcite (Apache Calcite)
SQL parser, relational algebra API, query planning engine.
Topics -
Camel (Apache Camel)
Message routing framework for Enterprise Integration Patterns. Cf suggested intros. Topics - Components reference: 350+ components.
-
Cassandra (Apache Cassandra)
Distributed database, (wide-) columnar, replicated (across data centers), tunable consistency. Hadoop integration. LSM-Trees. Topics Data model Query CQL Implementation Java Scale petabyte Created 2008 - 2012 in review: Performance / Jonathan Ellis: compares architecture/features with MongoDB, Riak, HBase.
- Cassandra and Solid State Drives (slides) / Rick Branson, 2012
-
Cayenne (Apache Cayenne)
ORM framework. Topics -
ChartMuseum
Helm chart repository server. Topics -
ClickHouse
Columnar database for OLAP. No transactions. Topics Data model Query SQL-like Implementation C++ Scale petabyte Created 2009 -
CockroachDB (CRDB)
Distributed database, OLTP, PostgreSQL compatible. Topics Data model Query SQL Created 2015 Alternatives - How to Choose Between PostgreSQL and CockroachDB / Steve Croce, 2019-08-15
-
continuous (CI/CD: continuous integration/delivery/deployment)
Topics - CD tools drawbacks / discussion on Reddit /r/devops, 2019-04: *Spinnaker, *GitLab, *Helm…
- CNCF SIG Application Delivery Charter: scope and topics of the lifecycle of cloud-native applications.
- Model of Cloud-Native App Delivery | Google Slides
-
Couchbase (Couchbase Server, formerly Membase)
Distributed (shared-nothing) database, JSON document oriented, in-memory, swapped to disk, append-only. Topics Data model Indexes hash, B+ tree, full text Query N1QL (SQL extended for JSON) Implementation Erlang Created 2009 -
CouchDB (Apache CouchDB, Cluster Of Unreliable Commodity Hardware)
Document database. Topics Data model Query REST API Implementation Erlang Created 2005 -
CrateDB
Distributed (shared-nothing) SQL database, document oriented. Components from Presto, Lucene, Elasticsearch and *Netty. Topics Data model Query SQL Implementation Java Created 2014 -
Crux
Document database with bitemporal graph queries (indexing of schemaless documents), layer over LMDB, RocksDB, or Kafka. Topics Query EDM, Datalog Implementation Clojure Created 2019 - A Bitemporal tale – History. Of histories. / 2019-07-16: code example (Clojure interactive notebook?).
-
Cypher
Declarative graph query language. Topics Created 2011 -
Delta Lake
Storage layer over data lake (S3, Hadoop), provides ACID, data warehouse features, using Spark. Topics - Presto and Athena to Delta Lake integration / 2020-07-06
-
Dgraph
Distributed graph database. Uses RocksDB. Topics Data model Query GraphQL variant Implementation Go Created 2015 -
DocumentDB (Amazon DocumentDB)
Managed MongoDB-compatible database on AWS. Topics Data model Alternatives -
Dqlite
Distributed SQLite, embedded (C library), persistent RDBMS, Raft consensus. Topics Query SQL Implementation C Created 2017 Alternatives -
Drill (Apache Drill)
Distributed SQL query engine. Schema-less. Topics Data model Query SQL Implementation Java Created 2012 -
Druid (Apache Druid)
Distributed real-time analytics, uses ZooKeeper, HDFS and a relational database for metadata. Topics Data model Query SQL, REST/HTTP Created 2011 -
DuckDB
Embedded database for OLAP. ACID. Topics Data model Query SQL Implementation C++ Created 2018 Alternatives -
DynamoDB (Amazon DynamoDB)
AWS key-value and document store, auto-scaling, priced on throughput instead of storage, integration with Hadoop. Topics Data model Created 2012 - If DynamoDB sucks at sorting/querying… which database should we use? / 2020-03: Reddit discussion of DynamoDB features, strenghts/weaknesses, (general NoSQL) design considerations: use it for big data, not relational queries.
- DynamoDB’s new autoscaling in action for https://github.com/tj/gh-polls, low traffic either way but at least I don’t have to over-provision! / 2017-07, tweet by TJ Holowaychuk
EdgeDB
Topics | ||
---|---|---|
Query | EdgeQL, GraphQL |
|
Created | 2018 |
- EdgeDB: A New Beginning / Yury Selivanov, Elvis Pranskevichus, 2018-04: motivation behind EdgeDB.
Elasticsearch
Topics | |
---|---|
Data model | |
Query | SQL-like, HTTP |
Implementation | Java |
Created | 2010 |
Elliptics
Topics | |
---|---|
Data model | |
Implementation | C++, Python, Go |
Created | 2008 |
FoundationDB
Topics | |
---|---|
Data model | |
Implementation | C++ |
Created | 2009 |
Geode (Apache Geode)
Topics | |
---|---|
Data model | |
Query | OQL |
Implementation | Java |
Created | 2002 |
GitOps
Topics |
---|
GlusterFS (Red Hat Gluster Storage)
Topics |
---|
Hadoop (Apache Hadoop)
Topics | |
---|---|
Implementation | Java |
Hazelcast (IMDG, In-Memory Data Grid)
Topics | |
---|---|
Data model | |
Implementation | Java |
Created | 2008 |
Alternatives |
HBase (Apache HBase)
Topics | |
---|---|
Data model | |
Implementation | Java |
Created | 2007 |
Helix (Apache Helix)
Topics | |
---|---|
Implementation | Java |
Ignite (Apache Ignite)
Topics | |
---|---|
Data model | |
Implementation | Java |
Created | 2014 |
in-toto
Topics |
---|
Infinispan
Topics | |
---|---|
Data model | |
Implementation | Java |
Created | 2009 |
InfluxDB
Topics | |
---|---|
Data model | |
Query | SQL, HTTP |
Implementation | Go |
Created | 2013 |
Kafka (Apache Kafka)
Topics | |
---|---|
Implementation | Java |
key-value (KV)
Topics |
---|
Kubernetes (Kubes, Kube, K8s)
Topics | |
---|---|
Implementation | Go |
Created | 2014 |
Alternatives |
- awesome-kubernetes: curated bookmarks.
- 10 Most Common Reasons Kubernetes Deployments Fail (Part 1) / Ross Kukulinski, 2017-02: nice intro to Kubernetes, orchestration complexity.
Kudu (Apache Kudu)
Topics | |
---|---|
Data model | |
Implementation | C++ |
Created | 2016 |
LMDB (Lightning Memory-Mapped Database Manager)
Topics | |
---|---|
Query | custom API |
Implementation | C++ |
Created | 2011 |
Alternatives |
- LMDB Technical Information: benchmarks, publications, projects using LMDB
MariaDB
Topics | ||
---|---|---|
Data model | ||
Query | SQL | |
Implementation | C/C++ | |
Stored procedures | SQL | |
Created | 2009 |
Memcached
Topics | |
---|---|
Data model | |
Implementation | C |
Created | 2003 |
Alternatives |
- An SRE’s guide to Memcached for building scalable applications | Lobsters / 2020-03: lively discussion about caching, HA, tradeoffs, features…
microservices
MongoDB
Topics | |
---|---|
Data model | |
Implementation | C++, JavaScript |
Alternatives |
Neo4j
Topics | |
---|---|
Data model | |
Implementation | Java/Scala |
Created | 2007 |
NoSQL
Topics |
---|
OLAP
Topics |
---|
OrbitDB
Topics | |
---|---|
Data model |
- OrbitDB: Peer-to-peer databases for the decentralized web | Hacker News / 2020-05: discussion, theory…
OrientDB
Topics | |
---|---|
Data model | |
Query | partial SQL, Gremlin, REST |
Implementation | Java |
Created | 2010 |
Packer
Topics | |
---|---|
Alternatives |
- Packer, where DevOps begins (slides) / Jeff Hung, 2015-08: overview, with examples.
Percona Server
Topics | |
---|---|
Created | 2006 |
Alternatives |
Pig (Apache Pig)
Topics |
---|
PostgreSQL (Postgres)
Topics | |
---|---|
Data model | |
Isolation | either read committed, repeatable read, or serializable |
Concurrency | MVCC |
Replication | WAL |
Implementation | C |
Stored procedures | SQL, PL/SQL, C |
Created | 1996 |
PouchDB
Topics | |
---|---|
Data model | |
Implementation | JavaScript |
Created | 2010 |
Pravega
Topics | |
---|---|
Implementation | Java |
Presto (PrestoDB)
Topics | |
---|---|
Data model | |
Query | SQL |
Raft (Raft Consensus)
Topics |
---|
RDS (Amazon Relational Database Service, Amazon RDS)
Topics |
---|
Redis (Remote Dictionary Server)
Topics | |
---|---|
Data model | |
Implementation | C |
Created | 2009 |
RethinkDB
Topics | |
---|---|
Data model | |
Query | ReQL |
Implementation | C++ |
Alternatives |
Riak
Topics | |
---|---|
Data model | |
Implementation | Erlang |
Created | 2009 |
RocksDB
Topics | |
---|---|
Data model | |
Implementation | C++ |
Created | 2012 |
Rook
Topics |
---|
- Rook-Ceph Storage Operator
- Rook Storage – Software Defined Storage / 2019-03: detailed introduction.
- CII Best Practices badge
RPC
Topics |
---|
rqlite
Topics | |
---|---|
Query | SQL |
Implementation | Go |
Alternatives |
- Rqlite | Lobsters / 2020-02
Scylla
Topics | |
---|---|
Implementation | C++ |
Created | 2014 |
Alternatives |
serialization
serverless
service mesh
Topics |
---|
ShardingSphere (Apache ShardingSphere)
Topics | |
---|---|
Implementation | Java |
Created | 2018 |
SimpleDB (Amazon SimpleDB)
Topics | |
---|---|
Data model | |
Implementation | Erlang |
SODA (Simple Oracle Document Access)
Topics | |
---|---|
Data model |
Solr (Apache Solr)
Topics | |
---|---|
Data model | |
Implementation | Java |
Created | 2004 |
Stolon
Topics |
---|
- An introduction to stolon / Simone Gotti, 2016-01
- Setting up PostgreSQL on Kubernetes using Stolon / Diane Strutner, 2019-03
storage
Topics |
---|
Swarm (Swarm mode)
Topics | |
---|---|
Alternatives |
- Why I’m leaving Kubernetes for Swarm / Jonathan Kosgei, 2017-02: “Humans should not have to write/read config files”. Discussion.
- docker-swarm-cluster: configurations for combining several tools for creating a Docker Swarm cluster: Swarm Dashboard, Traefik, Portainer, Prometheus, Grafana…
- terraform-docker-swarm-aws: Terraform script to set up a Docker Swarm on AWS .
Tarantool
Topics | |
---|---|
Data model | |
Implementation | C, Lua |
Created | 2008 |
TDengine
Topics |
---|
TiDB
Topics | |
---|---|
Query | SQL |
Implementation | Go |
Created | 2015 |
TiKV
Topics | |
---|---|
Data model | |
Implementation | Rust |
Created | 2016 |
TimescaleDB
Topics | |
---|---|
Query | SQL |
Created | 2017 |
Vitess
Topics | |
---|---|
Implementation | Go |
Created | 2011 |
- CNCF webinar: Transactional microservices with Vitess: coordination without state / Daniel Kozlowski, 2020-10: implementation details of distributed database for 12FA microservices.
YugabyteDB (YugaByte DB)
Topics | |
---|---|
Query | SQL, CQL |
Implementation | C++ |
Created | 2016 |
Alternatives |
- Looking for an alternative to CockroachDB?
- Bringing Truth to Competitive Benchmark Claims: YugabyteDB vs CockroachDB, Part 2 / Karthik Ranganathan, 2020-05
Zenko
Topics | |
---|---|
Implementation | JavaScript |