Cloud Whatis

Dictionary of cloud native ecosystem. (Work in progress; last updated: 2021-04-28.)

Format

  1. I am considering converting this into an app, adding features. Blog post is a quick-and-dirty attempt at releasing "early" — this is a small sample, and mostly drafts — I've already nearly 500 entries. Look at all this red ink. Crazy landscape.
  2. Still thinking about RDFa, microformats… (Exposes how Q&D I did data modeling for this. And how hard taxonomies are. ;o)
  3. Not showing timestamps per entry, yet.

Classifications

  1. My attempt at a more (semi-)structured knowledge base.
  2. Also, to normalize (standardize?) and de-hype terminology, characterizations.
  3. … turned out to be an impossible task.
  4. Recommended resources:
    1. Detailed classifications of many databases done by dbdb — are linked as canonical.
    2. CNCF SIG Storage whitepaper gives a rudimentary taxonomy.
    3. The Dictionary of Cloud-Native App Delivery

Dictionary

(Reminder: search with Ctrl+F, or "/" on FF. ;o)

Major topics: NoSQL, Kubernetes, continuous, microservices, storage, Hadoop

    1. h
    2. doc

    ACM (AWS Certificate Manager)

    Manage SSL/TLS certificates on AWS resources. WUI. API. CLI.
    Topics
    1. h
    2. c
    3. W

    ActiveMQ (Apache ActiveMQ)

    Message broker.
    Topics
    ImplementationJava
    Created2004
    1. h
    2. c
    3. doc
    4. img
    5. W
    6. SO
    7. Tw

    Airflow (Apache Airflow)

    Workflow orchestration framework, pipelines (workloads) automation/scheduling.
    Topics
    ImplementationPython
    1. c
    2. dbdb

    Akutan (formerly Beam)

    Distributed knowledge graph store — aka RDF or triple store.
    Topics
    Data model
    ImplementationGo
    Created2019
    1. h
    2. c
    3. doc
    4. CNCF
    5. W
    6. Tw

    Alluxio (formerly Tachyon)

    Virtual distributed file system, used as data orchestration layer, provides multiple data access interfaces, unified mounting name-space, and hierarchical cache, between cloud storage and, eg, Kubernetes workloads, Presto, Spark, Hive, Kubeflow. Multi-cloud.
    $ ./bin/alluxio fs mount alluxio://master:port/nfs /mnt/nfs
    Topics
    ImplementationJava
    Scalepetabyte
    Created2013
    1. CSI implementation to provide POSIX access to Alluxio in containerized environments such as Kubernetes
    2. Using Alluxio to Optimize and Improve Performance of Kubernetes-Based Deep Learning in the Cloud / 2020-05: whitepaper (PDF), detailed usecase, performance analysis.
    3. Improving Presto Latencies with Alluxio Data Caching / Rohit Jain, et al, 2020-06: architecture, performance benchmarks.
    1. W

    Ambari (Apache Ambari)

    Hadoop cluster provisioning, monitoring, management.
    Topics
    1. W

    AMQP (Advanced Message Queuing Protocol)

    Application layer protocol for message oriented middleware: queuing, routing, point-to-point, publish-subscribe. Wire-level protocol: specifies binary data format.
    Topics
    Created2003
    Alternatives
    1. W

    Ant (Apache Ant)

    Make-like build tool.
    Topics
    ImplementationJava
    Created2000
    1. c

    AnyRPC

    Multi-protocol remote procedure call C++ library; supports JSON-RPC, XML-RPC, MessagePack RPC.
    Topics
    ImplementationC++
    1. h
    2. W

    Apache Shiro

    Authn/z framework.
    Topics
    ImplementationJava
    Created2004
    1. h
    2. c

    Artifactory

    Container image registry, artifact repository.
    Topics
    ImplementationRuby
    1. h
    2. dbdb
    3. W

    Aurora (Amazon Aurora)

    AWS managed (by RDS) RDBMS compatible with MySQL and PostgreSQL. Distributed, fault-tolerant, self-healing, low-latency read replicas, point-in-time recovery, continuous backup to S3, and replication across three Availability Zones.
    Topics
    Created2014
    Alternatives
    1. c
    2. Tw

    Baucis

    Framework for REST APIs, based on Mongoose/MongoDB, Express/Node. HTTP methods for CRUD, full text search.
    Topics
    ImplementationJavaScript
    1. h

    Bitbucket Pipelines

    Bitbucket's integrated CI/CD.
    Topics
    1. h
    2. c

    BooPickle

    Serialization format, binary, efficient. Scala.
    Topics
    ImplementationScala
    1. h
    2. c
    3. W

    Brooklyn (Apache Brooklyn)

    Framework for modeling, deploying and managing distributed applications defined using declarative YAML blueprints.
    Topics
    ImplementationJava
    Created2012
    1. h
    2. c
    3. W
    4. Tw

    Buildr (Apache Buildr)

    Build tool for Java-based applications.
    Topics
    ImplementationRuby
    1. h
    2. W

    Calcite (Apache Calcite)

    SQL parser, relational algebra API, query planning engine.
    Topics
    1. W

    Cayenne (Apache Cayenne)

    ORM framework.
    Topics
    1. h
    2. W

    CBOR (Concise Binary Object Representation)

    Serialization format, binary, schemaless. IETF RFC 8949, et al: signing, encryption, web tokens, data definition language.
    Topics
    1. RFC 8949
    1. h
    2. doc

    CloudFormation (AWS CloudFormation, CFN)

    AWS provisioning templates, *IaC, YAML or JSON.
    Topics
    1. h
    2. c

    Cluster API (CAPI)

    Declarative APIs and tooling to simplify provisioning, upgrading, and operating multiple Kubernetes clusters. Kubernetes sub-project started by CNCF SIG Cluster Lifecycle.
    Topics
    1. c

    CNI (Container Network Interface)

    Networking for Linux containers.
    Topics
    1. h
    2. c
    3. doc
    4. CNCF
    5. Tw

    Concourse

    Pipelines automation, containerized. YAML configurations.
    Topics
    ImplementationGo, Elm
    1. h
    2. c
    3. doc

    Conjur (CyberArk Conjur)

    Secrets management for microservices. RBAC. CLI, REST. Uses Nginx, Postgres.
    Topics
    ImplementationRuby
    1. CNCF

    continuous (CI/CD: continuous integration/​delivery/​deployment)

    Automation…
    1. Continuous Delivery Foundation (CDF); @CDeliveryFdn
    2. CD tools drawbacks / discussion on Reddit /r/devops, 2019-04: *Spinnaker, *GitLab, *Helm…
    3. CNCF SIG Application Delivery Charter: scope and topics of the lifecycle of cloud-native applications.
    4. Model of Cloud-Native App Delivery | Google Slides
    1. h
    2. c
    3. doc
    4. img
    5. dbdb
    6. CNCF
    7. W
    8. SO
    9. Tw

    Couchbase (Couchbase Server, formerly Membase)

    Distributed (shared-nothing) database, JSON document oriented, in-memory, swapped to disk, append-only.
    Topics
    Data model
    Indexeshash, B+ tree, full text
    QueryN1QL (SQL extended for JSON)
    ImplementationErlang
    Created2009
    1. h
    2. c
    3. doc
    4. dbdb
    5. W

    CouchDB (Apache CouchDB, Cluster Of Unreliable Commodity Hardware)

    Document database.
    Topics
    Data model
    QueryREST API
    ImplementationErlang
    Created2005

    CRD (Custom Resource Definition)

    Kubernetes API extensions to manage resources other than native objects such as pods and services.
    Topics
    1. Kubernetes docs: Extending Kubernetes: Custom Resources
    2. doc.crds.dev: documentation browser for CRDs.
    1. h
    2. c
    3. CNCF

    CRI-O (neé OCID)

    implementation of the Kubernetes Container Runtime Interface (*CRI) designed to enable the use of Open Container Initiative (OCI) compatible runtime — allows Kublet to use different container runtimes, without needing to recompile Kubernetes.
    Topics
    Created2016
    1. CNCF to host CRI-O
    1. h
    2. doc
    3. W

    Cypher

    Declarative graph query language.
    Topics
    Created2011
    1. c

    Distroless

    Build Docker images without OS stuff, just your app.
    Topics
    ImplementationStarlark (Bazel)
    1. h
    2. c
    3. doc
    4. dbdb

    Dqlite

    Distributed SQLite, embedded (C library), persistent RDBMS, Raft consensus.
    Topics
    QuerySQL
    ImplementationC
    Created2017
    Alternatives
    1. h
    2. c
    3. doc
    4. CNCF
    5. Tw

    Drone

    Container-native CD: uses YAML — a superset of Docker Compose — to define and execute pipelines inside containers.
    Topics
    1. W

    Dynamo

    Highly available distributed key-value storage.
    Topics
    Created2004
    1. h
    2. dbdb
    3. W

    DynamoDB (Amazon DynamoDB)

    AWS key-value and document store, auto-scaling, priced on throughput instead of storage, integration with Hadoop.
    Topics
    Data model
    Created2012
    1. If DynamoDB sucks at sorting/querying… which database should we use? / 2020-03: Reddit discussion of DynamoDB features, strenghts/weaknesses, (general NoSQL) design considerations: use it for big data, not relational queries.
    2. DynamoDB's new autoscaling in action for https://github.com/tj/gh-polls, low traffic either way but at least I don't have to over-provision! / 2017-07, tweet by TJ Holowaychuk
    1. h

    EBS (Amazon Elastic Block Store)

    Block level storage volumes for EC2 instances. Volumes behave like a raw, unformatted, external block devices.
    Topics
    1. h
    2. doc

    EKS (Amazon Elastic Kubernetes Service)

    Managed Kubernetes over EC2.
    Topics
    Created2017
    1. h

    eksctl

    CLI for EKS.
    ImplementationGo
    1. h
    2. doc

    ELB (Elastic Load Balancing)

    AWS ELB provides "three types of load balancers: Application Load Balancers (ALB; layer 7 — HTTP/2, WebSockets), Network Load Balancers (NLB; connection level — layer 4), and Classic Load Balancers (CLB; formerly ELB, 2009-2016)". Distributes incoming application traffic across EC2 instances, containers, and availability zones.
    Topics
    Alternatives
    1. Compare: ALB, NLB, "Classic LB" (formerly ELB)
    1. c
    2. doc
    3. dbdb
    4. W

    Elliptics

    Distributed storage, eventually consistent, distributed hash table (DHT).
    Topics
    Data model
    ImplementationC++, Python, Go
    Created2008
    1. h
    2. c
    3. doc
    4. dbdb
    5. SO
    6. Tw

    Geode (Apache Geode)

    Distributed in-memory data grid supporting caching and data computation.
    Topics
    Data model
    QueryOQL
    ImplementationJava
    Created2002
    1. h
    2. c
    3. doc
    4. W
    5. Tw

    GlusterFS (Red Hat Gluster Storage)

    Network file system, software-defined storage.
    Topics
    1. h
    2. c
    3. CNCF
    4. W
    5. Tw

    gRPC

    RPC framework, uses Protocol Buffers, a language-independent specification, over HTTP/2. Replaces JSON for APIs.
    Topics
    Alternatives
    1. Discussion in a Reddit thread / 2019-05: lots of information and opinions. Some responses seem weird — maybe tongue-in-cheek?
    1. h
    2. W

    Helix (Apache Helix)

    Cluster management framework partitioned and replicated distributed resources.
    Topics
    ImplementationJava
    1. h
    2. c
    3. doc
    4. dbdb
    5. W

    Ignite (Apache Ignite)

    Distributed in-memory key-value store.
    Topics
    Data model
    ImplementationJava
    Created2014
    1. h

    KMS (AWS Key Management Service)

    Creation and control of encryption keys.
    Topics
    1. h
    2. c
    3. CNCF
    4. Tw

    kOps

    Kubernetes Operations: kubectl for clusters.
    Topics
    ImplementationGo
    Created2016
    1. c

    Kryo

    Binary object graph serialization framework for Java.
    Topics
    ImplementationJava
    1. h
    2. c
    3. doc
    4. CNCF
    5. W
    6. Tw

    Kubernetes (K8s)

    Automating deployment, scaling, and management of multi-container applications; orchestration for ephemeral workloads. CNCF's de facto (de jure?) standard.
    Topics
    ImplementationGo
    Created2014
    Alternatives
    1. awesome-kubernetes: curated bookmarks.
    2. 10 Most Common Reasons Kubernetes Deployments Fail (Part 1) / Ross Kukulinski, 2017-02: nice intro to Kubernetes, orchestration complexity.
    3. Best Practices badge
    1. h
    2. c
    3. doc

    Kubestack

    Terraform GitOps framework, infrastructure automation for cloud managed Kubernetes.
    Topics
    1. c
    2. dbdb
    3. W

    LevelDB

    Embedded key-value database, derived from Chromium's Web Storage API library.
    Topics
    Data model
    ImplementationC++
    Created2011
    1. h
    2. c
    3. doc
    4. CNCF

    MetalK8s

    Opinionated K8s distribution, to simplify on-premises deployments.
    Topics
    1. W

    NoSQL

    Databases: no SQL, not relational, not only SQL, NewSQL…
    1. Jepsen: Breaking distributed systems so you don't have to. Clojure library.
    1. h
    2. c
    3. CNCF

    Notary

    Trusted distribution; an implementation of The Update Framework (TUF).
    Topics
    ImplementationGo
    1. h
    2. c
    3. Tw

    OAM (Open Application Model)

    Open standard for defining cloud native apps.
    Topics
    1. h
    2. c
    3. W

    OCI (Open Container Initiative)

    Open standard for containerization. Cf *runC.
    Topics
    Created2015
    1. c

    Orbiter

    Autoscaler for Docker Swarm.
    Topics
    ImplementationGo
    1. h
    2. c

    Photon OS

    Minimal, hardened, Linux distribution optimized for hosting containers, Kubernetes.
    Topics
    1. h
    2. c
    3. doc
    4. dbdb
    5. W
    6. SO
    7. Tw

    PouchDB

    Document database in JavaScript, designed to run in a browser, derived from/inspired by CouchDB. Browser's local storage synchronized, when online, with server-side CouchDB or LevelDB.
    Topics
    Data model
    ImplementationJavaScript
    Created2010
    1. h
    2. c
    3. doc
    4. CNCF
    5. Tw

    Quay (Project Quay)

    Container images registry, build and distribution.
    Topics
    1. h
    2. c
    3. W

    Raft (Raft Consensus)

    Consensus algorithm designed as an alternative to *Paxos.
    Topics
    1. h
    2. doc

    RDS (Amazon Relational Database Service, Amazon RDS)

    Web service that makes it easier to set up, operate, and scale a relational database. Available for, eg, Aurora, PostgreSQL, MySQL, MariaDB, Oracle.
    Topics
    1. h
    2. c
    3. img

    RedisAI (Redis ML)

    Redis module for serving integrated ML models and data.
    Topics
    1. h
    2. c
    3. dbdb
    4. W

    Riak

    Distributed key-value database, pluggable backend store, supports LevelDB.
    Topics
    Data model
    ImplementationErlang
    Created2009
    1. h
    2. c
    3. W
    4. Tw

    RocketMQ (Apache RocketMQ)

    Distributed messaging and streaming platform. AMQP.
    Topics
    ImplementationJava
    Created2012
    1. doc

    SCIM (System for Cross-domain Identity Management)

    Single Sign-On (SSO).
    Topics
    1. h
    2. c
    3. CNCF
    4. Tw

    ShardingSphere (Apache ShardingSphere)

    JDBC middleware/proxy for sharding, distributed transactions. Planned sidecar.
    Topics
    ImplementationJava
    Created2018
    1. h
    2. c

    SODA (Simple Oracle Document Access)

    Set of NoSQL-style APIs to use Oracle database as document store.
    Topics
    Data model
    1. h
    2. c
    3. dbdb
    4. W

    Solr (Apache Solr)

    Full-text search server with a REST-like API. Features include hit highlighting, faceted search, near real-time indexing, dynamic clustering, database integration, and geospatial search. Uses/merged into Lucene.
    Topics
    Data model
    ImplementationJava
    Created2004
    1. doc

    Swarm (Swarm mode)

    Docker includes swarm mode for natively managing a cluster.
    Topics
    Alternatives
    1. Why I'm leaving Kubernetes for Swarm / Jonathan Kosgei, 2017-02: "Humans should not have to write/read config files". Discussion.
    2. docker-swarm-cluster: configurations for combining several tools for creating a Docker Swarm cluster: Swarm Dashboard, Traefik, Portainer, Prometheus, Grafana…
    3. terraform-docker-swarm-aws: Terraform script to set up a Docker Swarm on AWS .
    4. Docker Swarm Rocks
    1. h
    2. c
    3. doc

    Talos OS

    Minimal Linux distribution optimized for Kubernetes. Immutable, hardened: runs in memory from a SquashFS read-only filesystem. No shell, SSH: access via API only.
    Topics
    1. h
    2. c
    3. doc
    4. dbdb

    Tarantool

    Multi-threaded database, stored procedures in Lua/C, partial SQL.
    Topics
    Data model
    ImplementationC, Lua
    Created2008
    1. h
    2. c
    3. CNCF
    4. W

    Travis CI

    Continuous integration tool, SaaS.
    Topics
    ImplementationRuby
    Created2011
    1. h
    2. c
    3. doc
    4. pkg
    5. CNCF
    6. Tw

    Zenko

    Multi-cloud data management, multiple backends, file systems and S3 compatible. Collection of microservices written mostly in JavaScript.
    Topics
    ImplementationJavaScript
    1. h
    2. c
    3. W

    ZeroMQ (ØMQ)

    Asynchronous messaging library. In-memory, no persistence, no delivery guarantees.
    Topics
    ImplementationC++

--
The real world is a special case

Canonical
https://decodecode.net/elitist/cloud-whatis
Tagged
Updated
Created