Sunday, May 19, 2024

The 2023 MAD (Machine Studying, Synthetic Intelligence & Knowledge) Panorama – Matt Turck

It has been lower than 18 months since we printed our final MAD panorama, and it has been stuffed with drama.

Once we left, the information world was booming within the wake of the big Snowflake IPO, with a complete ecosystem of startups organizing round it. 

Since then, after all, public markets crashed, a recessionary economic system appeared and VC funding dried up. An entire technology of information/AI startups has needed to adapt to a brand new actuality.

In the meantime, the previous few months noticed the unmistakable, exponential acceleration of Generative AI, with arguably the formation of a brand new mini-bubble. Past technological progress, it feels that AI has gone mainstream, with a broad group of non-technical individuals all over the world now attending to expertise its energy firsthand.

The rise of information, ML and AI is among the most elementary tendencies in our technology. Its significance goes effectively past the purely technical, with a deep affect on society, politics, geopolitics and ethics.

But it’s a difficult, technical, and quickly evolving world that’s usually complicated even for practitioners within the area. There’s a jungle of acronyms, applied sciences, merchandise and firms on the market which might be laborious to maintain observe of, not to mention grasp:

The annual MAD (Machine Studying, Synthetic Intelligence and Knowledge) panorama is our try at making sense of this vibrant area.  Its normal philosophy, very like our occasion sequence Knowledge Pushed NYC, has been to open supply work that we’d do anyway, and begin a dialog with the group.

So, right here we’re once more, in 2023. That is our ninth annual panorama and “state of the union” of the information and AI ecosystem. Listed below are the prior variations: 2012, 2014, 2016, 2017, 2018, 2019 (Half I and Half II), 2020 and 2021

This annual state of the union publish is organized in 4 elements:

After a lot analysis and energy, we’re proud to current the 2023 model of the MAD panorama. Once I say “we”, I imply slightly group, whose nights might be haunted for months to come back by recollections of transferring tiny logos out and in of crowded little containers on a PDF: Katie Mills, Kevin Zhang and Paolo Campos. Immense because of them. And sure, I meant it once I advised them on the onset “oh, it’s a lightweight venture, perhaps a day or two, it’ll be enjoyable, please signal right here”.

So, right here it’s (cue in drum roll, smoke machine).  The MAD panorama is available in two modes of consumption this 12 months:

PDF (static) model:

<<<<<<<< CLICK HERE FOR PDF VERSION >>>>>>>>

(sure, it’s all very excessive decision, and you’ll simply zoom on each desktop and cellular)

<New!> Interactive model:

As well as, this 12 months for the primary time, we’re leaping head first into what the kids name the “World Vast Internet”, with a totally interactive model of the MAD Panorama that ought to make it enjoyable to discover the assorted classes.  


Notes on the interactive model:

  • Every emblem is clickable – whenever you click on a pop up reveals up on the underside proper nook
  • There’s a “panorama” and a “card” view (see high proper nook)… and likewise, an evening mode!
  • This can be a first model, and we’ll add extra performance ASAP (search, filtering, and many others.)
  • For this interactive model, we partnered with Gotta Go Quick for the app construct and CB Insights for the information that seems within the playing cards.  Many because of each for his or her partnership. 

For all questions and feedback, please e-mail 

Common strategy

First, we’ve made the choice this 12 months once more to preserve each information infrastructure and ML/AI on the identical panorama. One may argue that these two worlds are more and more distinct. Nevertheless, we proceed to imagine that there’s a vital symbiotic relationship between these areas. Knowledge feeds ML/AI fashions. The excellence between an information engineer and a machine studying engineer is commonly fairly fluid. Enterprises have to have a strong information infrastructure in place so as earlier than correctly leveraging ML/AI.

The panorama is constructed kind of on the identical construction as each annual panorama since our first model in 2012. The free logic is to comply with the movement of information, from left to proper – from storing and processing to analyzing to feeding ML/AI fashions and constructing user-facing, AI-driven or data-driven purposes.

This 12 months once more, we’ve saved a separate “open supply” part. It’s all the time been a little bit of an ungainly group as we successfully separate business corporations from the open supply venture they’re usually the primary sponsor of. However equally, we need to seize the fact that for one open supply venture (for instance, Kafka), you’ve got many business corporations and/or distributions (for Kafka – Confluent, Amazon, Aiven, and many others.). Additionally, some open supply initiatives showing within the field are usually not totally business corporations but.

The overwhelming majority of the organizations showing on the MAD panorama are distinctive corporations, with a really massive variety of VC-backed startups. Quite a lot of others are merchandise (comparable to merchandise provided by cloud distributors) or open supply initiatives.

Firm choice

This 12 months, we’ve a complete of 1,416 logos showing on the panorama.   For comparability, there have been 139 in our first model in 2012.

Every year we are saying we will’t presumably match extra corporations on the panorama and every year, by some means, we have to. This comes with the territory of masking one of the explosive areas of expertise.

Nevertheless, this 12 months particularly, we’ve needed to take a extra editorial, opinionated strategy to deciding which corporations make it to the panorama. Regardless of the surging variety of corporations within the class, we’re gone the stage the place we will match almost everybody, so we’ve needed to make decisions.

In prior years, we tended to offer disproportionate illustration to growth-stage corporations, based mostly on funding stage (usually Sequence B-C or later) and ARR (when obtainable), along with all the big incumbents. Nevertheless this 12 months, notably given the explosion of brand name new areas like Generative AI the place most corporations are 1 or 2 years previous, we’ve made the editorial choice to function many extra very younger startups on the panorama.

A few disclaimers:

  • We’re VCs, so we’ve a bias in the direction of startups, though hopefully we’ve finished job masking bigger corporations, cloud vendor choices, open supply and occasional bootstrapped corporations
  • We’re based mostly within the US, so we most likely over-emphasize US startups. We do have robust illustration of European and Israeli startups on the MAD panorama. Nevertheless, whereas we’ve a couple of Chinese language corporations, we most likely under-emphasize the Asian market in addition to Latin America and Africa (which simply had a powerful information/AI startup success with the acquisition of Tunisia-born Instadeep by BioNTech for $650M)


One of many tougher elements of the method is categorization – particularly, what to do when an organization’s product providing straddles two or extra areas. It’s changing into a extra salient difficulty yearly, as many startups progressively increase their providing, a development we talk about in “Half III – Knowledge Infrastructure”.

Equally, it will be simply untenable to place each startup in a number of containers on this already overcrowded panorama.

Due to this fact, our normal strategy has been to categorize an organization based mostly on its core providing, or what it’s largely recognized for.  In consequence, startups usually seem in just one field, even when they do greater than only one factor.

We make exceptions for the cloud hyperscalers (many AWS, Azure and GCP merchandise throughout the assorted containers), in addition to some public corporations (e.g. Datadog) or very massive personal corporations (e.g., Databricks).

What’s new this 12 months

Important modifications in “Infrastructure”:

  • We (lastly) killed the Hadoop field, to replicate the gradual disappearance of the OG Massive Knowledge expertise – the tip of an period! We had determined to maintain it one final time within the MAD 2021 panorama to replicate the prevailing footprint. Hadoop is definitely not useless, and elements of the Hadoop ecosystem are nonetheless being actively used (e.g., Hive) – see The Hadoop Dialog Is Now About What’s Subsequent . But it surely has declined sufficient that we determined to merge the assorted distributors and merchandise supporting Hadoop into Knowledge Lakes (and saved Hadoop and different associated initiatives in our Open Supply class).
  • Talking of information lakes, we rebranded that field to “Knowledge Lakes / Lakehouses” to replicate the lakehouse development (which we had mentioned within the 2021 MAD panorama)
  • Within the ever evolving world of databases, we created three new subcategories:
    • “GPU-accelerated Databases” (used for streaming information and real-time machine studying)
    • “Vector Databases” (used for unstructured information to energy AI purposes, see What’s a Vector Database?)
    • “Database Abstraction”, a considerably amorphous time period meant to seize the emergence of a brand new group of serverless databases that summary away a whole lot of the complexity concerned in managing and configuring a database. For extra, right here’s overview: 2023 State of Databases for Serverless & Edge (mentions a variety of distributors, greater than we may match within the field)
  • We thought of including an Embedded Database” class with DuckDB for OLAP, KuzuDB for Graph, SQLite for RDBMS and Chroma for search however needed to make laborious decisions given restricted actual property – perhaps subsequent 12 months.
  • We added a “Knowledge Orchestration” field to replicate that rise of a number of business distributors in that area (we already had a “Knowledge Orchestration” field in “Open Supply” in MAD 2021)
  • We merged two subcategories “Knowledge observability” and “Knowledge High quality” into only one field, to replicate the truth that corporations within the area, whereas generally coming from completely different angles, are more and more overlapping – a sign that the class could also be ripe for consolidation.
  • We created a new “Absolutely Managed” information infrastructure subcategory. This displays the emergence of startups that summary away the complexity of sewing collectively a sequence of information merchandise (see our ideas on the Fashionable Knowledge Stack in Half III), saving their clients time, not simply on the technical entrance, but in addition on contract negotiation, funds, and many others.

Important modifications in “Analytics”:

  • For now, we killed the “Metrics Retailer” subcategory we had created within the 2021 MAD panorama. The thought was that there was a lacking piece within the trendy information stack. The necessity for the performance definitely stays, however it’s unclear whether or not there’s sufficient there for a separate subcategory.  Early entrants within the area quickly developed: Supergrain pivoted, Hint* constructed a complete layer of analytics on high of its metrics retailer, and Rework was just lately acquired by dbt Labs. 
  • We created a “Buyer Knowledge Platform” field, as this subcategory, lengthy within the making, has been heating up.
  • On the danger of being “very 2022”, we created a “Crypto/web3 Analytics” field — we proceed to imagine there are alternatives to construct essential corporations within the area.

Important modifications in “Machine Studying / Synthetic Intelligence”:

  • In our 2021 MAD panorama, we had damaged down “MLOps” into a number of subcategories – “Mannequin Constructing”, “Characteristic Shops” and “Deployment and Manufacturing”. On this 12 months’s MAD, we’ve merged the whole lot again into one huge MLOps field. This displays the fact that many distributors’ choices within the area are actually considerably overlapping – one other class that’s ripe for consolidation.
  • We nearly created a brand new “LLMOps” class subsequent to MLOps to replicate the emergence of a brand new group of startups targeted on the particular infrastructure wants for big language fashions. However the variety of corporations there (no less than that we’re conscious of) continues to be too small and people corporations actually simply bought began. 
  • We renamed “Horizontal AI” to “Horizontal AI / AGI” to replicate the emergence of an entire new group of research-oriented outfits, lots of which brazenly state Synthetic Common Intelligence as their final aim.
  • We created a “Closed Supply Fashions” field, to replicate the unmistakable explosion of latest fashions during the last 12 months, particularly within the subject of Generative AI. We’ve additionally added a brand new field in “Open Supply” to seize the open supply fashions.
  • We added an “Edge AI” class – not a brand new matter, however there appears to be acceleration within the area

Important modifications in “Purposes”:

  • We created a brand new “Purposes/Horizontal” class, with subcategories comparable to code, textual content, picture, video, and many others. The brand new field captures the explosion of latest Generative AI startups over the previous few months. In fact, lots of these corporations are thin-layers on high of GPT and will or is probably not round within the subsequent few years, however we imagine it’s a essentially new essential class and needed to replicate it on the 2023 MAD panorama. Notice that there are a couple of Generative AI startups talked about in “Purposes/Enterprise” as effectively.
  • To be able to make room for this new class:
    • We deleted the “Safety” field in “Purposes/Enterprise”. We made this editorial choice as a result of, at this level, nearly each one of many 1000’s of safety startups on the market use ML/AI, and we may commit a complete panorama to them.
    • We trimmed down the “Purposes/Trade” field. Particularly, as many bigger corporations in areas like finance, well being or industrial have constructed some degree ML/AI into their product providing, we’ve made the editorial choice to focus totally on “AI-first” corporations in these areas.

Different noteworthy modifications:

  • We added a brand new ESG information subcategory to “Knowledge Sources & APIs” on the backside, to replicate its rising (if generally controversial) significance.

We significantly expanded our “Knowledge Companies” class and rebranded it “Knowledge & AI Consulting”, to replicate the rising significance of consulting providers to assist clients going through a fancy ecosystem, in addition to the truth that some pure-play consulting retailers are beginning to attain early scale.


Related Articles


Please enter your comment!
Please enter your name here

Latest Articles