Why Scalability is the Silent Killer of AI (And How Exadata Solves It)

Many of you knows that the landscape of enterprise data management is changing. Relational databases have been key for business operations, managing structured data with consistency and best performance. Then came the era of unstructured data, big data, and now, the rise of artificial intelligence, especially Large Language Models (LLMs).

This evolution brought with it a new set of challenges: data fragmentation across different systems, the complexity of integrating specialized vector databases, increased latency due to data movement, and (for sure!) the security risks of maintaining data silos.

We have witnessed this fragmentation firsthand. We’ve seen architectures become increasingly complex, with data shuttling between OLTP systems, data warehouses, data lakes, and now, specialized vector stores, just to power AI applications. This multi-system approach, while enabling certain AI capabilities, introduces inherent inconsistencies, delays, and a significant operationall burden.

The promise of AI (particularly agentic AI that can make real time decisions) demands more. It requires data that is not only fresh and accurate but also immediately accessible within a unified, secure, and scalable platform.

This is the pain point Oracle AI Database 26ai, especially when deployed on Oracle Exadata, is design to address. It’s about eliminating the need to deploy and integrate separate systems for transactional, analytical, or vector processing, consolidating these workloads into a single, converged database engine, running on the best escalable database-designed infrastructure.

26ai database is specifically designed for AI

Oracle has integrated AI capabilities directly into the oracle AI Database 26ai kernel. The key to this integration for AI workloads is AI Vector Search. This feature allows semantic search on unstructured data to be combined with relational search on business data within a single, unified system. This means you no longer need to extract, transform, and load (ETL) your data into separate vector databases, eliminating data fragmentation and its associated complexities. 26ai introduces a native VECTOR data type, enabling direct storage of high-dimensional vector embeddings within table columns. These vectors are numerical representations that capture the semantic meaning of various data types like text, images, audio, or video. Storing them alongside your traditional business data in a single table simplifies your data model and streamlines queries.

To accelerate similarity searches on these vectors, 26ai provides three specialized vector index types:

Hierarchical Navigable Small World (HNSW) Indexes: Graph-based indexes, highly efficient for searches on large datasets, particularly suited for scenarios requiring high precision and speed. These indexes are designed for scenarios where speed and precision are essential. Because they ideally reside in memory (within the dedicated vector_pool separate from the buffer cache) they are the best choice when the dataset fits in RAM and you need milliseconds level latency for pure similarity searches. They are the “Formula 1” option.
Inverted File Flat – IVF Indexes (a type of Neighbor Partition vector index): These offer a balance between speed and memory utilization, optimized for disk and providing good performance for massive datasets. When you are dealing with massive datasets that don’t fit reasonably into memory, IVF indexes are your best friend. They partition vectors into clusters and are optimized for disk storage. On Exadata, these are particularly powerful because they allow AI Smart Scan (we will talk about this later…) to offload the vector processing directly to the storage servers, freeing up your database compute resources. They are the “workhorse” option.
Hybrid Vector Indexes (HVI): These combine multiple indexing strategies to combine vector and non-vector data efficiently. They are the ideal choice when your application requires combined text and vector search with unified relevance scoring, or when you need to manage the index through the DBMS_SEARCH package for complex, multi-modal search pipelines.

Note: Not an index type per se, but we can also mention “Neighbor Partitions with the INCLUDE Clause” strategy here – While HVIs are great for complex scoring, for pure raw speed in standard hybrid queries (e.g., “find similar vector + filter by price”), Exadata offers a superior architectural approach. With Exadata System Software 25.2, you can create a standard IVF index with the INCLUDE clause. This stores scalar columns (like price or zipcode) directly inside the vector index structure , eliminating the expensive “Table Access by Rowid” operation and boosting query performance by up to 3x. More about this later.

Warp Speed? Exadata is your AI “Enterprise”

Exadata introduces AI Smart Scan for AI Vector Search. This is a family of Exadata specific optimizations designed to improve the performance of AI vector query operations by orders of magnitude. It transparently offloads vector processing from the database servers to the storage servers, reducing network traffic and freeing up database server CPUs.

This offloading includes:

Vector Distance Computation and Top-K Filtering: The VECTOR_DISTANCE() SQL function is pushed down to the storage servers. Each storage server independently tracks its “top-k” (closest) vectors, performing efficient filtering at the data source. This can lead to 30 times faster AI workloads.
Adaptive Top-K Filtering: As AI Smart Scan reads data chunks, it maintains a “running top-k” list, sending back only vectors closer than those already found. This can achieve up to 4.7 times more data filtering.
Vector Distance Projection: If the query needs the distance value, AI Smart Scan sends the computed distance as a virtual column rather than the entire vector, accelerating queries by up to 4.6 times.
Included Columns in Neighbor Partition Vector Indexes (Exadata System Software 25.2): This critical enhancement allows you to include non-vector columns (e.g., price, zipcode) directly within IVF indexes. This eliminates the need for secondary ROWID-based lookups, speeding up hybrid queries by up to 3x.

Ready to test it?

Simple hybrid scenario example

Imagine a real estate application where users want to find houses not just by price and location, but also by visual similarity to a preferred home. This is a perfect use case for hybrid vector search.

Step 1: The Database Schema

First, we define a table to store our house listings, including a VECTOR column for image embeddings:

CREATE TABLE houses (
    id NUMBER GENERATED ALWAYS AS IDENTITY,
    zipcode NUMBER,
    price NUMBER,
    description CLOB,
    house_photo BLOB, 
    house_vector VECTOR(1024, FLOAT32)
);

Here, house_vector is defined with 1024 dimensions and FLOAT32 precision.

Step 2: The Index Strategy (The Exadata Advantage)

To accelerate similarity searches, we create a vector index. If we create a standard index, filtering on relational data (like price) might still incur additional I/O on the base table. However, leveraging the INCLUDE clause in Exadata System Software 25.2, we can optimize this to avoid the TABLE ACCESS BY USER ROWID bottleneck:

CREATE VECTOR INDEX house_idx ON houses(house_vector)
INCLUDE (price, zipcode)
ORGANIZATION NEIGHBOR PARTITIONS;

With this index, a query combining vector similarity and relational predicates runs incredibly fast because price and zipcode are directly available in the same index.

SELECT price, zipcode
FROM houses t
WHERE price < 700000 AND zipcode = 33005
ORDER BY VECTOR_DISTANCE(house_vector, :query_vector)
FETCH APPROX FIRST 10 ROWS ONLY;

Step 3: Checking the plan

Now that we have our data and our Exadata optimized index, we execute the hybrid query. We are looking for houses similar to our input image (vector search) but strictly filtered by price and location (relational search).

In a traditional setup, or without the INCLUDE clause, the execution plan would show a TABLE ACCESS BY USER ROWID (“explicit rowid provided”) step. This means that for every candidate vector found in the index, the database would have to make an expensive trip back to the base HOUSES table just to check the price and zipcode.

However, because we used the INCLUDE clause in Step 2, our execution plan on Exadata tells a different story. The optimizer realizes that price and zipcode are already resident in the vector index.

If we look at the execution plan for our optimized setup, the TABLE ACCESS BY USER ROWID is completely gone:

----------------------------------------------------------------------------------------------------------
| Id | Operation | Name |
----------------------------------------------------------------------------------------------------------
...
| 11 | TABLE ACCESS STORAGE FULL | VECTOR$VIDX_IVF..._CENTROIDS |
| 12 | PARTITION LIST ITERATOR | |
|* 13 | TABLE ACCESS STORAGE FULL | VECTOR$VIDX_IVF..._CENTROID_PARTITIONS |
----------------------------------------------------------------------------------------------------------

The absence of the base table access confirms that AI Smart Scan is handling the entire operation—vector similarity and relational filtering—in a single pass at the storage layer.

Wrapping up

Ultimately, the industry is littered with promising AI initiatives that failed to survive the leap from a controlled PoC to the reality of massive scale. Scalability is the silent killer of AI projects. If you are building a prototype, any tool might suffice; but if you need a system engineered to thrive under the crushing weight of real-world enterprise AI workloads, Oracle AI Database 26ai on Exadata stands alone.

While the database natively handles the complexity of vector embeddings, it is the Exadata System Software that transforms this capability into extreme performance. The synergy here is unmatched by any standalone vector database running on commodity hardware. By leveraging AI Smart Scan, we aren’t just indexing data; we are pushing the heavy lifting of vector distance computations and Top-K filtering down to the storage layer.

This architecture does not simply make queries faster; it changes the physics of the workload. Processing vector operations directly on the Exadata storage servers and sending only the relevant results back to the database translates into a performance that is orders of magnitude ahead of the competition. Just give it a try.

Why Scalability is the Silent Killer of AI (And How Exadata Solves It)

Share this: