// momentum

tracking since W47

// why it's here

  • Active: last commit today
  • Health 99/100 with only 22 stars
  • 22 forks for 22 stars: people build on it

// how to help

// readme

Spark-Milvus

Read and write Milvus Collection data with Apache Spark.

Spark-Milvus brings Milvus Collection data into Spark DataFrames for analysis, data cleaning and batch updates using Spark SQL, Scala or PySpark.

Why Spark-Milvus

Milvus powers online vector search. Spark-Milvus makes the same Collection data available to Spark for offline processing, connecting retrieval and data preparation workflows.

With Spark-Milvus, you can:

  • Analyze Collection data: Join it with other data sources, inspect field values and measure data quality using familiar Spark operations.
  • Update existing fields in batches: Clean text, standardize metadata or compute replacement values, then write the results to Milvus data files.

For example, Milvus can serve similar-product searches from a product Collection. Spark can read that Collection, join it with a product catalog to identify inconsistent descriptions, and write back corrected values. Your application controls when those updates are submitted to Milvus and become visible to online queries.

Read the full README on GitHub โ†—